Nonlinear time-series analysis revisiteda)

Elizabeth Bradleyb)

Department of Computer Science, University of Colorado, Boulder CO 80309-0430 USA andSanta Fe Institute, Santa Fe, New Mexico 87501, USA

Holger Kantzc)

Max Planck Institute for the Physics of Complex Systems, Noethnitzer Str. 38 D 01187 Dresden Germany

In 1980 and 1981, two pioneering papers laid the foundation for what became known as nonlinear time-seriesanalysis: the analysis of observed data—typically univariate—via dynamical systems theory. Based on theconcept of state-space reconstruction, this set of methods allows us to compute characteristic quantities suchas Lyapunov exponents and fractal dimensions, to predict the future course of the time series, and even toreconstruct the equations of motion in some cases. In practice, however, there are a number of issues thatrestrict the power of this approach: whether the signal accurately and thoroughly samples the dynamics,for instance, and whether it contains noise. Moreover, the numerical algorithms that we use to instantiatethese ideas are not perfect; they involve approximations, scale parameters, and finite-precision arithmetic,among other things. Even so, nonlinear time-series analysis has been used to great advantage on thousandsof real and synthetic data sets from a wide variety of systems ranging from roulette wheels to lasers to thehuman heart. Even in cases where the data do not meet the mathematical or algorithmic requirements toassure full topological conjugacy, the results of nonlinear time-series analysis can be helpful in understanding,characterizing, and predicting dynamical systems.

PACS numbers: 05.45.TpKeywords: time series analysis, other things

Nonlinear time-series analysis comprises a setof methods that extract dynamical informationabout the succession of values in a data set. Thisframework relies critically on the concept of re-construction of the state space of the systemfrom which the data are sampled. The founda-tions for this approach were laid around 1980,when deterministic chaos became a popular fieldof research and scientists were looking for evi-dence of chaos in natural and laboratory systems.One of the first—and still most spectacular—applications was the prediction of the path ofa ball on a roulette wheel, which nicely demon-strated the power of these methods. Since then,nonlinear time-series analysis has left this narrowniche and moved into much broader use acrossall branches of science and engineering, as well associal science, the humanities, and beyond.

I. WHY NONLINEAR TIME SERIES ANALYSIS?

The goal of time-series analysis is to learn aboutthe dynamics behind some observed time-ordered data.Early approaches to this employed linear stochasticmodels—more precisely, autoregressive (AR) and mov-ing average (MA) models1. These stationary Gaussian

a)EB thanks the Max-Planck-Institut fur Physik komplexer Sys-teme for hosting the visit during which this paper was written.b)Electronic mail: lizb@cs.colorado.educ)Electronic mail: kantz@pks.mpg.de

stochastic processes are fully characterized by their two-point auto-correlation function

c(τ) =〈(xt − xt+τ )2〉

〈x2t 〉(1)

or by their power spectrum, respectively. There are manydata sets where this type of analysis leads to a good char-acterization, such as temperature anomalies: differencesbetween the daily (maximum, mean, minimum) temper-ature at a given place and the many-year average of thatquantity for the corresponding calendar day. Data ofthis type possess an almost Gaussian distribution with analmost exponentially decaying auto-correlation function;typically the null hypothesis that they are generated byan AR(1) process cannot be rejected easily on the basisof observed data.

Of course, we know that temperatures can be predictedmuch more accurately by high-dimensional physics-basedmodels of the atmosphere than by AR(1) models. Thatscalar temperature data look like AR data comes fromthe projection of dynamics in a high-dimensional statespace onto a single quantity. This illustrates that, de-pending on one’s point of view and one’s access to a sys-tem’s variables, the very same system might appear tohave very different complexity.

As in any other analysis, the choice of a specific time-series analysis method requires justification by some hy-pothesis about the appropriate data model. Time-seriesanalysis is essentially data compression: we compute afew characteristic numbers from a large sample of data.This reduced information can only enhance our knowl-edge about the underlying system if we can interpret it,and it becomes interpretable through the fact that the

arX

iv:1

503.

0749

3v1

[nl

in.C

D]

25

Mar

201

5

2

chosen number has some specific meaning within somemodel framework. If the data do not stem from theappropriate model class, the chosen quantity might notmake much sense, even if we can compute its numericalvalue on the given data set using some numerical algo-rithm. An illustrative example is the computation of themean and the variance of some sample: we know how todo this, but are these two numbers always meaningful?If the hypothesis is well justified that the observed dataare a sample from a Gaussian distribution, then thesenumbers characterize it completely and there is nothingelse to compute. If, on the other hand, the data stemfrom a bimodal distribution, then the (still well defined)mean value is very atypical and the variance is not themost interesting feature.

The collection of ideas and techniques known as nonlin-ear time-series analysis can be extremely effective whenthe data model is deterministic dynamics in some statespace. This analysis framework allows us to solve aninverse problem of considerable complexity: from data,we can infer properties of the invariant measure of somehidden dynamical system. In the best case, we can evendetermine equations of motion. And, if the underlyingsystem is deterministic and low dimensional, this analysisframework brings out the relationships between geometry(fractal dimension), instability (Lyapunov exponents),and unpredictability (K-S entropy), which is a beauti-ful theoretical result from ergodic theory. Of course, theassumption of determinism makes these methods largelyunsuitable for characterizing stochastic aspects of data.Anomalous diffusion, as first observed in Hurst’s studyof time-series data of the river Nile2, is nowadays studiedusing detrended fluctuation analysis3; behavior like thisis a signature of both nonlinearity and non-Gaussianityin the underlying stochastic process.

In this article, we want to describe—without too muchdetail or any attempt at a comprehensive bibliography—the ideas and concepts of nonlinear time-series analysis,to give a fair account of their usefulness, and to offer someperspectives for the future. Readers who want to enterthis subject more deeply should consult one of the manycomprehensive review articles or useful monographs onthis topic, such as4,5.

II. THE BASICS

State-space reconstruction is the foundation of nonlin-ear time-series analysis. This quite remarkable result,which was first proposed by Packard et al. in 1979 and19806,7 and formalized by Takens soon thereafter8, allowsone to reconstruct the full dynamics of a complicatednonlinear system from a single time series, in principle.The reconstruction is not, of course, identical to the inter-nal dynamics, or this procedure would amount to a gen-eral solution to control theory’s observer problem: how toidentify all of the internal state variables of a system andinfer their values from the signals that can be observed.

Even so, these reconstructions—if done right—can stillbe extremely useful because they are guaranteed to betopologically identical to the full dynamics. And sincemany important properties of dynamical systems are in-variant under diffeomorphism, this means that conclu-sions drawn about the reconstructed dynamics also holdfor the true dynamics of the system.

A. Delay-coordinate embedding

The standard strategy for state-space reconstruction isdelay-coordinate embedding, where a series of past val-ues of a single scalar measurement y from a dynamicalsystem are used to form a vector that defines a point ina new space. Specifically, one constructs m-dimensional

reconstruction-space vectors ~R(t) from m time-delayedsamples of the measurements y(t), such that

~R(t) = [y(t), y(t− τ), y(t− 2τ), . . . , y(t− (m− 1)τ)]

An example is shown in Figure 1. Mathematically, onecan equivalently take forward delays instead of backwardones, but for practical purposes (e.g., predictions) it isbetter to obey causality in one’s notation. If τ is verysmall, the m coordinates in each of these vectors arestrongly correlated and so the embedded dynamics lieclose to the main diagonal of the reconstruction space;as τ is increased, that reconstruction ‘unfolds’ off thatsubspace.

The original embedding theorems only require that τbe nonzero and not a multiple of any any orbit’s pe-riod. This is only valid, however, when one is using real-valued arithmetic on an infinite amount of noise-free datafrom perfect sensors. In practice—when noisy, finite-length time-series data and floating-point arithmetic areinvolved—one needs a higher τ to properly unfold thedynamics off the main diagonal. The τ = 1 embeddingin Figure 1, for instance, will be indistinguishable from adiagonal line if its thickness is smaller than the measure-ment noise level. Since improperly unfolded reconstruc-tions are not topologically conjugate to the true dynam-ics, this is a real problem. For this and other reasons,it can be a challenge to estimate good values for the τparameter, as described in more depth in Section II B.

The original embedding theorems also require m > 2dto assure topological conjugacy, where d is the true di-mension of the underlying dynamics. The trajectorycrossings in the two-dimensional projection of the embed-ded Rossler data in Figure 1, for instance, do not exist inthe real attractor, and so the two structures do not havethe same topology. Sauer et al.9 loosened this require-ment to m > 2dA, where dA is the capacity dimensionof the attractor. In practice, however, d is rarely knownand dA cannot be calculated without first embedding thedata. A large number of heuristic methods have beenproposed to work around this quandary. Many of thesemethods are computationally expensive, most of them

3

t ×104

0 0.5 1 1.5 2

x

-15

-10

-5

0

5

10

15

20

FIG. 1. A time series from the Rossler system (top) and a number of delay-coordinate embeddings of that time series withdifferent values of the delay parameter, τ .

require significant interpretation by—and input from—ahuman expert, and all of them break down if one hasa short or noisy time series. These methods, and theirlimitations, are also discussed in Section II B.

There are two other requirements in the delay-coordinate embedding theorems, one of which is implicitin the formula above: that one has evenly spaced valuesof y. Data-acquisition systems do not have perfect timebases, so this can be a problem in practice. An obviousworkaround here is interpolation, but then one is reallystudying a mix of real and interpolated dynamics. Thefinal requirement is that the measurement process thatproduces y is a smooth, generic function on the statespace of the system. This will not be the case, for in-stance, if some event counter in the data-acquisition sys-tem overflows. It can be hard to know whether the mea-surement function satisfies the theoretical requirement;strategies for doing so include changing the sampling fre-quency or measuring a different quantity and then re-peating the analysis. If the results do not change, one canbe more confident that they are correct. Formal proofsof that correctness, of course, are not possible because ofthe nature of real-world data and digital computers.

Multivariate time-series data are useful for other rea-sons besides the corroboration that is afforded via indi-vidual analyses of different components. It is also possi-ble to perform multi-element reconstructions that com-bine the information in those components. In their 1980paper7, Packard et al. conjectured that any m quan-tities that “...uniquely and smoothly label the statesof the attractor” could serve as effective elements of areconstruction-space vector. This powerful idea is usedsurprisingly rarely, even though it is fully supported byall routines of the tisean software package10. With the

improvement of sensor technology, the dynamical analy-sis of multivariate data will likely become more importantin the coming years, as discussed in Section V.

The kind of ‘due diligence’ exercise mentioned above iscritical to the success of any nonlinear time-series anal-ysis task. Data length, noise, nonstationarity, algorithmparameters, and the like have strong effects on the re-sults, and the only way to know whether those effects areat work in one’s results is to repeat the analysis while ma-nipulating the data (downsampling, for instance, or an-alyzing the first and last half of the data set separately)and the analysis parameters—the m and τ values, thealgorithmic scale parameters, etc. If one can also ma-nipulate the experimental parameters, repeated analysescan reveal whether the data are sampled too coarsely intime to capture the details of the dynamics, or for tooshort a period to sample its overall structure.

B. Estimation of embedding parameters

The theoretical requirements on the embeddingparameters—the delay τ and the embedding dimensionm—are, as mentioned in the previous section, quitestraightforward. In practice, however, one does notknow the dimension of the system under study, nor doesone have perfect data or a computer that uses infinite-precision arithmetic. Estimating good value for m andτ in the face of these difficulties is one of the main chal-lenges of delay-coordinate embedding. Dozens of meth-ods for doing so have been developed in the past fewdecades; we will only cover a few representative mem-bers of this set.

In traditional practice, one chooses τ first, most of-

4

ten by computing a statistic that measures the inde-pendence of τ -separated points in the time series. Thefirst zero of the autocorrelation function of the time se-ries, for instance, yields the smallest τ that maximizesthe linear independence of the coordinates of the em-bedding vector; the first minima of the average mutualinformation11 or the correlation sum12,13 occur at τ val-ues that maximize more-general forms of independence.(One wants the smallest τ that is reasonable becausethe reconstructed attractor can fold over on itself as τgrows, causing other problems.) There are also geometry-based strategies for estimating τ by, for example, ex-amining the continuity on the reconstructed attractoror the amount of space that it fills. While there hasbeen some theoretical discussion14 of what constitutesan optimal τ , there are no universal strategies for puttingthose ideas into practice—especially since the process issystem-dependent, and since a τ that works well for onepurpose (e.g., prediction) may not work well for another(e.g., computing dynamical invariants).

After choosing a value for τ , the next step is to es-timate the embedding dimension m. As in the case ofτ , bigger is not necessarily better—since a single noisypoint in the time series will affect m of the points inan m-embedding—so one wants the smallest m that af-fords a topologically correct result. There are two broadfamilies of approaches to this, one based on the falsenear neighbor (FNN) algorithm of Kennel et al.15 andanother that might be termed the “asymptotic invari-ant” approach. In the latter, one embeds the data fora range of dimensions, computes some dynamical invari-ant (e.g., those discussed in Section III), and selects them where the value of that invariant settles down. In anFNN-based method, one embeds the data, computes eachpoint’s near neighbors, increases the embedding dimen-sion, and repeats the near-neighbor calculation. If any ofthe relationships change—i.e., some neighbor in k dimen-sions is no longer a neighbor in k+1 dimensions—that istaken as an indication that the dynamics were not prop-erly unfolded with m = k. Noise also disturbs neighborrelationships, though, and thus can affect the operationof FNN-based algorithms. No member of either familyof methods provides a guarantee, but both offer effectivestrategies for estimating m. Again, it can be very usefulto employ several different methods to corroborate one’sresults.

This two-step process is not the only approach. Ithas been noted that what really matters is the m ∗ τproduct—i.e., how much of the data are spanned by theembedding vector—and thus that estimating the two pa-rameters at the same time, in combination, may be moreeffective16,17. It has also been suggested that one neednot use the same τ across all m coordinates of the embed-ding vector—i.e., that a systematically skewed embed-ding space may correspond better to the true dynamics18.

III. MATHEMATICAL BEAUTY: CHARACTERIZATIONOF THE INVARIANT MEASURE

The invariant measure of a dynamical system can becharacterized in a number of different ways: the frac-tal dimension of the invariant set, for instance, from thepoint of view of state-space geometry, or the Kolmogorov-Sinai (K-S) entropy if one is interested in uncertaintyabout the future of a chaotic trajectory. The stabilitywith respect to infinitesimal perturbations can be quanti-fied by the Lyapunov exponents. The topological equiva-lence guaranteed by the embedding theorems allows all ofthese quantities—and many others not mentioned here—to be determined from the time-series data.

1. Dimension estimates

There is a whole family of fractal dimensions Dq, usu-ally called the Renyi dimensions. Their most intuitivedefinition is through a partitioning of the state space:the number of boxes Nε of size ε needed to cover a frac-tal set with dimension D0 scales with the box size ε asε−D0 . This is an evident generalization of the integer di-mensions, as one can easily verify: a line segment, forinstance, will yield D0 = 1 via this procedure, regardlessof whether the surrounding space has two, three or moredimensions. D0, often called the capacity dimension, isclosely related to the Hausdorff dimension1. For the gen-eralized dimensions, one has to determine the measureon every box from the partition and raise that measureto the power q, with

∑pqi ∝ ε(1−q)Dq for ε → 0 and pi

being the weight on the ith box.Direct application of these box-counting methods to

the points in the reconstructed state space is possible, butinvolves significant memory and processing demands andits results can be very sensitive to data length. A more-efficient, more-robust estimator of fractal dimensions isthe Grassberger-Procaccia correlation sum19. We recallonly the simplest version, which yields D2. Rather thancount boxes that are occupied by data points, one insteadexamines the scaling of the correlation sum as a functionof ε:

C2(m, ε) :=1

2N(N − T )

∑i

∑j<i−T

Θ(ε−||~xi−~xj ||) , (2)

where Θ is the Heaviside step function. C2(m, ε) rep-resents the fraction of pairs of data points in the m-dimensional embedding space whose spatial distance(measured by the Euclidean or maximum norm) issmaller than the scale ε. This number scales as εD2 if

1 There is a prominent exception to this statement: while theHausdorff dimension of the rational numbers is zero—as for anycountable set of isolated points—their capacity dimension is 1because they are dense.

5

m > D220. The parameter T , going back to Theiler21, en-

sures that the temporal spacing between potential pairsof points is large enough to represent an independentlyidentically distributed sample2.

Formally, of course, the dimension of any finite point-set data should be zero. In the limit as ε → 0, methodsthat simply count occupied boxes correctly reflect thisfact. In nonlinear time-series analysis, however, we areinterested in the dimension of the set that is representedby the point-set data. The correlation sum provides anunbiased estimator for that quantity, and one that isaccurate for small ε—unlike the box method, which isstrongly biased towards small D values in this limit22.

There is a conundrum involved in any estimation ofthe dimension of a delay-coordinate embedding, whichis sometimes known as the conflict between redundancyand irrelevancy14. Specifically, in order to assure thatsuccessive elements of a delay vector are independent, thetime lag τ should be sufficiently large. This can, however(as mentioned in the second paragraph of Section II B)‘overfold’ the reconstructed dynamics—especially if theembedding dimension is high. In these situations, it canrequire extremely well-sampled data in order to correctlyresolve the folds and voids in complicated chaotic attrac-tors. One can turn this reasoning around to estimatethe number of data points N needed to estimate the di-mension of a data set; a pessimistic answer to this23 is

N ≈√

100D2eD2h2τ , where h2 is the correlation entropyof the dynamics, τ the time delay of the reconstruction,and eD2h2τ describes the effects of folding in the delayembedding space due to the minimal embedding dimen-sion m > D2. Among other things, this means that thenumber of points needed to estimate the dimension ofchaotic dynamics reconstructed from a scalar time seriesis much larger than in the original state space, where theentropy factor can be ignored and N > 42D2 has beensuggested24.

2. Lyapunov exponents

Dimension estimate have pitfalls and caveats, but theyare quite robust. Estimates of Lyapunov exponents areunstable. A number of creative strategies have been de-veloped for estimating the full set of m Lyapunov expo-nents λk in the m-dimensional embedding space (e.g.,25);there are also many algorithms for estimating λ1, thelargest exponent, alone (e.g.,26,27). Every one of thesealgorithms involves free parameters, however, and theirresults are often extremely sensitive to the values of thoseparameters—as well as to data length, noise, and the like.When working with reconstructed dynamics, one mustalso be aware of the issue of spurious exponents, since the

2 If T is too small, the estimate of D2 might be biased towards toosmall numbers, e.g., by intermittency effects.

number of Lyapunov exponents is equal to the numberof dimensions in the ambient space. Scalar time-seriesdata sampled from D-dimensional dynamical systems aretypically embedded in m dimensions with m > D, andthose dynamics havem Lyapunov exponents. Ideally, onewould like to find D exponents that correspond to thoseof the original dynamics—or at least to identify the m−Dextra ones that are spurious. There is a neat theory thatpredicts the numerical values of these spurious exponentsin lowest-order approximation28, but this cannot usuallybe reproduced in practice due to inaccessability of thesescales29.

3. The Kolmogorov-Sinai entropy

Theoretically, the K-S entropy (rate) hKS can be de-termined via Pesin’s identity30, which states that it isthe sum of the positive Lyapunov exponents. Since spu-rious exponents are hard to identify, though, and caneven be positive, it is difficult to put this into practicein the context of embedded data (or to use the Kaplan-Yorke formula in order to determine the Lyapunov di-mension). Rather, one typically estimates hKS throughrefined partitions, closely following its definition (e.g.,31).The most straightforward implementation of this ap-proach discretizes the space of joint probabilities andsearches for sequences of successive delay vectors in spe-cific sequences of boxes. As in the case of box-countingimplementations of fractal dimension calculations, thiscan lead to underestimation: a sequence that exists inthe underlying dynamics may not be ‘sampled’ by a givenset of observations. In the box-counting implementation,every sequence with estimated probability 0 will system-atically reduce the estimate of the K-S entropy. A wayaround this is to compute the correlation entropy (rate)h2, which can be estimated by the correlation sum. Todo this, one calculates Eq.(2) for a range of dimensionsm that are larger than the assumed minimum for an em-bedding, obtaining hm(ε) = lnC(m, ε) − lnC(m + 1, ε).Ideally, for some range of ε, one should see a convergenceof hm(ε) → h2 for large m3. For a consistency check,one can then go back to Pesin’s identity and comparethe estimate of h2 to the sum of the positive Lyapunovexponents.

IV. WHAT PRACTITIONERS NEED

A precise characterization of the invariant measure isnot the goal of most time-series analyses; moreover, fewreal-world data sets are measured by perfect sensors op-erating on low-dimensional dynamics, which means that

3 ε values above this range lead to underestimation; ε values belowit lead to large fluctuations.

6

a proper determination of, e.g., Lyapunov exponents, isout of reach, anyway. In practice, one typically wants todescribe a signal in some formalized manner, perhaps inorder to discriminate between it and some other signal.Other important tasks include noise reduction, detectionof changes in dynamical properties within a given signal,or prediction of its future values. In all of these situ-ations, nonlinear time-series analysis has something tocontribute.

A. Signal and system characterization

A typical task is to characterize a single time series bya small set of numbers—for the purposes of classification,for instance, or comparison with other time series. Ex-amples include medical diagnostics (is a patient healthyor sick?) or monitoring of machines (is a lathe bearingwearing out?). In these and many other important appli-cations, nonlinear time-series analysis offers a large zooof useful approaches, a few of which we describe below.

1. Surrogate data

In cases where strong evidence for some property ismissing, one must resort to statistical hypothesis test-ing. With a finite data set, one can never prove resultsabout the underlying dynamics; one can only calculateprobabilities that a particular finding is unprobable us-ing a simple null hypothesis. This approach can providesome evidence that a more-complex (nonlinear, chaotic)dynamics is plausible, for instance.

In nonlinear time-series analysis, the test statistics—Lyapunov exponents, entropies, prediction errors, etc.—are complicated and their probability distributions undersimple null hypotheses are typically unknown. Further-more, the “simple” null hypotheses are typically not sosimple. In the face of these challenges, one can proceedas follows. First, one chooses a particular statistical esti-mator (e.g., the violation of time inversion invariance32,which is a nonlinear property). Second, one determinesits value vd on the target data set. Third, one interpretsthat value by comparing it to the distribution of valuesvs obtained from a large number of time series that fulfilla certain null hypothesis (e.g., of AR processes). De-pending on where the computed value vd falls in this vsdistribution, one can compute the probability of obtain-ing that value “by chance.” This provides a confidencelevel by which the null hypothesis can be rejected.

How does one obtain the distribution of the test statis-tics under the null hypothesis? This is where surrogatedata33 enter the game. These are data that share certainproperties of the time series under study and also fulfilla certain null hypothesis. The idea is that if one canproduce a number of such surrogate time series, one cannumerically compute the distribution of the test statisticon the null hypothesis. The critical questions here are

• which properties of the original data should beshared by the surrogates?

• what should the null hypothesis be?

Some of the answers are easy: since insufficient time-series length poses severe problems, the individual surro-gate data sets should have the same length as the seriesunder study. Others are not: ideally, for instance, each ofthese sequences should represent the same marginal prob-ability distribution as the original data. Since a ratherpowerful null model is the class of ARMA models, it isreasonable to require the surrogate data to have the samepower spectrum (more precisely, the same periodogram)as the original data—i.e., that temporal two-point corre-lations are identical. This is very useful when one wantsto test for nonlinearities, which express themselves innontrivial temporal n-point-correlations.

The technical way to create surrogate data withidentical two-point correlations and identical marginaldistribution34 is to Fourier transform one’s original data,randomize the relative Fourier phases, back transform(this creates close-to-Gaussian random numbers withidentical Fourier spectrum), and map the results ontothe original time-series values by rank ordering. Thethird step restores the original marginal distribution butpartly destroys the correlations, so the power spectrumhas to be re-adjusted by Wiener filtering. Some iterationof these steps is generally required until the features ofthe surrogate data converge. See35 for a careful discus-sion of this family of methods.

While surrogate data tests are very useful—and verydifferent than the bootstrapping techniques used in otherdata-analysis fields—there are a number of caveats ofwhich one must be aware when using them. Prominentamong these is the nonstationarity trap: surrogates, byconstruction, are stationary, whereas the original datamay be nonstationary. A difference in test statistics be-tween surrogates and original data, then, might have itsorigin in nonstationarities rather than in nonlinearities.

2. Permutation Entropy

Since the 1950s, entropy has been a well-establishedmeasure of the complexity and predictability of a timeseries36. This is all very well in theory; in practice, how-ever, estimating the entropy of an arbitrary, real-valuedtime series is a significant challenge. The K-S entropy, forinstance, is defined as the supremum of the Shannon en-tropy rates of all partitions37, but not any partition willdo for this computation. There are creative ways to workaround this, as described in Section III 3. The main issueis discretization: these entropy calculations require cat-egorical data—symbols drawn from a finite alphabet—but time-series data are usually real-valued and binningreal-valued data from a dynamical system with anythingother than a generating partition can destroy the corre-spondence between the true and symbolized dynamics38.

7

Permutation entropy39 is an elegant way to workaround this problem. Rather than computing the statis-tics of sequences of categorical values, as in the calcula-tion of K-S and Shannon entropy, permutation entropyconsiders the statistics of ordinal permutations of shortsubsequences of the time series. If (x1, x2, x3) = (9, 1, 7),for example, then its ordinal pattern, φ(x1, x2, x3), is 231since x2 ≤ x3 ≤ x1. The ordinal pattern of the permu-tation (x1, x2, x3) = (9, 7, 1) is 321. To compute the per-mutation entropy, one considers all the permutations πin the set Sn of all n! permutations of order n, determinesthe relative frequency with which they occur in the timeseries, {xt}t=1,...,T :

p(π) =|{t|t ≤ T − n, φ(xt+1, . . . , xt+n) = π}|

T − n+ 1

where | · | is set cardinality, and computes

HPE(n) = −∑π∈Sn

p(π) log2 p(π)

Like many algorithms in nonlinear time-series analysis,this calculation has a free parameter: the length n of thesubsequences used in the calculation. The key consider-ation in choosing it is that the value be large enough toexpose forbidden ordinal patterns but small enough thatreasonable statistics over the ordinals can be gatheredfrom the given time series. When this value is chosenproperly, permutation entropy can be a powerful tool;among other things, it is robust to noise, requires noknowledge of the underlying mechanisms, and is iden-tical to the Shannon entropy for many large classes ofsystems40.

3. Recurrence plots

A recurrence plot41 is a two-dimensional visualizationof a sequential data set—essentially, a graphical repre-sentation of the recurrence matrix of that sequence. Thepixels located at (i, j) and (j, i) on a recurrence plot (RP)are black if the distance between the ith and jth pointsin the time series falls within some threshold corridor

δl < ||~xi − ~xj || < δh

for some appropriate choice of norm, and white other-wise. These plots can be very beautiful, particularly inthe case of chaotic signals; see Figure 2 for an example.(There are also “unthresholded” RPs, which use color-coding schemes to represent a range of distances accord-ing to hue; these are even more striking.)

RPs are useful in that they bring out correlations at allscales in a manner that is obvious to the human eye, andthey are one of the few analysis techniques that work withnonstationary time-series data, but their rich geometricstructure—which, in the case of chaotic signals, is relatedto the unstable periodic orbits in the dynamics42—can

FIG. 2. A signal and its recurrence plot. Reproduced withpermission from Chaos. 2:596 (2002). Copyright 2002 AIPPublishing.

make them hard to interpret. Recurrence quantificationanalysis (RQA)43 defines a number of quantitative met-rics to describe this structure: the percentage of blackpoints on the plot, for example, or the percentage of thoseblack points that are contained in lines parallel to (butexcluding) the main diagonal. RQA has been appliedvery successfully to many different kinds of time-seriesdata, notably from physiological experiments (e.g.,44).An extremely useful review article is45.

4. Network characteristics for time series

Recently, recurrence plots have been interpreted in avery different way, namely as the adjacency matrix of anundirected network46. In this approach, an RP of an N -point time series is converted into a network of N nodes,pairs of which are connected where the corresponding en-tries of the adjacency matrix are non-zero. One can thendetermine numerical values for different network charac-teristics, such as centrality, shortest path length, cluster-ing coefficients, and many more. There are some evidentquestions, the most relevant being about the invarianceof findings under variation of the threshold value δl, sincethis value determines the link density of the network andall network characteristics become trivial in the limit offull connectivity.

8

B. Prediction

Prediction strategies that work with state-space mod-els have a long history—and a rich tradition—in non-linear dynamics. The reconstruction machinery of Sec-tion II plays a critical role in these strategies, as it allowsthem to be brought to bear on the problem of predict-ing a scalar time series47. In 1969, for instance, Lorenzproposed his “Method of Analogues,” which searches theknown state-space trajectory for the nearest neighbor ofa given point and takes that neighbor’s forward path asthe forecast48; not long after the original embedding pa-pers, Pikovsky showed that the Method of Analogues alsoworks in reconstructed state spaces49.

Of course the canonical prediction example in deter-ministic nonlinear dynamics is the roulette work of theChaos Cabal at the University of California at SantaCruz, a project that not only catalyzed a lot of nonlinearscience—including the original embedding paper7—butalso a lot of interest in the field from both scientific andlay communities50 that continues to this day51.

In the decades since Lorenz’s Method of Analogues andthe roulette project, a large number of creative strategieshave been developed to predict the future course of a non-linear dynamical system52. Most of these methods buildsome flavor of local model in ‘patches’ of a reconstructedstate space and then use that model to make the predic-tion of the next point. Early examples include24,53–55.This remains an active area of research and has evenspawned a time-series prediction competition56.

The Method of Analogues is not only applicable todeterministic dynamics. The short-term transition prob-ability density of a Markov process depends only on thecurrent state, which can be approximated by a delayvector. The “futures” of delay vectors from a smallneighborhood can be viewed as a sample of the distri-bution, one time step ahead. This approach has beenused for modeling57 and predicting58 nonlinear stochas-tic processes.

Surprisingly, perfect embeddings are not required forsuccessful predictions. In particular, reconstructions thatdo not satisfy the theoretical requirements on the embed-ding dimension m can give prediction methods enoughtraction to match or even exceed the accuracy of the samemethods working in a full embedding—particularly whenthe data are noisy59. One can then try to optimize, e.g.,the embedding parameters. Of course, overfitting can bean issue in any prediction strategy; one must be carefulnot to fool oneself by over-optimizing a predictor to thegiven data.

C. Noise and filtering

All real-world signals are contaminated by measure-ment noise. Most commonly, noise is treated as an ad-ditive random process on top of the true signal. Someforms of experimental apparatus contaminate the signal

in different ways, however: “shot” noise, for instance,which appears only intermittently, or systematic bias insome measurement device. Regardless of its form, noisecan interfere with nonlinear time-series analysis if it is toolarge—where “too large” depends greatly on the methodthat one wants to use.

Many studies in the literature are concerned withthe fundamental issue of distinguishing chaos and noise(see60 and references therein). This can be a real chal-lenge. Both types of signals exhibit irregular temporalfluctuations, with a fast decay of the auto-correlationfunction, and both are hard to forecast. They differ inthe dynamical origin of these features: chaos is a de-terministic process, noise not. In a deterministic sys-tem, the short-term futures of two almost-identical statesshould be similar; in a pure noise process that is im-probable. But, as mentioned above, noise takes on manyforms. The simplest and most tractable is white noise:sequences of independently identically distributed (iid)random numbers. Their statistical independence, as ex-pressed by the factorization of their joint probability dis-tributions, can be easily identified by statistical tests.

If the noise is not additive, the challenge mounts. Anoise-driven chaotic system—e.g., a nonlinear stochasticdifferential equation—produces something we might callnoise. Mathematically speaking, such a system will, inany delay-coordinate embedding space, generate an in-variant measure whose support has the full state-spacedimension without fractal structure. In such a system,infinitesimally close trajectories will not diverge expo-nentially fast, but rather separate diffusively, at least onshort time scales. Nonetheless, if such dynamical noiseor interactive noise is sufficiently weak, one can still iden-tify and characterize the deterministic properties of thesystem. However, there is often a smooth transition be-tween chaos and noise, leaving the whole issue without aclear resolution.

It is, however, our impression that this issue is over-emphasized. In most time-series applications, it is notmost critical to “distinguish between chaos and noise,”but rather to decide on the complexity of the process:whether is it linear or nonlinear, where it falls on thespectrum between redundancy and irrelevancy, etc. Andthen we are much better off, as there exist quite pow-erful tools for answering these questions (see, e.g., Sec-tions IV A 1 and IV A 2).

Removing noise from a signal can also be a real chal-lenge. Traditional filtering strategies discriminate be-tween signal and noise using some sort of frequencythreshold: e.g., removing all of the high-frequency com-ponents of the signal. In a chaotic signal, where the fre-quency spectrum is broad band, such a scheme will filtersignal out along with the noise61. To be effective, filteringstrategies for nonlinear time-series data must be tailoredto and informed by the unique properties of nonlinear dy-namics. One can, for instance, use the native geometry ofthe stable and unstable manifolds in a chaotic attractor62

or local models of the dynamics on the attractor63,64, to

9

reduce noise. One can also exploit the topology of suchattractors in nonlinear filtering schemes65.

D. Issues and limitations

Nonlinear time-series analysis in the reconstructedstate space is a powerful and useful idea, but it doeshas some practical limitations. These limitations are byno means fatal, but one has to be aware of them in orderto report correct results.

In theory, delay-coordinate embedding is only guar-anteed to work for an infinitely long noise-free observa-tion of a single dynamical system. This poses a numberof problems in practice, beginning with nonstationarity:embedding a time series gathered from a system thatis undergoing bifurcations, for instance, will produce atopological stew of those different attractors. “Invari-ants” computed from such a structure, needless to say,will not accurately describe any of the associated dynam-ical regimes. One can use the tests described at the endof Section II A to determine whether these effects are atwork in one’s results: e.g., repeating the analysis on dif-ferent subsequences of the data and seeing if the resultschange. The recurrence plots described in Section IV A 3can also be helpful in these situations, allowing one toquickly see if different parts of the signal have differentdynamical signatures.

The analysis of different subsequences of a time serieshas many other uses besides detecting nonstationarity,including determining whether or not one has enoughdata to support one’s analysis. The original embed-ding theorems require an infinite amount of data, butlooser bounds have since been established for differentproblems24,66,67. It is important to know and attend tothese limits; a computation of a Lyapunov exponent of afive-dimensional system from a data set that contains 100points, for instance, should probably not be trusted. It isalso important to keep these effects in mind when repeat-ing analyses on subsets of one’s data, since the changesin the results that one wants to use as a diagnostic toolcan simply be the result of short data lengths.

Dimension is a major practical issue for many reasons,not just because it is not known a priori and can be achallenge to estimate. Most of the results cited aboveregarding the data length that is necessary for success innonlinear time-series analysis scale with the dimension ofthe dynamical system—often quite badly. This becomeseven more of a challenge in spatially extended systems,where the state space is high (or even infinite) dimen-sional and the dynamics is spatio-temporal. In cases likethis, the full attractor cannot be reconstructed by delay-coordinate embedding. This can in some cases be circum-vented by exploiting homogeneity of the system, however:if the dynamics is translationally invariant, local dynam-ics can be reconstructed and used for predictions68,69.

Noise effects also scale with dimension, since any noisytime-series point will affect m of the points in an m-

dimensional embedding of those data. The detection andfiltering strategies mentioned in Section IV C can helpwith noise problems, and subsequence analysis can beused to explore whether the data are adequate to supportthe analysis, but in the end there is simply no way aroundnot having enough data.

Delay-coordinate embedding, as formulated at the be-ginning of Section II, requires data that are evenly sam-pled in time. If this is not true, constructing the delay

vector ~R is impossible without interpolation, which intro-duces spurious dynamics into the results. There is, how-ever, an elegant way around this issue if the data consistof discrete events, like the spikes generated by a neuron:one simply embeds the inter-spike intervals70. The ideahere is that if the spikes can be considered to be the re-sult of an integrate-and-fire process, then their spacingis an effective proxy for the integral of the correspondinginternal variable, and that is a wholly justifiable quan-tity to embed. Even without integrate-and-fire dynamics,one can interpret interspike intervals as a specific Poinaremap, which justifies their embedding71. This also appliesto the time series formed by all maxima (all minima) ofthe signal.

Even though for practical purposes it is quite handy,using the same value of τ in between successive elementsof a delay vector may not be optimal. Indeed, using delayvectors of the form y(t), y(t− τ1), y(t− τ1− τ2), ...., y(t−τ1− τ2− . . .− τm−1), with non-negative τi, can introducemore time scales into the reconstruction, which has beenshown to be useful in many situations17,72. Such strate-gies might be also a way to tackle signals from multi-scaledynamics: if there are different time and length scales in-volved, a fixed τ may be too large to resolve the shortones and/or too small to resolve the long ones. This isparticularly evident when embedding a human ECG sig-nal: using standard delay vectors, one can either unfoldthe QRS complex or represent the t-wave as a loop, butnot both4.

V. PERSPECTIVES

When getting involved in time-series analysis some 25years ago, we could not have anticipated the wealth ofdata that would be available in 2015, facilitated by cheapand powerful sensors for all sorts of quantities, data-acquisition systems with sub-microsecond sampling ratesand terabytes of memory, widespread remote-sensingtechnology, and incredible sense/compute power in smalldevices carried by the majority of the population ofEarth. Commercial hardware and software are available

4 Concerening spatial scales, it has been shown73 that spatial dis-tances might play a different role: so called finite-size Lyapunovexponents might detect different strengths of instability of dif-ferent spatial scales.

10

to monitor all kinds of things, from physiological pa-rameters obtained during daily activity by watch-sizedobjects to real-time traffic flows gathered by camerason highways. These data can be used to suggest life-changing health interventions, produce routing sugges-tions to avoid traffic jams that have not yet formed, andthe like.

All this involves data analysis: often, time-series anal-ysis. The bulk of the techniques used in the various aca-demic and commercial communities that are concernedwith this problem—data mining, machine learning, andthe like—are linear and statistical. Analysis techniquesthat accommodate nonlinearity and determinism couldbe an extremely important weapon in this arsenal, butnonlinear time-series analysis is currently underused out-side the field of nonlinear science. (Of course, much ofthis software is proprietary, so one must be careful aboutsuch generalizations; nonlinear time-series analysis mayalready be running on Google’s computers and it wouldbe hard for those outside the company to know.)

There are some serious barriers for the movement ofnonlinear time-series analysis beyond the university desksof physicists and into widespread professional practice,however. Linear techniques have a long history and aretaught in most academic programs. They are compara-tively easy to use and they almost always produce an-swers. Whether or not those answers are correct, ormeaningful, is a serious issue: cf., the discussion in Sec-tion I of the mean of a bimodal distribution. But to acommunity that is familiar with these linear techniques,the notion of learning a whole new methodology—onethat relies on more-complex mathematics and only worksif the data are good and the algorithm parameters are setright—can be daunting. One of us (EB) encountered sig-nificant resistance when attempting to convince the com-puter systems community to attend to nonlinearity andchaos in computer dynamics—an effect that could signif-icantly impact the designs of those systems. Only whenthose effects become apparent and meaningful to thosecommunities will nonlinear time-series analysis becomemore widespread. Another relevant issue here is whetherlow-dimensional deterministic dynamics is a good datamodel for broader use. So the only prediction that wemake here is that nonlinear time-series analysis is stillfar from its culmination point, in terms of application.

What will be the relevant issues concerning themethodology itself? Here we can only speculate. It isevident that nonstationarity is still a major problem andmany of its facets are not fully explored. Change-pointdetection is one of these. Distilling causality relationshipsfrom data is another critical open problem in nonlineartime-series analysis (e.g., couplings in climate science).Will this ever be possible? It is hard to say. On thealgorithmic end of things, the various free parameters—and the sensitivity of the results to their values—are im-portant issues. Will it be possible to design algorithmswhose free parameters can be chosen systematically, viaintuition, or perhaps even automatically? Such devel-

opments would streamline nonlinear time-series analysis,making it an indispensible tool to make sense out of thereal world.

1G. E. P. Box and F. M. Jenkins, Time Series Analysis: Fore-casting and Control (Holden Day, 1976) second edition.

2H. Hurst, “Long-term storage of water reserviors,” Transactionsof the American Society of Civil Engineers 116 (1951).

3C. Peng, S. Buldyrev, S. Havlin, M. Simons, H. Stanley, andA. Goldberger, “Mosaic organization of DNA nucleotides,” Phys-ical Review E 49, 1685 (1994).

4H. Abarbanel, Analysis of Observed Chaotic Data (Springer,1996).

5H. Kantz and T. Schreiber, Nonlinear Time Series Analysis(Cambridge University Press, 2004).

6J. Crutchfield, “Prediction and stability in classical mechanics,”(1979), senior thesis in physics and mathematics, University ofCalifornia, Santa Cruz.

7N. Packard, J. Crutchfield, J. Farmer, and R. Shaw, “Geometryfrom a time series,” Physical Review Letters 45, 712 (1980).

8F. Takens, “Detecting strange attractors in fluid turbulence,” inDynamical Systems and Turbulence, edited by D. Rand and L.-S.Young (Springer, Berlin, 1981) pp. 366–381.

9T. Sauer, J. Yorke, and M. Casdagli, “Embedology,” Journal ofStatistical Physics 65, 579–616 (1991).

10R. Hegger, H. Kantz, and T. Schreiber, “Practical implementa-tion of nonlinear time series methods: The TISEAN package,”Chaos: An Interdisciplinary Journal of Nonlinear Science 9, 413–435 (1999).

11A. Fraser and H. Swinney, “Independent coordinates for strangeattractors from mutual information,” Physical Review A 33,1134–1140 (1986).

12W. Liebert and H. Schuster, “Proper choice of the time delayfor the analysis of chaotic time series,” Physics Letters A 142,107–111 (1989).

13P. Grassberger and I. Procaccia, “Measuring the strangeness ofstrange attractors,” Physica D 9, 189–208 (1983).

14M. Casdagli, S. Eubank, J. Farmer, and J. Gibson, “State spacereconstruction in the presence of noise,” Physica D 51, 52–98(1991).

15M. B. Kennel, R. Brown, and H. D. I. Abarbanel, “Determiningminimum embedding dimension using a geometrical construc-tion,” Physical Review A 45, 3403–3411 (1992).

16W. Liebert, K. Pawelzik, and H. Schuster, “Optimal embed-dings of chaotic attractors from topological considerations,” Eu-rophysics Letters 14, 521 (1991).

17L. Pecora, L. Moniz, J. Nichols, and T. Carroll, “A unified ap-proach to attractor reconstruction,” Chaos: An InterdisciplinaryJournal of Nonlinear Science 17, 013110 (2007).

18P. Grassberger, T. Schreiber, and C. Schaffrath, “Nonlineartime sequence analysis,” International Journal of Bifurcation andChaos 1, 521 (1991).

19P. Grassberger and I. Procaccia, “Measuring the strangeness ofstrange attractors,” Physica D 9, 189 (1983).

20T. Sauer and J. Yorke, “How many delay coordinates do youneed?” International Journal of Bifurcation and Chaos 3, 737(1993).

21J. Theiler, “Spurious dimension from correlation algorithms ap-plied to limited time series data,” Physical Review E 34, 2427(1986).

22P. Grassberger, “Finite sample corrections to entropy and dimen-sion estimates,” Physics Letters A 128, 369 (1988).

23E. Olbrich and H. Kantz, “Inferring chaotic dynamics from timeseries: On which length scale determinism becomes visible,”Physics Letters A 232, 63–69 (1997).

24L. Smith, “Intrinsic limits on dimension calculations,” PhysicsLetters A 133, 283–288 (1988).

25J. Eckmann, S. Oliffson-Kamphorst, D. Ruelle, and S. Ciliberto,“Lyapunov exponents from time series,” Physical Review A 34,4971 (1986).

11

26A. Wolf, J. Swift, H. Swinney, and J. Vastano, “DeterminingLyapunov exponents from time series,” Physica D 16, 285 (1985).

27M. Sano and Y. Sawada, “Measurement of the Lyapunov spec-trum from a chaotic time series,” Physical Review Letters 55,1091 (1985).

28T. Sauer, J. Tempkin, and J. Yorke, “Spurious Lyapunov expo-nents in attractor reconstruction,” Physical Review Letters 81,4341 (1998).

29H.-L. Yang, G. Radons, and H. Kantz, “Covariant Lyapunovvectors from reconstructed dynamics: The geometry behind trueand spurious Lyapunov exponents,” Physical Review Letters109, 244101 (2012).

30Y. Pesin, “Characteristic Lyapunov exponents and smooth er-godic theory,” Russian Mathematical Surveys. 32, 55 (1977).

31H. Schuster and W. Just, Deterministic chaos (Wiley, 2005).32T. Schreiber and A. Schmitz, “Discriminating power of measures

for nonlinearity in a time series,” Physical Review E 55, 5443(1997).

33J. Theiler, S. Eubank, A. Longtin, B. Galdrikian, and J. Farmer,“Testing for nonlinearity in time series: The method of surrogatedata,” Physica D 58, 72–94 (1991).

34T. Schreiber and A. Schmitz, “Improved surrogate data for non-linearity tests,” Physical Review Letters 77, 635 (1996).

35T. Schreiber and A. Schmitz, “Surrogate time series,” Physica D142, 346–382 (2000).

36C. E. Shannon, “Prediction and entropy of printed English,” BellSystem Techical Journal 30, 50–64 (1951).

37H. Petersen, Ergodic Theory (Cambridge University Press, 1989).38D. Lind and B. Marcus, An Introduction to Symbolic Dynamicsand Coding (Cambridge University Press, 1995).

39C. Bandt and B. Pompe, “Permutation entropy: A natural com-plexity measure for time series,” Physical Review Letters 88,174102 (2002).

40J. Amigo, Permutation Complexity in Dynamical Systems: Or-dinal Patterns, Permutation Entropy and All That (Springer,2012).

41J.-P. Eckmann, S. Kamphorst, and D. Ruelle, “Recurrence plotsof dynamical systems,” Europhysics Letters 4, 973–977 (1987).

42E. Bradley and R. Mantilla, “Recurrence plots and unstable pe-riodic orbits,” Chaos: An Interdisciplinary Journal of NonlinearScience 12, 596–600 (2002).

43J. Zbilut and C. Webber, “Embeddings and delays as derivedfrom recurrence quantification analysis,” Physics Letters A 171,199–203 (1992).

44C. Webber and J. Zbilut, “Dynamical assessment of physiologicalsystems and states using recurrence plot strategies,” Journal ofApplied Physiology 76, 965–973 (1994).

45N. Marwan, M. Romano, M. Thiel, and J. Kurths, “Recurrenceplots for the analysis of complex systems,” Physics Reports 438,237 (2007).

46R. Donner, M. Small, J. Donges, N. Marwan, Y. Zou, R. Xi-ang, and J. Kurths, “Recurrence-based time series analysis bymeans of complex network methods,” International Journal ofBifurcation and Chaos 21, 1019–1046 (2011).

47J.-P. Eckmann and D. Ruelle, “Ergodic theory of chaos andstrange attractors,” Reviews of Modern Physics 57, 617 (1985).

48E. Lorenz, “Atmospheric predictability as revealed by naturallyoccurring analogues,” Journal of the Atmospheric Sciences 26,636–646 (1969).

49A. Pikovsky, “Noise filtering in the discrete time dynamical sys-tems,” Soviet Journal of Communications Technology and Elec-tronics 31, 911–914 (1986).

50T. Bass, The Eudaemonic Pie (Penguin, New York, 1992).51M. Small and C. Tse, “Predicting the outcome of roulette,”

Chaos: An Interdisciplinary Journal of Nonlinear Science 22,

033150 (2012).52M. Casdagli and S. Eubank, eds., Nonlinear Modeling and Fore-casting (Addison Wesley, 1992).

53J. Farmer and J. Sidorowich, “Predicting chaotic time series,”Physical Review Letters 59, 845–848 (1987).

54M. Casdagli, “Nonlinear prediction of chaotic time series,” Phys-ica D 35, 335–356 (1989).

55G. Sugihara and R. May, “Nonlinear forecasting as a way of dis-tinguishing chaos from measurement error in time series,” Nature344, 734–741 (1990).

56A. Weigend and N. Gershenfeld, eds., Time Series Prediction:Forecasting the Future and Understanding the Past (Santa FeInstitute Studies in the Sciences of Complexity, Santa Fe, NM,1993).

57F. Paparella, A. Provenzale, L. Smith, C. Taricco, and R. Vio,“Local random analogue prediction of nonlinear processes,”Physics Letters A 235, 233–240 (1997).

58M. Ragwitz and H. Kantz, “Markov models from data by sim-ple nonlinear time series predictors in delay embedding spaces,”Physical Review E 65, 056201 (2002).

59J. Garland and E. Bradley, “Prediction in projection,” (2015),arxiv.org/abs/1503.01678.

60M. Cencini, M. Falcioni, E. Olbrich, H. Kantz, and A. Vulpiani,“Chaos or noise: Difficulties of a distinction,” Physical ReviewE 62, 427 (2002).

61J. Theiler and S. Eubank, “Don’t bleach chaotic data,” Chaos:An Interdisciplinary Journal of Nonlinear Science 3, 771–782(1993).

62J. Farmer and J. Sidorowich, “Exploiting chaos to predict thefuture and reduce noise,” in Evolution, Learning and Cognition(World Scientific, 1988).

63E. Kostelich and J. Yorke, “Noise reduction in dynamical sys-tems,” Physical Review A 38, 1649–1652 (1988).

64P. Grassberger, R. Hegger, H. Kantz, C. Schaffrath, andT. Schreiber, “On noise reduction methods for chaotic data,”Chaos: An Interdisciplinary Journal of Nonlinear Science 3, 127(1993).

65V. Robins, N. Rooney, and E. Bradley, “Topology-based signalseparation,” Chaos: An Interdisciplinary Journal of NonlinearScience 14, 305–316 (2004).

66A. Tsonis, J. Elsner, and K. Georgakakos, “Estimating the di-mension of weather and climate attractors: Important issuesabout the procedure and interpretation,” Journal of the Atmo-spheric Sciences 50, 2549–2555 (1993).

67J.-P. Eckmann and D. Ruelle, “Fundamental limitations for es-timating dimensions and Lyapunov exponents in dynamical sys-tems,” Physica D 56, 185–187 (1992).

68M.Bar, R. Hegger, and H. Kantz, “Fitting partial differentialequations to space-time dynamics,” Physical Review E 59, 337(1999).

69U. Parlitz and C. Merkwirth, “Prediction of spatiotemporal timeseries based on reconstructed local states,” Physical Review Let-ters 84, 1890 (2000).

70T. Sauer, “Interspike interval embedding of chaotic signals,”Chaos: An Interdisciplinary Journal of Nonlinear Science 5, 127(1995).

71R. Hegger and H. Kantz, “Embedding of sequences of time in-tervals,” Europhysics Letters 38, 267–272 (1997).

72D. Holstein and H. Kantz, “Optimal Markov approximations andgeneralized embeddings,” Physical Review E 79, 056202 (2009).

73E. Aurell, G. Boffetta, A. Crisanti, G. Paladin, and A. Vulpi-ani, “Predictability in the large: An extension of the conceptof Lyapunov exponent,” Journal of Physics A: Mathematics andGeneral 30, 1 (1997).