F4: Large Scale Automated Forecasting Using Fractals

transcript

Deepayan Chakrabarti

CIKM 2002 1

-Deepayan Chakrabarti-Christos Faloutsos

CIKM 2002 2

Outline Introduction/Motivation Survey and Lag Plots Exact Problem Formulation Proposed Method

Fractal Dimensions Background Our method

Results Conclusions

CIKM 2002 3

General Problem Definition

Given a time series {xt}, predict its future course, that is, xt+1, xt+2, ...

Value?

CIKM 2002 4

Motivation

• Financial data analysis

• Physiological data, elderly care

• Weather, environmental studies

Traditional fields

Sensor Networks (MEMS, “SmartDust”)• Long / “infinite” series

• No human intervention “black box”

CIKM 2002 5

Results Conclusions

CIKM 2002 6

How to forecast? ARIMA but linearity assumption Neural Networks but large number

of parameters and long training times [Wan/1993, Mozer/1993]

Hidden Markov Models O(N2) in number of nodes N; also fixing N is a problem [Ge+/2000]

Lag Plots

CIKM 2002 7

Lag Plots

4-NNNew Point

Interpolate these…

To get the final prediction

Q0: Interpolation Method

Q1: Lag = ?

Q2: K = ?

CIKM 2002 8

Q0: InterpolationUsing SVD (state of the art) [Sauer/1993]

CIKM 2002 9

Why Lag Plots? Based on the “Takens’ Theorem”

[Takens/1981] which says that delay vectors

can be used for predictive purposes

CIKM 2002 10

Inside TheoryExample: Lotka-Volterra equations

ΔH/Δt = rH – aH*P ΔP/Δt = bH*P – mP

H is density of preyP is density of predators

Suppose only H(t) is observed. Internal state is (H,P).

CIKM 2002 11

Results Conclusions

CIKM 2002 12

Problem at hand Given {x1, x2, …, xN} Automatically set parameters

- L(opt) (from Q1) - k(opt) (from Q2)

in Linear time on N to minimise Normalized Mean

Squared Error (NMSE) of forecasting

CIKM 2002 13

Previous work/Alternatives Manual Setting : BUT infeasible

[Sauer/1992] CrossValidation : BUT Slow; leave-

one-out crossvalidation ~ O(N2logN) or more

“False Nearest Neighbors” : BUT Unstable [Abarbanel/1996]

CIKM 2002 14

Results Conclusions

CIKM 2002 15

Intuition

X(t-1)

The Logistic Parabola xt = axt-1(1-xt-1) + noise

Intrinsic Dimensionality≈ Degrees of Freedom≈ Information about Xt given Xt-1

CIKM 2002 16

Intuition

x(t-1)

x(t-2)

x(t-2) x(t-1)

x(t-1)

CIKM 2002 17

Intuition To find L(opt):

Go further back in time (ie., consider Xt-2, Xt-3 and so on)

Till there is no more information gained about Xt

CIKM 2002 18

Results Conclusions

CIKM 2002 19

Fractal Dimensions FD = intrinsic dimensionality

“Embedding” dimensionality = 3

Intrinsic dimensionality = 1

CIKM 2002 20

Fractal DimensionsFD = intrinsic dimensionality

[Belussi/1995]

log(r)

log( # pairs)

Points to note:• FD can be a non-integer• There are fast methods to compute it

CIKM 2002 21

Results Conclusions

CIKM 2002 22

Q1: Finding L(opt) Use Fractal Dimensions

to find the optimal lag length L(opt)

Lag (L)

epsilon

L(opt)

CIKM 2002 23

Q2: Finding k(opt) To find k(opt)

• Conjecture: k(opt) ~ O(f)

We choose k(opt) = 2*f + 1

CIKM 2002 24

Results Conclusions

CIKM 2002 25

Datasets Logistic Parabola:

xt = axt-1(1-xt-1) + noise Models population of flies [R. May/1976]

CIKM 2002 26

Datasets Logistic Parabola:

xt = axt-1(1-xt-1) + noise Models population of flies [R. May/1976]

LORENZ: Models convection currents in the air

CIKM 2002 27

Datasets

Error NMSE = ∑(predicted-true)2/σ2

Logistic Parabola: xt = axt-1(1-xt-1) + noise Models population of flies [R. May/1976]

LORENZ: Models convection currents in the air

LASER: fluctuations in a Laser over time (from the Santa Fe Time Series Competition, 1992)

CIKM 2002 28

Logistic Parabola

• FD vs L plot flattens out

• L(opt) = 1

Timesteps

ValueLag

CIKM 2002 29

Logistic Parabola

Timesteps

Our Prediction from here

CIKM 2002 30

Logistic Parabola

Timesteps

Comparison of prediction to correct values

CIKM 2002 31

Logistic Parabola

Our L(opt) = 1, which exactly minimizes NMSE

CIKM 2002 32

LORENZ

• L(opt) = 5

Timesteps

CIKM 2002 33

LORENZValue

Timesteps

Our Prediction from here

CIKM 2002 34

LORENZ

Timesteps

CIKM 2002 35

LORENZL(opt) = 5

Also NMSE is optimal at Lag = 5

CIKM 2002 36

• L(opt) = 7

Timesteps

CIKM 2002 37

Timesteps

Our Prediction starts here

CIKM 2002 38

Timesteps

CIKM 2002 39

LaserL(opt) = 7

Corresponding NMSE is close to optimal

CIKM 2002 40

Speed and Scalability Preprocessin

g is linear in N

Proportional to time taken to calculate FD

CIKM 2002 41

Results Conclusions

CIKM 2002 42

ConclusionsOur Method:

Automatically set parameters

L(opt) (answers Q1)

k(opt) (answers Q2)

In linear time on N

CIKM 2002 43

Conclusions Black-box non-linear time series

forecasting Fractal Dimensions give a fast,

automated method to set all parameters

So, given any time series, we can automatically build a prediction system

Useful in a sensor network setting

CIKM 2002 44

Snapshothttp://snapdragon.cald.cs.cmu.edu/TSPExtra

CIKM 2002 45

Future Work Feature Selection Multi-sequence prediction

CIKM 2002 46

Discussion – Some other problems

How to forecast?

•x1, x2, …, xN

•L(opt)

•k(opt)How to find the k(opt) nearest neighbors quickly?

Given:

CIKM 2002 47

MotivationForecasting also allows us to

• Find outliers anything that doesn’t match our prediction!

• Find patterns if different circumstances lead to similar predictions, they may be related.

CIKM 2002 48

Motivation (Examples)

• EEGs : Patterns of electromagnetic impulses in the brain

• Intensity variations of white dwarf stars

• Highway usage over time

Traditional

Sensors• “Active Disks” for forecasting / prefetching / buffering

• “Smart House” sensors monitor situation in a house

• Volcano monitoring

CIKM 2002 49

General Method• Store all the delay vectors {x{xt-1t-1, …, x, …, xt-L(opt)t-L(opt)} } and corresponding prediction xand corresponding prediction x tt

xt• Find the latest delay vector

L(opt) = ?

• Find nearest neighbors

K(opt) = ?

Interpolate• Interpolate

CIKM 2002 50

Intuition

• The FD vs L plot does flatten out

• L(opt) = 1

Fractal dimension

CIKM 2002 51

Inside Theory Internal state may be unobserved But the delay vector space is a

faithful reconstruction of the internal system state

So prediction in delay vector space is as good as prediction in state space

CIKM 2002 52

Fractal Dimensions Many real-world datasets have

fractional intrinsic dimension There exist fast (O(N)) methods to

calculate the fractal dimension of a cloud of points [Belussi/1995]

CIKM 2002 53

Speed and Scalability Preprocessin

g varies as L(opt)2

F4: Large Scale Automated Forecasting Using Fractals

Documents