Hidden Markov Models for wind time series

HMMStochastic wind generators

Forecast correction

Hidden Markov Models for wind time series

Pierre Ailliot1 Valérie Monbet2

1Université de Brest2Université de Bretagne Sud

Ailliot, Monbet Hidden Markov Models for wind time series


Forecast correction

Outline

1 Hidden Markov Models (HMM)ModelStatistical inference in HMMHMM with finite hidden stateHMM with continuous hidden state : state-space models

2 Stochastic wind generatorsMotivationsA HMM for the wind directionA HMM for the wind speedA joint HMM for the wind speed and the wind direction ?Conclusion (first part)

3 Numerical weather forecast correctionIntroductionA non linear state-space modelSome numerical results



Forecast correction

ModelStatistical inference in HMMHMM with finite hidden stateState-space models

Outline






Forecast correction


Hidden Markov Models

Definition

{Xt} = {St , Yt} ∈ {S × Y} , with {St} not observable ("hidden") and

P(St |S0 = s0, · · · , St−1 = st−1, Y1 = y1, · · · , Yt−1 = yt−1) = P(St |St−1 = st−1)

P(Yt |S0 = s0, · · · , St = st , Y1 = y1, · · · , Yt−1 = yt−1) = P(Yt |St = st)

State · · · → St−1 → St → St+1 → · · ·↓ ↓ ↓

Observation · · · Yt−1 Yt Yt+1 · · ·

Parametrization of a HMM

pθ(st |st−1) : transition probability

pθ(yt |st ) : emission probability



Forecast correction


Outline






Forecast correction


Statistical inference in HMM

Likelihood function

pθ(Y1 = y1, · · · , Yt = yT ) =

∫ST+1

pθ(s0)ΠTt=1pθ(st |st−1)pθ(yt |st)ds0ds1...dsT

= ΠTt=1

∫S

pθ(yt |st )pθ(st |y1, · · · , yt−1)dst

Prediction : evaluate pθ(st |y1, · · · , yt−1)

pθ(st |y1, · · · , yt−1) =

∫S

pθ(st |st−1)pθ(st−1|y1, · · · , yt−1)dst−1

Filtering : evaluate pθ(st |y1, · · · , yt )

pθ(st |y1, · · · , yt ) ∝ pθ(yt |st)pθ(st |y1, · · · , yt−1)

Smoothing : evaluate pθ(st |y1, · · · , yt , · · · , yT )

pθ(st |y1, · · · , yT )

= pθ(st |y1, · · · , yt)

∫S

pθ(st+1|st)

pθ(st+1|y1, · · · , yt )pθ(st+1|y1, · · · , yT )dst+1



Forecast correction


Outline






Forecast correction


HMM with finite state

Example :Yt = m(St ) + σ(St )Wt

Wt ∼ iidN (0, 1)

m(1) = 1 , m(2) = 2 , σ(1) = 0.2 , σ(2) = 0.5 , Q =

(0.95 0.050.1 0.9

)

1

3

0

1

P(S

t=1|

y 1,...,y

T)

Likelihood function, filtering and smoothing : forward-backward algorithm

Maximization of the likelihood function : EM and/or gradient-based algorithm



Forecast correction


Outline






Forecast correction


State-space models

Gaussian linear state-space models{

St = αSt−1 + β + σV VtYt = aSt + b + σW Wt

Vt ∼ iidN (0, 1), Wt ∼ iidN (0, 1)Likelihood function, filtering and smoothing : Kalman filterMaximization of the likelihood function : EM and/or gradient-based algorithm

Non-linear state-space modelsLikelihood function, filtering and smoothing : Monte-Carlo approximations

Particle filters

Maximization of the likelihood function : MCEM and/or stochastic gradient algorithm



Forecast correction

MotivationsA HMM for the wind directionA HMM for the wind speedA joint HMM for the wind speed and the wind direction ?Conclusion (first part)

Outline






Forecast correction


Motivations

Many natural phenomena and human activities depend on wind conditions

Production of electricity by wind turbines

Evolution of a coast line

Maritime transport

Drift of objects in the ocean

...

Wind data generally available on short periods of time

50 years of data maximum

Not enough to compute reliable estimates of the probability of complex events

Stochastic model used to simulate artificial wind conditions

Monte-Carlo methods



Forecast correction


Example : drift of an object in the ocean

ref : Ailliot, Frenod, Monbet, Multiscale Modeling and Simulation (2006)

What is the probability that a lost container ends up on the coast ?



Forecast correction


Example : drift of an object in the ocean

Object’s trajectory depends on current and wind conditions

Example

−3

0

3

0 0.2 0.4 0.6 0.8 1−3

0

3

Wind time series

→ODE→

0.9

1

1.1

1.2

0 0.2 0.4 0.6 0.8 10.7

0.8

0.9

1

1.1

Object’s trajectory



Forecast correction


Object’s drift can last several weeks

No wind forecast available

Artificial wind conditions simulated with a stochastic model

0.7 0.8 0.9 1 1.1 1.2 1.3

0.7

0.8

0.9

1

1.1

1.2

1.3

5%10%15%20%

E

N

W

S

5% 10%15%

E

N

W

S

< 01< 02< 03< 04

50 trajectories Wind rose Running aground locations(1000 trajectories)



Forecast correction


Outline






Forecast correction


A HMM for the wind direction

22 years of data, ∆t = 6hfocus on January : stationarity ?

Existence of different weather types ?Wind direction (Jan. 2000)

0 5 10 15 20 25 30W

S

E

N

W

Marginal distribution : 2 modes ?

S

W

N

E



Forecast correction



AssumptionsSt ∈ {1, · · · , M} : weather typeYt = Φt : wind directionVon-Mises distribution for P(Φt |St = st ) :

P(Φt = φ|St = st ) =1

2πI0(κ(st ))exp

(κ(st ) cos(φ − µ(st ))

)

Model selection

M 1 2 3 4 5 6BIC 9630 7454 6587 5690 5253 5324

Maximum likelihood estimates (M=2)

Regime 1 : µ(1) = 60o (ENE) , κ(2) = 1.79 - Easterlies, anticyclonic conditions

Regime 2 : µ(2) = 242o deg (WSW), κ(1) = 2.17 - Westerlies, cyclonic conditions

Transition matrix :[

0.953 0.0470.034 0.966

], stationary distribution :

[0.420.58

]



Forecast correction



Marginal distribution

S

W

N

E

S

W

N

E

S

W

N

E

Regime 1 (42%) Regime 2 (58%) Mixed ditributionSmoothing probabilities (Jan. 2000)

0

0.5

1

P(S

t=1|

y 1,...,y

T)

0 5 10 15 20 25 30WSENW

y t



Forecast correction



Simulated time series (slightly better results with M = 5)

0 5 10 15 20 25 30W

S

E

N

W

Observed time series (Jan. 2000)

0 5 10 15 20 25 30W

S

E

N

W

The model can not reproduce "small scale" dynamicsConditional independence assumptions too strong ?

P(Yt |S0 = s0, S1 = s1, Y1 = y1, · · · , St−1 = st−1, Yt−1 = yt−1) = P(Yt |St = st , Yt = yt−1)

Realistic simple parametric model for P(Yt |St = st , Yt = yt−1) ?



Forecast correction


Outline






Forecast correction


A HMM for the wind speed

Existence of different weather types ?Wind speed (Jan. 2000)

0 5 10 15 20 25 300

5

10

15

20

Wind direction (Jan. 2000)

0 5 10 15 20 25 30W

S

E

N

W

Higher volatility in cyclonic conditions ?



Forecast correction


A HMM for the wind speed

AssumptionsSt ∈ {1, · · · , M} : weather typeYt = Ut : wind speedMarkov-switching autoregressive model :

P(Yt |S0 = s0, · · · , St = st , Y0 = y0, · · · , Yt−1 = yt−1) = P(Yt |St = st , Yt = yt−1)

· · · → St−1 → St → St+1 → · · ·↓ ↓ ↓

· · · → Yt−1 → Yt → Yt+1 → · · ·Gamma distribution for P(Yt |St = st , Yt = yt−1), with

mean : a(st )yt−1 + b(st )

standard deviation : σ (st )

Model selection

M 1 2 3 4 5BIC 10485 10301 10307 10343 10387



Forecast correction


Maximum likelihood estimatesRegime 1 : low volatility, anticyclonic conditions

E [Yt |Yt−1 = yt−1, St = 1] = .79yt−1 + 1.46

var [Yt |Yt−1 = yt−1, St = 1] = 1.372

Regime 2 : higher volatility, cyclonic conditionsE [Yt |Yt−1 = yt−1, St = 1] = .77yt−1 + 2.24

var [Yt |Yt−1 = yt−1, St = 1] = 2.42

Transition matrix :[

0.98 0.020.03 0.97

], stationary distribution

[0.400.60

]

Smoothing probabilities (Jan. 2000)

0 5 10 15 20 25 300

10

20

0

1

0 5 10 15 20 25 300

1

Model improves results obtained with ARMA modelsMarginal distribution, autocorrelation function, storm and inter-storm durations...



Forecast correction


Outline






Forecast correction


A joint HMM for the wind speed and the wind direction ?

Correspondence between regimes identified on the wind speed and the winddirection ?

HMM for the wind direction

00.5

1

5 10 15 20 25 300

0.51

0 5 10 15 20 25 30WSENW

Wind direction in the 2 regimesS

W

N

E

S

W

N

E

HMM for the wind speed

0 5 10 15 20 25 300

10

20

0

1

0 5 10 15 20 25 300

1

Wind direction in the 2 regimes

3 regimes ?



Forecast correction


A joint HMM for the wind speed and the wind direction ?

A realistic model structure ?

· · · → St−1 → St → St+1 → · · ·↓ ↓ ↓

· · · → (Ut−1, Φt−1) → (Ut , Φt) → (Ut+1, Φt+1) → · · ·

Realistic simple parametric model for P(Ut , Φt |Ut−1, Φt−1, St = s) ?

Another model

· · · Φt−1 Φt Φt+1 · · ·↓ ↓ ↓

· · · → St−1 → St → St+1 → · · ·↓ ↓ ↓

· · · → Ut−1 → Ut → Ut+1 → · · ·

Ref Ailliot, Monbet , preprint (2007)



Forecast correction


Outline






Forecast correction


Conclusion (first part)

Major virtues of HMM ?

Distributional versatility

Ability to model diverse time scales

Open structure which allows for more physical models



Forecast correction

IntroductionA non linear state-space modelSome numerical results

Outline






Forecast correction


Example

Correct numerical weather predictions given in situ observations

Data from satellites, buoys, ships....

−40 −30 −20 −10 0

40

42

44

46

48

50

0

1

3

5

6

8

10



Forecast correction


A first model

Linear Gaussian spate-space model

bt = αbt−1 + β + σV Vt Error (hidden)

Y truet = Y for

t + bt "True" Yt (hidden)Yobs

t = Y truet + σW Wt Observed Yt

Vt ∼ iidN (0, 1), Wt ∼ iidN (0, 1)

Inference

Kalman Filter (1D)

Initializations : ma, PaFor t = 1,...,T,% predictionmpred (t) = αma + β

Ppred (t) = αPaα′ + σ2V

% analysisK = Ppred (t)/(Ppred (t) + σ2

W )

ma = (1 − K )mpred (t) + KO(t) =σ2

WPpred (t)+σ2

Wmpred (t) +

Ppred (t)

Ppred (t)+σ2W

O(t)

Pa = (1 − K )Ppred (t)End



Forecast correction


Need for a more advanced model ?

Two types of errors ?Intensity errorPosition error

Example : wind fields

−12 −10 −8 −6 −4 −2 043

44

45

46

47

48

49

50

51

52Champs analysé 09/15 18 H

"Observed" field (nowcast)

−12 −10 −8 −6 −4 −2 043

44

45

46

47

48

49

50

51

52Champs prédit 09/15 18 H

Predicted field (18 hforecast)

Position error not included in the linear model !



Forecast correction


First step : single location

Position error ↔ phase error

0 6 12 18 240

2

4

6

8

10

12

14

16

observed, forecast, * : nowcast

−6 −4 −2 042

43

44

45

46

47

48

49

50

51

52

Ouessant



Forecast correction


Outline






Forecast correction


Introduction of a phase correction

Model

∆t = α∆∆t−1 + β∆ + σ∆V∆(t) Phase error (hidden)bt = αbbt−1 + βb + σbVb(t) Intensity error (hidden)

Y truet = Y for

t+∆t+ bt "True" Yt (hidden)

Yobst = Y true

t + σW W (t) Observed Yt

V∆(t) ∼ iidN (0, 1), Vb(t) ∼ iidN (0, 1), W (t) ∼ iidN (0, 1)

InferenceParameter estimation : maximum likelihood by a stochastic gradient algorithm

θk = θk−1 + γk ∂θLT (θ)

Monte Carlo approximation of LT (θ) and ∂θLT (θ)



Forecast correction


Approximation of logLT (θ)

One needs to approximate

pθ(st |y1, · · · , yt ) ∝ pθ(yt |st )

∫S

pθ(st |st−1)pθ(st−1|yt−11 )dst−1

Bootstrap particle filter

Initialization :{si

0}i∈{1,...,N} ∼ pθ(S0)For t = 1 to T% predictionsi

t,pred ∼ pθ(St |sit−1)

% resamplingν i ∝ pθ(yt |si

t,pred ) with∑N

i=1 ν i = 1

pθ(st |yt1) ≈

∑Nj=1 ν jδ

sjt,pred

{sit}i∈{1,...,N} ∼ pθ(St |yt

1) : sit = sφ(i)

t,predEnd

−15 −10 −5 0 5 10−2

−1

0

1

2

−15 −10 −5 0 5 10−2

−1

0

1

2Prediction

−15 −10 −5 0 5 10−2

−1

0

1

2Redistribution



Forecast correction


Approximation of gradient ∂θ logLT (θ)

logLT (θ) =T∑

t=1

log∫

Spθ(yt |st)pθ(st |yt−1

1 )dst

∂θ logLT (θ) =T∑

t=1

∫S p′

θ(yt |st)pθ(st |yt−11 ) + pθ(yt |st)p′

θ(st |yt−11 )dst∫

S pθ(yt |st)pθ(st |yt−11 )dst

At time t , {sit−1}{i∈{i,··· ,N} ∼ pθ(st−1|yt−1

1 ) hence

∫S

h(st−1)p′θ(st−1|yt−1

1 )dst−1

=

∫S

h(st−1)p′

θ(st−1|yt−11 )

pθ(st−1|yt−11 )

pθ(st−1|yt−11 )dst−1

≈ 1

N

N∑i=1

ωit h(si

t−1)

with ωit =

dpit

pit

et pit ≈ pθ(si

t |yt1), dpi

t ≈ p′θ(si

t |yt1)



Forecast correction


Algorithm to compute the gradient

% predictionsi

t,pred ∼ p(.|sit−1)

% Weigths

pθ(st |yt1 − 1) =

∫S pθ(st |st−1)pθ(st−1|yt−1

1 )dst−1 → pit,pred = 1

N

∑Nj=1 pθ(si

t,pred |sjt−1)

dpit,pred = 1

N

∑Nj=1 p′

θ(sit,pred |sj

t−1) + 1N

∑Nj=1 pθ(si

t,pred |sjt−1)ω

jt−1

ωit,pred = dpi

t,pred /pit,pred

lt = lt−1 + log(

1N

∑Ni=1 pθ(yt |si

t,pred ))

dlt = ...

% Correction - resampling

ν i ∝ pθ(yt |sit,pred ) et si

t = sφ(i)t,pred

pθ ∝ pθ(yt |sit,pred )pθ(st |yt−1

1 ) → pit =

pθ(yt |sit )p

φ(i)t,pred∑

νi /N

ωit = ...

dpit = ωi

t pit



Forecast correction


Outline






Forecast correction


Another dataset : wave propagation

Two Hs time series

40 60 80 100 120 140 1600

2

4

6 largecote

Inference : ∂θLT (a∆)

−8 −6 −4 −2 0 2 4 6 8−72

−70

−68

−66

−64

−62

Values of the parameters :α∆ = αb = 1, β∆ = βb = 0, σ∆ = 1,σb = 1, σW = 0.5

−10 −8 −6 −4 −2 043

44

45

46

47

48

49

50

51

52



Forecast correction


Some results on wave propagation

One time step prediction

40 60 80 100 120 140 1600

2

4

6LargeLarge+biaisCote

40 60 80 100 120 140 1600

2

4

6LargeLarge recalle en tpsLarge recalle en tps+biaisHs cote



Forecast correction


Some results on wave propagation

Root Mean Square Errors

0 2 4 6 80

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Prediction horizon

RM

SE



Forecast correction


Back to Wind data (on going work !)

Values of the parameters : α∆ = αb = 1, β∆ = βb = 0, σ∆ = 1, σb = 1.5, σW = 0.5

Good results in some situations

Linear Model

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

Non linear Model

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

0 6 12 18 240

5

10

15

observation (plain line), assimilated observations (points), forecast (plain line)

phase+bias correction (plain line), phase correction (dashed line)



Forecast correction


Back to Wind data (on going work !)

Values of the parameters : α∆ = αb = 1, β∆ = βb = 0, σ∆ = 1, σb = 1.5, σW = 0.5

Good results in some situations

Root Mean Square Errors

0 2 4 6 8 100

0.5

1

1.5

2

2.5



Forecast correction


Concluding remarks

Lack of information to identify bias and phase from the dataBreaks in the data (assimilation at 00 :00 each day) : burning periodsQuality of the data : discretization step, measurement error

Next step : use spatial dataSpace-time model for the true wind fields including the motion of the fieldsAilliot et al., Baxevani et al.Conditional model for in-situ observations


Date post:	02-Jan-2022
Category:	Documents
Upload:	others
View:	5 times
Download:	0 times

Hidden Markov Models for wind time series

Documents