Introduction Statistics of stochastic processes · Introduction Statistics of stochastic processes...

Introduction

Statistics of stochastic processes

Generally statistics is performed on observations

y1, . . . , yn

assumed to be realizations of independent random variables

Y1, . . . ,Yn.

In statistics of stochastic processes (= time series analysis) we will assume

y1, . . . , yn

to be realizations of a stochastic process

. . .Y1, . . . ,Yn, . . .

with some rules for dependence .

14 settembre 2014 1 / 31

Introduction

Statistics of stochastic processes

Generally statistics is performed on observations

y1, . . . , yn

assumed to be realizations of independent random variables

Y1, . . . ,Yn.

In statistics of stochastic processes (= time series analysis) we will assume

y1, . . . , yn

to be realizations of a stochastic process

. . .Y1, . . . ,Yn, . . .

with some rules for dependence .14 settembre 2014 1 / 31

Introduction

Aims of time series analysis

Drawing inference from available data, but first we need to find anappropriate model.Once a model has been selected:

provide a compact and correct description of data (trend, seasonaland random terms)

adjust data (filtering, missing values) [separating noise from signal]

test hypotheses (increasing trend? influence of factors) → understandcauses

predict future values

Remarks

- Random terms generally described as a stationary process.

- Linear analysis (additive decomposition of trend, seasonaland stationary process)

14 settembre 2014 2 / 31

Introduction







Remarks



14 settembre 2014 2 / 31

Introduction







Remarks



14 settembre 2014 2 / 31

Introduction







Remarks



14 settembre 2014 2 / 31

Introduction







Remarks



14 settembre 2014 2 / 31

Introduction







Remarks



14 settembre 2014 2 / 31

Introduction

Stationary process

Definition

A stochastic process Xtt∈Z is (strictly) stationary if

the joint distribution of (Xt1 ,Xt2 , . . . ,Xtk ) is equal to

the distribution of (Xt1+h,Xt2+h, . . . ,Xtk+h)

∀ k ∈ N, h ∈ Z, t1, t2, . . . , tk ∈ Z.

In particular, if a stationary stochastic process has finite second moment,then

E(Xt) and Cov(Xt ,Xt+h) do not depend on t.

14 settembre 2014 3 / 31

Introduction

Stationary process. 2

Linear time series analysis looks only at second-order properties. Then

Definition

A stochastic process Xtt∈Z is stationary if it is in L2 and

E(Xt) = µ Cov(Xt ,Xt+h) = γ(h).

If a Gaussian process is stationary, then it is strictly stationary.A Gaussian process is such that all finite-dimensional distributions aremultivariate normal.

14 settembre 2014 4 / 31

Introduction



Definition



If a Gaussian process is stationary, then it is strictly stationary.

A Gaussian process is such that all finite-dimensional distributions aremultivariate normal.

14 settembre 2014 4 / 31

Introduction



Definition



If a Gaussian process is stationary, then it is strictly stationary.A Gaussian process is such that all finite-dimensional distributions aremultivariate normal.

14 settembre 2014 4 / 31

Introduction

Reminders on multivariate normal

Definition

Y = (Y1, . . . ,Yn) is multivariate normal if, ∀a ∈ Rn, atY is a univariatenormal.

Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b. =⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).

Alternative characterization via characteristic function.

If Cov(Y ) = S positive definite (i.e. invertible), Y ∼ N(µ, S) has density

fY (y) = (2π)−n/2|S |−1/2 exp−(y − µ)tS−1(y − µ)/2.

(non-singular distribution)

14 settembre 2014 5 / 31

Introduction


Definition


Equivalently, Y is multivariate normal ⇐⇒ there exists b ∈ Rn A(n ×m) matrix, X = (X1, . . . ,Xm) independent standard normal r.v. suchthat Y = AX + b.

=⇒ E(Y ) = b, Cov(Y ) = AAt , i.e. Y ∼ N(b,AAt).





14 settembre 2014 5 / 31

Introduction


Definition







14 settembre 2014 5 / 31

Introduction


Definition







14 settembre 2014 5 / 31

Introduction


Definition







14 settembre 2014 5 / 31

Introduction

Gaussian processes

Definition

A process Xt is Gaussian, if for any n > 0 and any (t1, . . . , tn) the vectorX = (Xt1 , . . . ,Xtn) has a non-singular multivariate normal distribution.

Then let µ = (µt1 , . . . , µtn) = E(X) andCov(X) = Γ = γ(ti , tj), i , j = 1 . . . n. X has density function

g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp

−1

2〈Γ−1(x − µ), x − µ〉

.

Xt is (weakly) stationary if µt ≡ µ and γ(ti , tj) = γ(|ti − tj |); then isalso strictly stationary, as the distribution depends only on µ and Γ.

Linear time series analysis is very well suited for Gaussian processes; less sofor non-Gaussian ones.

14 settembre 2014 6 / 31

Introduction

Gaussian processes

Definition



g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp

−1

2〈Γ−1(x − µ), x − µ〉

.



14 settembre 2014 6 / 31

Introduction

Gaussian processes

Definition



g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp

−1

2〈Γ−1(x − µ), x − µ〉

.



14 settembre 2014 6 / 31

Introduction

Gaussian processes

Definition



g(x , µ, Γ) = (2π)−n/2|Γ|−1/2 exp

−1

2〈Γ−1(x − µ), x − µ〉

.



14 settembre 2014 6 / 31

Introduction

Hilbert spaces

Many time series problems can be solved using Hilbert space theory.Indeed space L2(Ω) is a Hilbert space with

〈X ,Y 〉 = E(XY ), ‖X − Y ‖2 = E(|X − Y |2).

Restricting to the 0-mean subspace 〈X ,Y 〉 = Cov(X ,Y ).

14 settembre 2014 7 / 31

Introduction

Detrending data

Often data do not appeat as arising from stationary processes.

Estimating trend, and then study residuals (differences from trend)

smoothingpolynomial (esp. line) fitting

Study differenced series

In all cases, trasformations may be useful

More systematic model fitting in the future.

14 settembre 2014 8 / 31

Examples of time series

Johnson & Johnson quarterly earnings

J & J

Time

Ear

ning

s pe

r Sha

re

1960 1965 1970 1975 1980

05

1015

data3 points smoothing5 points smoothing

14 settembre 2014 9 / 31


Johnson & Johnson data: deviations from trend

Time

Dev

iatio

ns fr

om m

ovin

g av

erag

e

1960 1965 1970 1975 1980

-3-2

-10

12

14 settembre 2014 10 / 31


Johnson & Johnson data: deviations in log-scale

Time

Dev

iatio

ns (i

n lo

g sc

ale)

from

mov

ing

aver

age

1960 1965 1970 1975 1980

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

14 settembre 2014 11 / 31


Sunspots data 1700-1980

1700 1750 1800 1850 1900 1950

050

100

150

year

sunspots

14 settembre 2014 12 / 31


Sunspots data 1700-1980: square-root transformation

1700 1750 1800 1850 1900 1950

02

46

810

1214

year

sunspots

14 settembre 2014 13 / 31


PanAm international air passengers 1949-60

Time

Pas

seng

ers

(100

0's)

1950 1952 1954 1956 1958 1960

100

200

300

400

500

600

14 settembre 2014 14 / 31


PanAm yearly data

Annual air passengers

Time

aggregate(AP)

1950 1952 1954 1956 1958 1960

2000

3000

4000

5000

14 settembre 2014 15 / 31


PanAm monthly variation

1 2 3 4 5 6 7 8 9 10 11 12

100

200

300

400

500

600

Seasonal component in air passengers

Month

14 settembre 2014 16 / 31


Level of Lake Huron 1875-1972

Level of lake Huron

Time

ft

1880 1900 1920 1940 1960

67

89

1011

12

14 settembre 2014 17 / 31


Lake Huron level: deviations from trend

Deviations from trend in level of lake Huron

Time

ft

0 20 40 60 80 100

-2-1

01

2

14 settembre 2014 18 / 31


sales of red wine in Australia 1980-91

Red wine sales in Australia

Time

kilolitres

1980 1982 1984 1986 1988 1990 1992

500

1000

1500

2000

2500

3000

14 settembre 2014 19 / 31


Deviation from trend in wine sales

Deviations from trend in sales of red wine

Time

kilolitres

0 20 40 60 80 100 120 140

-1000

-500

0500

1000

14 settembre 2014 20 / 31


PanAm monthly variation

1 2 3 4 5 6 7 8 9 10 11 12

500

1000

1500

2000

2500

3000

Seasonal variation in wine sales (AUS)

14 settembre 2014 21 / 31


Global temperature data 1856-2005

Global temperature data

Time

Ano

mal

ies

from

196

1-90

mea

n

1900 1950 2000

-1.0

-0.5

0.0

0.5

Monthly averagesYearly averages

14 settembre 2014 22 / 31


Global temperature: recent years and trend

Global temperatures 1971-2005 (regression line in blue)

Time

Anomalies

1970 1975 1980 1985 1990 1995 2000 2005

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

14 settembre 2014 23 / 31


Measles data in England 1944-1967

1945 1950 1955 1960 1965

05000

10000

15000

20000

25000

Measles in England

year

case

s pe

r biw

eek

14 settembre 2014 24 / 31


EEG data from a subject with epilepsy

0 1000 2000 3000 4000

-150

-50

50150

time (arbitrary unit)

EEG

14 settembre 2014 25 / 31


De-trend and de-seasonalize (period T = 2q)

yearly average: mt =1

T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).

seasonal deviation: wk =1

n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .

seasonal component: sk = wk − 1T

T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .

deseasonalized data dt = xt − st .

mt trend component on deseasonalized data.

Yt = xt − mt − st random component.

Otherwise, difference data: ∇TXt := Xt − Xt−T .

∇TXt are de-seasonalized; then a trend can be eliminated from these.

14 settembre 2014 26 / 31




T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).


n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .


T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .






14 settembre 2014 26 / 31




T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).


n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .


T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .






14 settembre 2014 26 / 31




T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).


n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .


T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .






14 settembre 2014 26 / 31




T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).


n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .


T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .






14 settembre 2014 26 / 31




T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).


n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .


T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .






14 settembre 2014 26 / 31




T

(12xt−q +

q−1∑j=−(q−1)

xt+j + 12xt+q

).


n

n−1∑j=0

(xjT+k −mjT+k) , k = 1 . . .T .


T∑i=1

wi , k = 1 . . .T .

st = st−[ t−1T ]T , t > T .






14 settembre 2014 26 / 31


Autocovariance and autocorrelation functions

If a process Xt is stationary,

γ(h) := Cov(Xt ,Xt+h) is the Autocovariance function (ACVF).

Recall the correlation ρ(X ,Y ) =Cov(X ,Y )√V (X )V (Y )

.

For a stationary process V (Xt) = V (Xt+h) = γ(0). Hence

ρ(h) = ρ(Xt ,Xt+h) =γ(h)

γ(0)is the Autocorrelation function (ACF).

First properties of ACVF:

γ(h) = γ(−h) [stationarity =⇒ Cov(Xt ,Xt+h) = Cov(Xt−h,Xt)]

|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]

14 settembre 2014 27 / 31






.






|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]

14 settembre 2014 27 / 31






.






|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]

14 settembre 2014 27 / 31






.






|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]

14 settembre 2014 27 / 31






.






|γ(h)| ≤ γ(0) [as |ρ(X ,Y ) ≤ 1]

14 settembre 2014 27 / 31


Simple stationary processes and their ACVF

IID(0, σ2): Xtt∈Z independent and identically distributed r. v. withE(Xt) = 0, V(Xt) = σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.

WN(0, σ2) [white noise] Xtt∈Z uncorrelated random variables withmean 0 and variance σ2: γ(0) = σ2, γ(h) = 0 for |h| > 0.

WN(0, σ2) need not be independent. For instance if Ztt∈Z are IID andN(0,1) [normal r.v.], then

Xt =

Zt t odd

(Z 2t−1 − 1)/

√2 t even

is WN(0, 1) but not IID(0, 1).

It is not IID, since (e.g.) X1 and X2 are obviously not independent. Leftfor exercise that Xt is WN.

Less contrived examples of Xtt∈Z WN but not IID will be seen later inthe course.

14 settembre 2014 28 / 31






Xt =

Zt t odd

(Z 2t−1 − 1)/

√2 t even




14 settembre 2014 28 / 31






Xt =

Zt t odd

(Z 2t−1 − 1)/

√2 t even




14 settembre 2014 28 / 31






Xt =

Zt t odd

(Z 2t−1 − 1)/

√2 t even




14 settembre 2014 28 / 31


Moving average processes and their ACVF . 2

MA(1): moving average Xtt∈Z is MA(1) if

Xt = Zt + ϑZt−1, t ∈ Z where ϑ ∈ R, Zt ∼WN(0, σ2).

A simple computation:

γ(0) = σ2(1 + ϑ2), γ(1) = ϑσ2, γ(h) = 0 for |h| > 1.

Similarly Xtt∈Z ∼ MA(q) if

Xt = Zt + ϑ1Zt−1 + · · ·ϑqZt−q, t ∈ Z,with ϑ1, . . . , ϑq ∈ R, Zt ∼WN(0, σ2).

Another simple computation leads to γ(h) = 0 for |h| > q.

14 settembre 2014 29 / 31


Moving average processes and their ACVF . 2

MA(1): moving average Xtt∈Z is MA(1) if

Xt = Zt + ϑZt−1, t ∈ Z where ϑ ∈ R, Zt ∼WN(0, σ2).

A simple computation:

γ(0) = σ2(1 + ϑ2), γ(1) = ϑσ2, γ(h) = 0 for |h| > 1.

Similarly Xtt∈Z ∼ MA(q) if

Xt = Zt + ϑ1Zt−1 + · · ·ϑqZt−q, t ∈ Z,with ϑ1, . . . , ϑq ∈ R, Zt ∼WN(0, σ2).

Another simple computation leads to γ(h) = 0 for |h| > q.

14 settembre 2014 29 / 31


AutoRegressive processes

AR(1) [AutoRegressive] Xtt∈Z is AR(1) if is stationary and

Xt = φXt−1 + Zt , t ∈ Z where φ ∈ R, Zt ∼WN(0, σ2). (1)

(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.

Now, assume a stationary process Xtt∈Z exists satisfying (1) andE(XtZs) = 0 for t < s (this latter property seems natural as Xt should bedefined in terms of Zt and the previous ones).

Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)

= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.

Hence γ(0) =σ2

1− φ2(makes sense only if φ2 < 1 ).

14 settembre 2014 30 / 31





(1) is an (infinite set of) equation. It is not obvious that a stationaryprocess exists satisfying them (this will be discussed later).

We are not saying Xtt∈N is the Markov chain defined throughXt = φXt−1 + Zt , t > 0 with X0 some prescribed r.v.


Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)

= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.

Hence γ(0) =σ2


14 settembre 2014 30 / 31







Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)

= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.

Hence γ(0) =σ2


14 settembre 2014 30 / 31







Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)

= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.

Hence γ(0) =σ2


14 settembre 2014 30 / 31







Then γ(0) = V(Xt) = E((φXt−1 + Zt)2)

= φ2V(Xt−1) + σ2 + 2φE(Xt−1Zt) = φ2γ(0) + σ2.

Hence γ(0) =σ2


14 settembre 2014 30 / 31


AutoRegressive processes. 2

Remarks: we have found φ2 < 1 ⇐⇒ |φ| < 1 as a necessary conditionfor an AR(1) satisfying E(XtZs) = 0 for t < s. It will also be sufficient.

Implicit assumption in the computations: E(Xt) = 0 (this can be provedanalogously).

More simply, one can then compute γ(h) for h > 0 (left for exercise).

14 settembre 2014 31 / 31

Date post:	26-Aug-2018
Category:	Documents
Upload:	truongdien
View:	233 times
Download:	0 times

Introduction Statistics of stochastic processes · Introduction Statistics of stochastic processes...

Documents