Frequency Domain Techniques in
Forecasting with ARIMA Model
Dr. M. Jahanur RahmanAssociate Professor
Department of StatisticsUniversity of Rajshahi, Rajshahi-6205, Bangladesh
July 28, 2012
Dr. M. Jahanur Rahman, R.U.
Contents
Time Series?
Time domain forecasting with ARIMA model
Transformations from time domain to
frequency domain
Frequency domain techniques in forecasting
with ARIMA model
2
Dr. M. Jahanur Rahman, R.U. 3
Time Series
A sequence of observations indexed by time, denoted
by or
Monthly relative humidity time series of Rajshahi
Dr. M. Jahanur Rahman, R.U. 4
𝑦 𝑡+2
𝑦 𝑡+1𝑦 𝑡
𝑦 𝑡−1
𝑦 𝑡−2
= Realization
orSample pathTime Series?
A finite realization of a time series process
Let be a sequence of random variable indexed by time t.
Random Process to Time Series
Dr. M. Jahanur Rahman, R.U. 5
Note that
Time series is only one of the many possible
realizations (sample paths) that the history might
have generated.
Time series analysis is trying to draw statistical
inference from a single outcome (time series),
even partially observed.
Dr. M. Jahanur Rahman, R.U. 6
How can we make sensible
inferences on the underlying
time series process with a single
observation?We need a strong assumption:
Stationarity (or ergodicity). This assumption treat the time series as a random sample
from the same underlying population (process).
Dr. M. Jahanur Rahman, R.U. 7
A time series is weak stationary if its mean, variance, and autocovariance (at various lags) remain the same no matter at what point of time we measure them. That is, they are time invariant.
Stationary Time Series
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Dr. M. Jahanur Rahman, R.U. 8
Ergodicity
a realization(a sample path)
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Dr. M. Jahanur Rahman, R.U.
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Mea
n =
Varia
nce
= Co
varia
nce
=
Covariance = Variance =
Mean =
The covariance-stationary process is ergodic for the mean.
The covariance-stationary process is ergodic for the variance.
The covariance-stationary process is ergodic for the covariance.9
Ergodicity
Dr. M. Jahanur Rahman, R.U. 10
Non-stationary time series Non-stationary time series
-3
-2
-1
0
1
2
3
10 20 30 40 50 60 70 80 90 00
Stationary time series
(Non)Stationay Time Series
Dr. M. Jahanur Rahman, R.U. 11
Non-stationary time series contain two kinds of trends: Deterministic trend Stochastic (random) trend
Non-stationary Time Series
Dr. M. Jahanur Rahman, R.U. 12
Trend Stationary: If y(t) contains a deterministic trend and
{y(t) – trend}
becomes stationary. Then y(t) is known as trend stationary.
Difference Stationary: If y(t) contains a stochastic trend and
{y(t) – y(t-1)}
becomes stationary. Then y(t) is known as difference stationary.
Non-stationary Time Series
Dr. M. Jahanur Rahman, R.U. 13
Non-stationary Time Series
Dr. M. Jahanur Rahman, R.U. 14
Any time series can contain some or all of the following
components:
1. Trend (T)
2. Cyclical (C)
3. Seasonal (S)
4. Irregular (I)
Trend component
The trend is the long term pattern of a time series. A trend can be
positive or negative depending on whether the time series exhibits an
increasing long term pattern or a decreasing long term pattern.
Components of a Time Series
Dr. M. Jahanur Rahman, R.U. 15
Cyclical component
Any pattern showing an up and down movement around a given trend
is identified as a cyclical pattern. The duration of a cycle depends on
the type of business or industry being analyzed.
Any time series can contain some or all of the following
components:
1. Trend (T)
2. Cyclical (C)
3. Seasonal (S)
4. Irregular (I)
Components of a Time Series
Seasonal component
Seasonality occurs when the time series exhibits regular fluctuations
during the same month (or months) every year, or during the same
quarter every year. For instance, retail sales peak during the month of
December.
Any time series can contain some or all of the following
components:
1. Trend (T)
2. Cyclical (C)
3. Seasonal (S)
4. Irregular (I)
Components of a Time Series
Dr. M. Jahanur Rahman, R.U. 16
Dr. M. Jahanur Rahman, R.U. 17
Irregular component
This component is unpredictable. Every time series has some
unpredictable component that makes it a random variable. In prediction,
the objective is to “model" all the components to the point that the only
component that remains unexplained is the random component.
Any time series can contain some or all of the following
components:
1. Trend (T)
2. Cyclical (C)
3. Seasonal (S)
4. Irregular (I)
Components of a Time Series
Dr. M. Jahanur Rahman, R.U. 18
Components of a Time Series
Dr. M. Jahanur Rahman, R.U. 19
Two DomainsTraditionally, there are two ways to analyze time series data:
Time domain analysis
Time domain analysis examines how a time series process evolves
through time, with tools such as autocorrelation function.
Frequency domain analysis
Frequency domain analysis, also known as spectral analysis,
studies how periodic components at different frequencies
describe the evolution of a time series.
Dr. M. Jahanur Rahman, R.U. 20
Time
Time domainFrequency
Frequency domain
Two Domains
Dr. M. Jahanur Rahman, R.U. 21
The Autoregressive Integrated Moving Average (ARIMA)
models, or Box-Jenkins models, are a class of linear models
that is capable of representing stationary as well as non-
stationary time series.
Time Series Model
Dr. M. Jahanur Rahman, R.U. 22
ARIMA(p,d,q) Model
∆𝑑𝑌 𝑡=𝑐+𝜙1∆𝑑𝑌 𝑡− 1+𝜙2∆
𝑑𝑌 𝑡 −2+⋯𝜙𝑝 ∆𝑑𝑌 𝑡−𝑝+𝜀𝑡+𝜃1𝜀𝑡 −1+𝜃2𝜀𝑡− 2+⋯+𝜃𝑞𝜀𝑡 −𝑞
= Response (dependent) variable at time
= Respose variable at time lags
= difference operator ()
= White noise error term
= Errors in previous time periods
= parameters
= Number of autoregressive terms
= Number of moving average terms.
Dr. M. Jahanur Rahman, R.U. 23
ARIMA(p,0,q) = ARMA(p, q) Model
ARIMA(p,0,0) = AR(p) Model
ARIMA(0,0,q) = MA(q) Model
Dr. M. Jahanur Rahman, R.U. 24
ARIMA models rely heavily on both autocorrelation (AC) and
partial autocorrelation (PAC) patterns in time series data.
Dr. M. Jahanur Rahman, R.U. 25
ARIMA(2,0,0) = AR(2)
AC PA
C
1 2 3 4 5 6 7 …….. Lags 1 2 3 4 5 6 7 …….. Lags
ARIMA(0,0,2) = MA(2)1 2 3 4 5 6 7 …….. Lags 1 2 3 4 5 6 7 …….. Lags
AC
PAC
ARIMA(2,0,2) = ARMA(2, 2)
AC
PAC
1 2 3 4 5 6 7 …….. Lags1 2 3 4 5 6 7 …….. Lags
Theoretical pattern of ACF and PACFACF 0PACF = 0 for lag > 2
ACF = 0 for lag > 2; PACF 0
ACF PACF 0
Dr. M. Jahanur Rahman, R.U. 26
Choosing an ARIMA model
ARIMA modeling requires a great deal of skill, which of course come from practice.
Model Pattern of ACF Pattern of PACFAR(p) Spikes decays exponentially. Significance spikes through lags p
MA(q) Significance spikes through lags q Spikes decays exponentially
ARMA(p,q) Spikes decays exponentially Spikes decays exponentially
Theoretical pattern of ACF and PACF
Dr. M. Jahanur Rahman, R.U. 27
1. Identification of the model
(choosing tentative p, d, p for ARIMA)
2. Parameter estimation of the chosen model.
4. Forecasting
3. Diagnostic checking
(are the estimated residual white noise?)
yes
no
Box-Jenkins (BJ) Methodology
Dr. M. Jahanur Rahman, R.U. 28
Steps of modeling a observed time series
with ARIMA
Dr. M. Jahanur Rahman, R.U. 29
Original time series [
Log transformation
Dr. M. Jahanur Rahman, R.U. 30Regular differencing
Seasonal differencing
Dr. M. Jahanur Rahman, R.U. 31
Correlogram: Shows SAC & SPAC at different lags
Dr. M. Jahanur Rahman, R.U. 32
ARIMA(2,0,0) = AR(2)
AC PA
C
1 2 3 4 5 6 7 …….. Lags
1 2 3 4 5 6 7 ……..
ARIMA(0,0,2) = MA(2)1 2 3 4 5 6 7 …….. Lags
1 2 3 4 5 6 7 ……..
AC
PAC
ARIMA(2,0,2) = ARMA(2, 2)
AC
PAC
1 2 3 4 5 6 7 ……1 2 3 4 5 6 7 …….. Lags
Competitive Models:
(1) ARMA(2, 0) (2) ARMA(0, 1) (3) ARMA(2, 1)
Choosing an ARIMA model
Dr. M. Jahanur Rahman, R.U. 33
Model MSE
ARMA(2, 0) 3521
ARMA(0, 1) 4019
ARMA(2, 1) 3998
Looking for the best Model
Dr. M. Jahanur Rahman, R.U. 34
Finally Forecasting
Frequency Domain Fourier Transformation Short-time Fourier Transformation Wavelet Transformation
Dr. M. Jahanur Rahman, R.U. 36
Frequency Domain
To get further information from the time series that is not
readily available in the time domain.
Frequency Domain
Time Domain
Dr. M. Jahanur Rahman, R.U. 37
Autocorrelation function or correlogram is used for analyzing time series in time domain.
rk
k
Correlogram
t
Yt
Periodic process with noise
Time Domain
Periodicities in data can be best determined by analyzing the time series in frequency domain.
Time series
Dr. M. Jahanur Rahman, R.U. 38
Frequency Domain
Frequency Domain
Time Domain
Transformations:
Fourier Transformation (FT)
Short-time Fourier Transformation (SFT)
Wavelet Transformation (WT) ………
Dr. M. Jahanur Rahman, R.U. 39
Time series analysis in frequency domain is known as
frequency domain analysis or spectral analysis.
Dr. M. Jahanur Rahman, R.U. 40
The Basics
= Amplitude (height of the function), R > 0
= Frequency (cycle/unit time)
= Phase (the starting point of the cosine function
which lies in )
(period).
𝑌 𝑡=𝑅𝑐𝑜𝑠 (2𝜋 𝑓𝑡+𝜙 )
Dr. M. Jahanur Rahman, R.U. 41
𝑌 𝑡=𝑅𝑐𝑜𝑠 (2𝜋 𝑓𝑡+𝜙 )
t = running from 1 to 100. Red curve has , = 4/100 = 1/25 and Blue curve has , = 4/100 = 1/25 and Black curve has , = 4/100 = 1/25 and
The Basics
Dr. M. Jahanur Rahman, R.U. 42
as
𝑌 𝑡=𝑅𝑐𝑜𝑠 (2𝜋 𝑓𝑡+𝜙 )
The Basics
Dr. M. Jahanur Rahman, R.U. 43
𝑦 2𝑡=2 cos (2𝜋 5 𝑓𝑡 )+3sin (2𝜋 5 𝑓𝑡 )
𝑦 1𝑡=2cos (2𝜋 2 𝑓𝑡 )+3sin (2𝜋 2 𝑓𝑡 )
𝑦 𝑡=𝑦1 𝑡+𝑦2 𝑡
Jean Baptiste Joseph Fourier(1768-1830)
The Basics
Dr. M. Jahanur Rahman, R.U. 44
Mat
hem
atic
al C
once
pts
Dr. M. Jahanur Rahman, R.U. 45
𝑓 (𝑥 )= ⟨ 𝑓 (𝑥) ,1 ⟩⟨1,1 ⟩
1+∑𝑛=1
∞ ⟨ 𝑓 (𝑥) ,cos (𝑛𝑥) ⟩⟨cos (𝑛𝑥 ) ,cos (𝑛𝑥) ⟩
cos (𝑛𝑥)+∑𝑛=1
∞ ⟨ 𝑓 (𝑥) , sin (𝑛𝑥 )⟩⟨sin (𝑛𝑥 ), sin (𝑛𝑥 )⟩
sin (𝑛𝑥 )
So 𝑎0=1
2𝜋 ∫−𝜋
𝜋
𝑓 (𝑥 )𝑑𝑥
𝑎𝑛=1𝜋 ∫
−𝜋
𝜋
𝑓 (𝑥 )cos (𝑛𝑥 )𝑑𝑥
𝑏𝑛=1𝜋 ∫
−𝜋
𝜋
𝑓 (𝑥 )𝑠𝑖𝑛 (𝑛𝑥 )𝑑𝑥 𝑛=1,2,3 ,……
𝑓 (𝑥 )=𝑎0+∑𝑛=1
∞
𝑎𝑛cos (𝑛𝑥 )+∑𝑛=1
∞
𝑏𝑛sin (𝑛𝑥)
According to Fourier, any function in can be defined as
Fourier Series
Dr. M. Jahanur Rahman, R.U. 46
or equivalently,
Where
for
Fourier Series
Dr. M. Jahanur Rahman, R.U. 47
Time (t)
y(t)
Frequency (f)
Y(f)
Time domain to Frequency Domain
Time Domain Frequency Domain
Dr. M. Jahanur Rahman, R.U. 48
𝑦 2𝑡=2 cos (2𝜋 5 𝑓𝑡 )+3sin (2𝜋 5 𝑓𝑡 )
𝑦 1𝑡=2cos (2𝜋 2 𝑓𝑡 )+3sin (2𝜋 2 𝑓𝑡 )
𝑦 𝑡=𝑦1 𝑡+𝑦2 𝑡
2f 5f
5f
2f
Time Domain Frequency Domain
Time domain to Frequency Domain
Dr. M. Jahanur Rahman, R.U. 49
Discrete Fourier Transformation (DFT)Let a time sequence
and a frequency sequence
The DFT of is
Where
Note that and are of the same size.
The DFT of a time series is another time series.
Dr. M. Jahanur Rahman, R.U. 50
The inverse DFT (IDET) of , is given by
Inverse Discrete Fourier Transformation (IDFT)
DFT
IDFT
3f 40f6f
Fourier Transformation
Inverse Fourier Transformation.
Dr. M. Jahanur Rahman, R.U. 51
Spectral Density
Spectral density () is the amount of variance per interval of frequency: Angular frequency:
The plot of vs is called spectrum and as a whole called spectral density.
Dr. M. Jahanur Rahman, R.U. 52
Spectral Density
𝐼 (𝑘 )
𝜔𝑘
Spectral density transforms information from time domain to frequency domain
Spectral density
Dr. M. Jahanur Rahman, R.U. 53
Total area under the spectrum is equal to the variance of the process.
All information in frequency domain is extracted from the spectral density (or spectrum).
Spectral density () is the amount of variance per interval of frequency.A peak in the spectrum indicates an important contribution to
the variance at frequencies close the peak.Prominent spikes indicate periodicity. Several expressions for spectrum exist in literature.
Spectral Density
Dr. M. Jahanur Rahman, R.U. 54
Spectral Density For a complete random series the spectral density function is
constant – termed as white noise.White noise indicates that no frequency interval contains
more variance than any other frequency intervals.
𝐼 (𝑘 )
𝜔𝑘Spectral density of a White noise
Dr. M. Jahanur Rahman, R.U. 55
3f 40f6f
DFT
Time Series Periodogram
IDFT
Filtered Series
Remove high
frequency component
ReconstructionDecomposition
Application of Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 56
0 0.2 0.4 0.6 0.8 1-3
-2
-1
0
1
2
3
0 5 10 15 20 250
100
200
300
400
500
600
All frequency components exist at all times
Stationary Time Series
Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 57
Frequency
Am
plitu
de
Fourier TransformationNon-stationary Series
Dr. M. Jahanur Rahman, R.U. 58
0 0.5 1-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 250
50
100
150
Time Frequency0 0.5 1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 250
50
100
150
Time Frequency
Different in Time Domain
Same in Frequency Domain
FT only gives frequency components exist in the seriesThe time and frequency Information can not be seen at the same
time which is required.
Fourier TransformationNon-stationary Series
Dr. M. Jahanur Rahman, R.U. 59
Short-Time Fourier TransformationDennis Gabor (1946) In STFT, the signal is divided into small
enough segments, where these segments of the series can be assumed to be stationary.
For this purpose, a fixed window function "w" is chosen
Dr. M. Jahanur Rahman, R.U. 60
Short-Time Fourier TransformationDennis Gabor (1946) In STFT, the signal is divided into small
enough segments, where these segments of the series can be assumed to be stationary.
For this purpose, a fixed window function "w" is chosen
Dr. M. Jahanur Rahman, R.U. 61
FT X
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 62
FT X
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 63
FT X
Advantage & Disadvantage
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 64
FT X
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 65
FT X
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 66
Time stepFrequency
Am
plitu
deShort-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 67
Time
Freq
uenc
y
Better time resolution; Poor frequency resolution
Better frequency resolution; Poor time resolution
Narrow windows give good time resolution, but poor frequency resolution.
Wide windows give good frequency resolution, but poor time resolution.
Advantage & Disadvantage
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 68
Wide windows do not provide good localization at high frequencies
Advantage & Disadvantage
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 69
Use narrower windows at high frequencies
Advantage & Disadvantage
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 70
Narrow windows do not provide good localization at low frequencies
Advantage & Disadvantage
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 71
Use wider windows at low frequencies
Advantage & Disadvantage
Short-Time Fourier Transformation
Dr. M. Jahanur Rahman, R.U. 72
= Small value
(Wavelet coefficient)
.54321
TimeSc
ale
Scale = 1, time = 0
Changing Window at Multi-resolution Analysis
Dr. M. Jahanur Rahman, R.U. 73
= Small value
(Wavelet coefficient)
.54321
TimeSc
ale
Scale = 1, time = 100
Changing Window at Multi-resolution Analysis
Dr. M. Jahanur Rahman, R.U. 74
= Moderate value
(Wavelet coefficient)
.54321
TimeSc
ale
Scale = 1, time = 500
Changing Window at Multi-resolution Analysis
Dr. M. Jahanur Rahman, R.U. 75
= Large value
(Wavelet coefficient)
.54321
TimeSc
ale
Scale = 1, time = 800
Changing Window at Multi-resolution Analysis
Dr. M. Jahanur Rahman, R.U. 76
Mag
nitu
de
20 Hz 50 Hz 120 Hz
Changing Window at Multi-resolution Analysis
Dr. M. Jahanur Rahman, R.U. 77
The Discrete Wavelet Transform (DWT)
𝐷𝑊𝑇 (𝑚 ,𝑛 )=∑𝑡𝑌 (𝑡 )Ψ (𝑚 ,𝑛 )
= wavelet function = mother wavelet m = is location parameter n = scale parameter = time series
Dr. M. Jahanur Rahman, R.U. 78
Wavelet families
Dr. M. Jahanur Rahman, R.U. 79
Multi-level Decomposition of a Series with Wavelets
Time Series
cA3 cD3
cA2 cD2
cA0 cD0The decomposition tree can be schematically described as:
Dr. M. Jahanur Rahman, R.U. 80
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0 - 0 0 0 0 0 00 0 - 0 0 0 00 0 0 0 - 0 00 0 0 0 0 0 -
0 0 0 00 0 0 0
- - 0 0 0 00 0 0 - -
- - - -
Basis Constriction
Dr. M. Jahanur Rahman, R.U. 81
Example
Dr. M. Jahanur Rahman, R.U. 82
Multi-level Decomposition of a Series with Wavelets
Time Series1, 3, 5, 11, 12, 13, 0, 1
cA32.8, 11.3, 17.6, 0.7
cD3-1.4, -4.2, -0.7, -0.7
0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0 - 0 0 0 0 0 00 0 - 0 0 0 00 0 0 0 - 0 00 0 0 0 0 0 -
Level-1 orScale-1
Dr. M. Jahanur Rahman, R.U. 83
Multi-level Decomposition of a Series with Wavelets
Time Series1, 3, 5, 11, 12, 13, 0, 1
cA32.8, 11.3, 17.6, 0.7
cD3-1.4, -4.2, -0.7, -0.7
Level-1 orScale-1
Wavelet coefficients = {2.8, 11.3, 17.6, 0.7, -1.4, -4.2, -0.7, -0.7}
Dr. M. Jahanur Rahman, R.U. 84
Multi-level Decomposition of a Series with Wavelets
Time Series1, 3, 5, 11, 12, 13, 0, 1
cA32.8, 11.3, 17.6, 0.7
cD3-1.4, -4.2, -0.7, -0.7
cA210.0, 13.0
cD2-6.0, 12.0
Level-1 orScale-1
Level-2 orScale-2
0 0 0 00 0 0 0
- - 0 0 0 00 0 0 - -
Wavelet coefficients = {10.0, 13.0, -6.0, 12.0, -1.4, -4.2, -0.7, -0.7}
Dr. M. Jahanur Rahman, R.U. 85
Multi-level Decomposition of a Series with Wavelets
Time Series1, 3, 5, 11, 12, 13, 0, 1
cA32.8, 11.3, 17.6, 0.7
cD3-1.4, -4.2, -0.7, -0.7
cA210.0, 13.0
cD2-6.0, 12.0
cA016.26
cD0-2.12
Level-1 orScale-1
Level-2 orScale-2
Level-3 orScale-3
- - - -
Wavelet coefficients = {16.2, -2.1, -6.0, 12.0, -1.4, -4.2, -0.7, -0.7}
Dr. M. Jahanur Rahman, R.U. 86
Multi-level Decomposition of a Series with Wavelets
Dr. M. Jahanur Rahman, R.U. 87
DWT Analysis
IDWT Reconstruction
Discrete Wavelet Transform (IDWT).
The input can be reconstructed using the Inverse
Frequency Domain Application-1 Fourier Transformation
Spectral density
Determining periods
Removing Periodicities
Modeling and Forecasting
Dr. M. Jahanur Rahman, R.U. 89
Autocorrelation function or correlogram is used for analyzing time series in time domain.
Time series rk
k
Correlogram
t
Yt
Periodic process with noise
Why Frequency Domain?
Dr. M. Jahanur Rahman, R.U. 90
The steps of analyzing the data
Plot the time series
Plot the correlogram (to suspect the periodicity)
Plot the spectrum (to identify the periodicities)
Ik
wk
rk
k
Yk
t
Dr. M. Jahanur Rahman, R.U. 91
The spectrum shows prominent spikes which represents the periodicities inherent in the data.
The period corresponding the any value of may be computed by
While the correlogram indicate the presence of periodicities in the data, the spectral analysis helps identify the significant periodicities themselves
Periodicities
Dr. M. Jahanur Rahman, R.U. 92
Ik
wk
Ik
wk
Ik
wk
Ik
wk
Reconstruct new series by
removing the significance
periodicity one after another as
Removing Periodicities
Dr. M. Jahanur Rahman, R.U. 93
The periodicities are tested for signoficance by defining the
following statistics (Kashyap and Rao 1976)
Where
Statistical significance of periodicities
Dr. M. Jahanur Rahman, R.U. 94
Removing Periodicities
Significant periodicities are removed from the original time series to get a new series , where
The series containing the previous periodicities
where d is no. of periodicities removed (which are known to be significant)
The spectrum of new series is plotted and the spikes are observed until the series become random.
Dr. M. Jahanur Rahman, R.U. 95
Example A monthly time series over the period 1987-2011 is used. Objective: Modeling and forecasting
Time Series ACF
Periodogram PACF
Dr. M. Jahanur Rahman, R.U. 96
Spectral density shows tree significant periodic patterns with frequencies at
with corresponding periods 30 months 12 months 06 months respectively.
Correlogram & SpectrumCorrelogram indicates the presence
of periodicity.
ACF
Periodogram
Dr. M. Jahanur Rahman, R.U. 97
One-period removed series
Two-period removed series
Three-period removed series
Correlogram & Spectrum
Dr. M. Jahanur Rahman, R.U. 98
Model Selection
Seasonally differenced seriesARIMA(2,0,0) = AR(2)
Periodicities removed seriesARIMA(2,0,0) = AR(2)
PAC
PACAC
AC
Dr. M. Jahanur Rahman, R.U. 99
Trail data: January, 1964 – December, 2007Test data: January, 2008 – December, 2008
Forecasting
Dr. M. Jahanur Rahman, R.U. 100
Comparison of Forecasting PerformanceMean Absolute Error (MAE)Root Mean Squared Error (RMSE)Mean Absolute Percentage Error (MAPE)
Lower-value implies best forecast
Dr. M. Jahanur Rahman, R.U. 101
seasonally differenced series Parameters are significant R-square is high DW-statistic is near 2
Periodicities removed series Parameters are significant R-square is moderate DW-statistic is about 2
EstimationAR(2) model
Dr. M. Jahanur Rahman, R.U. 102
Forecasting
RMSE MAE PR series 1.173 4.739D series 0.852 3.914
Dr. M. Jahanur Rahman, R.U. 103
Decomposition
Discrete Wavelet Transformation
Modeling & Forecasting
Humidity & Maximum Temperature of Rajshahi
Data obtained from BARC
Frequency Domain Application-2
Dr. M. Jahanur Rahman, R.U. 104
Monthly Humidity of Rajshahi(1964-2008)
Dr. M. Jahanur Rahman, R.U. 105
Humidity y(t)
A1
A2
A3
A4
D1
D2
D3
D4
Y = A1 + D1
Y = A2 + D2 +D1
Y = A3 + D3 + D2 +D1
Y = A4 + D4 + D3 + D2 +D1
Decom
positi
on by
wavele
t
Trans
form
ation
Monthly humidity of Rajshahi over 1964-2008
Dr. M. Jahanur Rahman, R.U. 106
DecompositionLevel =3 Wavelet : Doubasis-5
Monthly humidity of Rajshahi over 1964-2008
Dr. M. Jahanur Rahman, R.U. 107
Input series
Wavelet Transformatio
nForecasting
Inverse Wavelet
Transformation
Output series
ForecastingInput series
Wavelet Transformatio
n Processing
Inverse Wavelet
Transformation
Output series
Method-1
Method-2
Forecasting Framework
Dr. M. Jahanur Rahman, R.U. 108
Forecasting Framework
Proposed by: Rocha, T. et.al. (2010)
Dr. M. Jahanur Rahman, R.U. 109
Method-1Step-1: The wavelet transformation of type Daubechies-5 and decomposition level 3 is applied to the series ( result in 4 series denoted by and .
Step-2: Use a specific ARIMA model of each one of the constitutive series to forcast its future values, that is,
Step-3: The inverse wavelet transform is used to reconstruct the forecast series as
Dr. M. Jahanur Rahman, R.U. 110
Method-2Step-1: The wavelet transformation of type Daubechies-5 and decomposition level 3 is applied to the series ( result in 4 series denoted by and .
Step-2: Reconstruct the series by removing the high frequency component
Step-3: Use the appropriate ARIMA model to the reconstructed series to forecast the future values;
Dr. M. Jahanur Rahman, R.U. 111
Trail data: January, 1964 – December, 2007
Test data: January, 2008 – December, 2008
Forecasting of Humidity
Dr. M. Jahanur Rahman, R.U. 112
Comparison of Forecasting PerformanceMean Absolute Error (MAE)Root Mean Squared Error (RMSE)Mean Absolute Percentage Error (MAPE)
Lower-value implies best forecast
Dr. M. Jahanur Rahman, R.U. 113
Method MAE RMSE MAPE
Time domain Method 0.1633 0.2187 18.566
Frequency Domain Method-1
0.1336 0.1591 15.3654
Frequency Domain Method-2
0.0654 0.0799 9.1472
Forecasting performance over Jan-2008 to Dec-2008
Forecasting of HumidityModels
Time Domain: ARIMA(0,1,1)(0,1,1)
Frequency Domain-1: ARIMA(8,2,8)(0,0,1), ARIMA(0,1,2)(0,1,1),
ARIMA(0,0,4)(0,1,0), ARIMA(2,0,2)(0,0,0)
Frequency Domain-2: ARIMA(2,1,1)(0,1,1)
Dr. M. Jahanur Rahman, R.U. 114
Frequency Domain Method-2 Time Domain Method
Method MAE RMSE MAPE
Time domain Method 0.1633 0.2187 18.566
Frequency Domain Method-1
0.16835 0.2296 19.001
Frequency Domain Method-2
0.0654 0.0799 9.1472
Forecasting of Humidity
Dr. M. Jahanur Rahman, R.U. 115
Monthly Maximum Temperature of Rajshahi(1964-2009)
Trail data: Jan, 1964 – Dec, 2008
Test data: Jan, 2009 – Dec, 2009
Dr. M. Jahanur Rahman, R.U. 116
Forecasting of Maximum Temperature
Monthly humidity of Rajshahi over 1964-2008
Method MAE RMSE MAPE
Time domain Method 0.7339 1.0507 2.2315
Frequency Domain Method-1
Frequency Domain Method-2
0.6995 0.8286 2.1402
Models
Time Domain: ARIMA(0,0,1)(0,1,1)
Frequency Domain-1: ARIMA(1,2,8)(0,0,1), ARIMA(4,1,4)(0,1,1),
ARIMA(0,0,6)(0,1,1), ARIMA(1,0,4)(0,1,1)
Frequency Domain-2: ARIMA(0,0,1)(0,1,1)
Dr. M. Jahanur Rahman, R.U. 117
Forecasting of Maximum Temperature
Method MAE RMSE MAPE
Time domain Method 0.7339 1.0507 2.2315
Frequency Domain Method-1
Frequency Domain Method-2
0.6995 0.8286 2.1402
Dr. M. Jahanur Rahman, R.U. 118
In modeling time series both time domain and
frequency domain analysis should be done
together.
For high frequency and noisy data, frequency
domain analysis performs better.
Recommendation
Dr. M. Jahanur Rahman, R.U. 119
Than
ks