Real Time Trend Extraction and Seasonal Adjustment: a Generalized Direct Filter

Approach

ISF 2011, PragueMarc Wildi

Zurich University of Applied SciencesMarc.wildi@zhaw.ch

Signalextraction vs. Forecasting

Signal

X Noisy Data

Filter: a set of weights such that

is `fr

Signa

ee of noise'

is the

Trend, Seasonally Adjusted Component, Cyc

l

l

e

t

k

t k t kk

t

Y X

Y

Filters:

• Ad hoc designs: no explicit modelling of the data – HP-Filter, CF-Filter, BK-Filter, Henderson Filter, …

• Model-based designs– TRAMO/SEATS, X-12-ARIMA, Stamp

• Non-parametric filters (Loess)

• Very general setting!

k

Real-Time SignalextractionTime Domain

1

0

1 2

1 1

0

1 1

0

Y `senses' the (X ,X ,...)

Real-Time Finite Sample

ˆ ˆ

Model-Ba

future

ˆ

ˆ

sed Approaches (MBA):

ˆ

One- a

T k T k T Tk

T k T k

T

k T k k T k kk T

T

k k

T

k T k k

k

k

T k

kT k

X

Y X

X X

X

X

X

nd ahead multi-s foretep casts

Example1

1| |

( 1)

1 1| | | |

0 ( 1)

1 1| | | | | |

0 ( 1)

1 0| | |

1 ( 1)

AR(1) Process :

Filter: sym. exponential weighting

ˆ ˆ

( )

t t t

Tk

t t kk T

Tk k

T T k T kk k T

Tk k k

T k Tk k T

Tk

T kk k T

X aX

Y c X

Y c X c X

c X c a X

c X c a

|

1| |

1

Very cumbersome way to define a one-sided filt

1

er!

kT

Tk

T k Tk

X

cc X X

a

Forecasting

1

Y

1,Forecasting:

0 for k -1

This is a very particular (asymmetric) `Signal' Definition

Model-Based One-step ahead Forecast!

T k T kk

k

X

Frequency Domain

Real-Time SignalextractionFrequency Domain

1

0

1

0

Target: Y

ˆ ˆReal-Time Estimate:

Transferfunctions

( ):= exp( ) ( if symmetric)

ˆ ˆ( ):= exp( )

T k T kk

T k T k

kk

k

T

k

T

k

X

Y X

ik

ik

Example: European IPI

TRAMO/SEATS (Airline-Model in red)

Forecasting

1

( ):= exp( )

1,Forecasting:

0 for k -1

( ) 1*exp( )

( ) is a very particular (allpass) Filter/Transferfunction

Replicates Traditional Model-Based One-step ahead Forecast in F-D!

kk

k

ik

i

Optimization Criterion: Mean-Square

2FILTERWEIGHTS

2FILTERWEIGHTS

2

ˆ

ˆFilter error:

Criterion: E[ ] min

ˆ| ( ) ( ) | ( ) min

Real-World:

ˆˆ( ) ( ) S( ) min

t t t

t

k k kk

r Y Y

r

dS

Choice of Spectral Estimate • Model-based: – TRAMO (airline-model), X-12-ARIMA, state-space

• Ad-hoc: – implicit model (HP, CF, BK, Henderson,…)

• Non Parametric– Periodogram

• This choice is to some extent arbitrary: it depends on the preference/experience/expertise of the user.

• Very general setting!

ˆ( )S

Generalized DFA: Very General Setting!

• Arbitrary signals – Including as a special case traditional one-step ahead

forecasting• Arbitrary finite sample Spectral Estimate– ad hoc, model-based, non-parametric

• Generalizes– Ad hoc filters– Model-based filters– DFA (based on the periodogram)– Traditional (one-step ahead) ARIMA-modelling, state-space

modelling– Extends to multivariate filtering!

Frequency-Domain: Timeliness-Reliability Dilemma

Control of Timeliness/Speed: Cosine Law applied to

ˆ ( )

ˆ( ) ( )

ˆ ( )

( )

2

2

ˆ( ) ( )

ˆ ˆ ˆ( ) ( ) 2 ( ) ( ) 1 cos( ( ))

2ˆ( ) ( )

Timeliness-Criterion

/2 2

1

/2 2

1

/2

1

ˆˆ( ) ( ) ( )

ˆ ˆA( ) A( ) ( )

ˆ ˆˆ2A( )A( ) 1 cos( ( )) ( )

Mean-Square: 1

Faster Filter : >1

Slower Filter: <1

T

k k kk

T

k k kk

T

k k k kk

S

S

S

Emphasize Noise Rejection in Stop Band (Reliability/Smoothness)

/2 2

1

/2

1

ˆ ˆA( ) A( ) ( ) ( )

ˆ ˆˆ2A( )A( ) 1 cos( ( )) ( )

( ) assigns m amplitude in stop band

time-shift in pas

ore weight to

assigns more weigh s t band to

T

k k k kk

T

k k k kk

k

W S

S

W

Essence of Generalized DFA

• The new optimization criterion IS the timeliness-reliability-dilemma and conversely

• `Philosophy’ may be contrasted with – Maximum likelihood (particular parametric setting

lambda/expweight)– Maximum entropy

• Contrast:– Manipulate Real-Time filter characteristics explicitly on

the edge of the fundamental dilemma– User relevant priorities (risk-aversion)

Effect of `Expweight’

Effect of Lambda

Example : European IPI

Replicate TRAMO RT-Performance:TRAMO (red) vs. Gen. DFA (blue)

New Target: Customized Design

• Instead of optimal mean-square estimate the user could specify a `faster’ and/or `smoother’ real-time estimate

• The new estimate is still purely model-based!– It IS TRAMO (it could be X-12, Stamp,…)– But it becomes faster/smoother (timeliness-

reliability dilemma)

Mean-Square vs. Enhanced TRAMO

• Typically, TRAMO-filter (blue) is noisy (poor noise suppression in stop-band)

• The `customized’ filter (green) barely loses in terms of time-shift in the pass-band. It clearly wins in terms of noise suppression in the stop-band: better compromise

TRAMO (red) vs. Enhanced (green)

Conclusion

• As expected, the `customized’ real-time filter (green) is as `fast’ as the MS-filter by TRAMO (red) and it is much smoother (better noise suppression)

SA vs. Customized RT-Trend

• Real-time customized trend filter is as fast as traditional SA-filter and much (much) smoother.

Conclusion

Philosophy Generalized DFA

The new criterion IS the timeliness-reliability dilemma

Consequences• Generalizes classical filter approaches (ad hoc,

model-based)• Emphasizes user relevant priorities explicitly

Practicality

• Numerically (very) fast– Closed-from approximation (I-DFA/open source)– Fast exact optimization (Eurostat/proprietary)

• Short piece of (R-) code– Could easily dock to any existent software/tool

Web:

• SEFblog: http://blog.zhaw.ch/idp/sefblog• USRI: http://www.idp.zhaw.ch/usri • MDFA-XT: http://www.idp.zhaw.ch/MDFA-XT• SEF-page: http://www.idp.zhaw.ch/sef

Selected SEFBlog-Entries

• Forecasting the EURO-BUND-Future (6 months, one Year)– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/

186-Forecasting-the-EURO-Bund-Future-6-months-and-One-Year-Ahead-FirstPreliminary-Draft.html

• OECD-CLI: leading indicator for the US– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/

173-Tutorial-I-MDFA-Part-II-The-OECD-CLI-for-the-US.html

– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/175-Injecting-the-ZPC-Gene-into-I-MDFA-an-Application-to-the-OECD-CLI-for-the-US.html

SEFBlog-Entries

• Algorithmic Trading:– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/

157-A-Generalization-of-the-GARCH-in-Mean-Model-Vola-in-I-MDFA-filter.html

• Tutorials Univariate Filter:– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/

159-I-DFA-Exercises-Part-I-Mean-Square-Criterion.html– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/

160-I-DFA-Exercises-Part-II-Customization-SpeedReliability.html

SEFBlog-Entries

• Tutorials Multivariate Filter:– http://blog.zhaw.ch/idp/sefblog/index.php?/

archives/172-Tutorial-I-MDFA-Part-I-Simulated-Time-Series.html

– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/173-Tutorial-I-MDFA-Part-II-The-OECD-CLI-for-the-US.html