Forecasting without forecasters

transcript

Rob J Hyndman

Forecasting withoutforecasters

Outline

1 Motivation

2 Exponential smoothing

3 ARIMA modelling

4 Time series with complex seasonality

5 Hierarchical and grouped time series

6 Functional time series

Forecasting without forecasters Motivation 2

Motivation

1 Common in business to have over 1000products that need forecasting at least monthly.

2 Forecasts are often required by people who areuntrained in time series analysis.

3 Some types of data can be decomposed into alarge number of univariate time series thatneed to be forecast.

Specifications

Automatic forecasting algorithms must:

å determine an appropriate time series model;

å estimate the parameters;

å compute the forecasts with prediction intervals.Forecasting without forecasters Motivation 4

Motivation

Specifications

Motivation

Specifications

Motivation

Specifications

Motivation

Specifications

Example: Asian sheep

Numbers of sheep in Asia

1960 1970 1980 1990 2000 2010

Example: Asian sheep

Automatic ETS forecasts

1960 1970 1980 1990 2000 2010

Example: Cortecosteroid sales

Monthly cortecosteroid drug sales in Australia

1995 2000 2005 2010

Example: Cortecosteroid sales

Automatic ARIMA forecasts

1995 2000 2005 2010

M3 competition

3003 time series.

Early comparison of automatic forecastingalgorithms.

Best-performing methods undocumented.

Limited subsequent research on generalautomatic forecasting algorithms.

M3 competition

3003 time series.

M3 competition

3003 time series.

M3 competition

3003 time series.

Outline

1 Motivation

3 ARIMA modelling

Forecasting without forecasters Exponential smoothing 8

Exponential smoothing

Classic Reference

Makridakis, Wheelwright andHyndman (1998) Forecasting:methods and applications, 3rded., Wiley: NY.

Classic Reference

å “Unfortunately, exponential smoothingmethods do not allow the easy calculation ofprediction intervals.” (MWH, p.177)

å No satisfactory way to select an exponentialsmoothing method.

Classic Reference

å “Unfortunately, exponential smoothingmethods do not allow the easy calculation ofprediction intervals.” (MWH, p.177)

å No satisfactory way to select an exponentialsmoothing method.

Classic Reference

Current Reference

Hyndman and Athanasopoulos(2013) Forecasting: principlesand practice, OTexts: Australia.OTexts.com/fpp.

Exponential smoothing methods

Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

M (Multiplicative) M,N M,A M,M

Md (Multiplicative damped) Md,N Md,A Md,M

N,N: Simple exponential smoothing

N,N: Simple exponential smoothingA,N: Holt’s linear method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ methodA,M: Multiplicative Holt-Winters’ method

There are 15 separate exponential smoothingmethods.

There are 15 separate exponential smoothingmethods.Each can have an additive or multiplicativeerror, giving 30 separate models.

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↑

TrendExamples:

A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

General notation E T S : ExponenTial Smoothing↑ ↖

Trend SeasonalExamples:

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:

Innovations state space models

å All ETS models can be written in innovationsstate space form (IJF, 2002).

å Additive and multiplicative versions give thesame point forecasts but different predictionintervals.

Automatic forecasting

From Hyndman et al. (IJF, 2002):

Apply each of 30 models that are appropriate tothe data. Optimize parameters and initialvalues using MLE (or some other criterion).

Select best method using AIC:

AIC = −2 log(Likelihood) + 2p

where p = # parameters.

Produce forecasts using best method.

Obtain prediction intervals using underlyingstate space model.

Forecasts from ETS(M,A,N)

1960 1970 1980 1990 2000 2010

fit <- ets(livestock)fcast <- forecast(fit)plot(fcast)

Forecasts from ETS(M,A,N)

1960 1970 1980 1990 2000 2010

Forecasts from ETS(M,Md,M)

1995 2000 2005 2010

fit <- ets(h02)fcast <- forecast(fit)plot(fcast)

Forecasts from ETS(M,Md,M)

1995 2000 2005 2010

M3 comparisons

Method MAPE sMAPE MASE

Theta 17.83 12.86 1.40

ForecastPro 18.00 13.06 1.47

ETS additive 18.58 13.69 1.48

ETS 19.33 13.57 1.59

References

RJ Hyndman, AB Koehler, RD Snyder, andS Grose (2002). “A state space framework forautomatic forecasting using exponentialsmoothing methods”. International Journal ofForecasting 18(3), 439–454.

RJ Hyndman, AB Koehler, JK Ord, and RD Snyder(2008). Forecasting with exponentialsmoothing: the state space approach.Springer-Verlag.

RJ Hyndman and G Athanasopoulos (2013).Forecasting: principles and practice. OTexts.OTexts.com/fpp/.

Outline

1 Motivation

3 ARIMA modelling

Forecasting without forecasters ARIMA modelling 19

ARIMA modelling

Classic Reference

ARIMA modelling

Classic Reference

å “There is such a bewildering variety of ARIMAmodels, it can be difficult to decide which modelis most appropriate for a given set of data.”(MWH, p.347)

Auto ARIMA

Forecasts from ARIMA(0,1,0) with drift

1960 1970 1980 1990 2000 2010

Auto ARIMA

fit <- auto.arima(livestock)fcast <- forecast(fit)plot(fcast)

Forecasts from ARIMA(0,1,0) with drift

1960 1970 1980 1990 2000 2010

Auto ARIMA

Forecasts from ARIMA(3,1,3)(0,1,1)[12]

1995 2000 2005 2010

Auto ARIMA

fit <- auto.arima(h02)fcast <- forecast(fit)plot(fcast)

Forecasts from ARIMA(3,1,3)(0,1,1)[12]

1995 2000 2005 2010

How does auto.arima() work?

A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εt

Need to select appropriate orders p,q,d, andwhether to include c.

Algorithm choices driven by forecast accuracy.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p,q, c by minimising AIC.Use stepwise search to traverse model space,starting with a simple model and consideringnearby variants.

A seasonal ARIMA process

Φ(Bm)φ(B)(1− B)d(1− Bm)Dyt = c + Θ(Bm)θ(B)εt

Need to select appropriate orders p,q,d, P,Q,D, andwhether to include c.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select D using OCSB unit root test.Select p,q, P,Q, c by minimising AIC.Use stepwise search to traverse model space,starting with a simple model and consideringnearby variants.

M3 comparisons

Method MAPE sMAPE MASE

Theta 17.83 12.86 1.40

ForecastPro 18.00 13.06 1.47

BJauto 19.14 13.73 1.55

AutoARIMA 18.98 13.75 1.47

ETS-additive 18.58 13.69 1.48

ETS 19.33 13.57 1.59

ETS-ARIMA 18.17 13.11 1.44Forecasting without forecasters ARIMA modelling 27

M3 conclusions

Simple methods do better.

Exponential smoothing is better than ARIMA.

The best methods are hybrid approaches.

ETS-ARIMA (the simple average of ETS-additiveand AutoARIMA) is the only fully documentedmethod that is comparable to the M3competition winners.

I have an algorithm that does better than all ofthese, but it takes too long to be practical.

M3 conclusions

References

RJ Hyndman and Y Khandakar (2008).“Automatic time series forecasting : theforecast package for R”. Journal of StatisticalSoftware 26(3)

RJ Hyndman (2011). “Major changes to theforecast package”.robjhyndman.com/hyndsight/forecast3/.

RJ Hyndman and G Athanasopoulos (2013).Forecasting: principles and practice. OTexts.OTexts.com/fpp/.

Outline

1 Motivation

3 ARIMA modelling

Forecasting without forecasters Time series with complex seasonality 30

Examples

US finished motor gasoline products

1992 1994 1996 1998 2000 2002 2004

Examples

Number of calls to large American bank (7am−9pm)

5 minute intervals

3 March 17 March 31 March 14 April 28 April 12 May

Examples

Turkish electricity demand

2000 2002 2004 2006 2008

TBATS model

TBATSTrigonometric terms for seasonality

Box-Cox transformations for heterogeneity

ARMA errors for short-term dynamics

Trend (possibly damped)

Seasonal (including multiple and

non-integer periods)

Examples

fit <- tbats(gasoline)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(0.999, {2,2}, 1, {<52.1785714285714,8>})

1995 2000 2005

Examples

fit <- tbats(callcentre)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(1, {3,1}, 0.987, {<169,5>, <845,3>})

5 minute intervals

3 March 17 March 31 March 14 April 28 April 12 May 26 May 9 June

Examples

fit <- tbats(turk)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(0, {5,3}, 0.997, {<7,3>, <354.37,12>, <365.25,4>})

2000 2002 2004 2006 2008 2010

References

Automatic algorithm described inAM De Livera, RJ Hyndman, and RD Snyder(2011). “Forecasting time series with complexseasonal patterns using exponentialsmoothing”. Journal of the American StatisticalAssociation 106(496), 1513–1527.

Slightly improved algorithm implemented inRJ Hyndman (2012). forecast: Forecastingfunctions for time series.cran.r-project.org/package=forecast.

More work required!

Outline

1 Motivation

3 ARIMA modelling

Forecasting without forecasters Hierarchical and grouped time series 37

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Manufacturing product hierarchies

Pharmaceutical sales

Net labour turnover

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Net labour turnover

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Net labour turnover

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Net labour turnover

Hierarchical/grouped time seriesA hierarchical time series is a collection ofseveral time series that are linked together in ahierarchical structure.

Example: Pharmaceutical products are organizedin a hierarchy under the Anatomical TherapeuticChemical (ATC) Classification System.

A grouped time series is a collection of timeseries that are aggregated in a number ofnon-hierarchical ways.

Example: daily numbers of calls to HP call centresare grouped by product type and location of callcentre.

Hierarchical data

Yt : observed aggregate of allseries at time t.

YX,t : observation on series X attime t.

Bt : vector of all series atbottom level in time t.

Hierarchical data

Y t = [Yt, YA,t, YB,t, YC,t]′ =

1 1 11 0 00 1 00 0 1

YA,tYB,tYC,t

Hierarchical data

1 1 11 0 00 1 00 0 1

︸︷︷︸

YA,tYB,tYC,t

Hierarchical data

1 1 11 0 00 1 00 0 1

︸︷︷︸

YA,tYB,tYC,t

︸︷︷︸

Hierarchical data

1 1 11 0 00 1 00 0 1

︸︷︷︸

YA,tYB,tYC,t

︸︷︷︸

BtY t = SBt

Grouped data

YtYA,tYB,tYX,tYY,tYAX,tYAY,tYBX,tYBY,t

1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1

︸︷︷︸

YAX,tYAY,tYBX,tYBY,t

︸︷︷︸

Grouped data

1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1

︸︷︷︸

Grouped data

1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1

︸︷︷︸

Y t = SBt

Forecasts

Key idea: forecast reconciliation

å Ignore structural constraints and forecast everyseries of interest independently.

å Adjust forecasts to impose constraints.

Let Yn(h) be vector of initial forecasts for horizon h,made at time n, stacked in same order as Y t.

Optimal reconciled forecasts:

Yn(h) = S(S′S)−1S′Yn(h)

Forecasts

Independent of covariance structure of hierarchy!

Optimal reconciliation weights are S(S′S)−1S′,independent of data.

Features

Forget “bottom up” or “top down”. Thisapproach combines all forecasts optimally.

Method outperforms bottom-up and top-down,especially for middle levels.

Covariates can be included in base forecasts.

Adjustments can be made to base forecasts atany level.

Point forecasts are always aggregate consistent.

Very simple and flexible method. Can work withany hierarchical or grouped time series.

Conceptually easy to implement: OLS on baseforecasts.Forecasting without forecasters Hierarchical and grouped time series 43

Features

Challenges

Computational difficulties in big hierarchies dueto size of the S matrix and non-singularbehavior of (S′S).

Need to estimate covariance matrix to produceprediction intervals.

Challenges

Computational difficulties in big hierarchies dueto size of the S matrix and non-singularbehavior of (S′S).

Need to estimate covariance matrix to produceprediction intervals.

Example using Rlibrary(hts)

# bts is a matrix containing the bottom level time series# g describes the grouping/hierarchical structurey <- hts(bts, g=c(1,1,2,2))

# Forecast 10-step-ahead using optimal combination method# ETS used for each series by defaultfc <- forecast(y, h=10)

# Select your own methodsally <- allts(y)allf <- matrix(, nrow=10, ncol=ncol(ally))for(i in 1:ncol(ally))

allf[,i] <- mymethod(ally[,i], h=10)allf <- ts(allf, start=2004)# Reconcile forecasts so they add upfc2 <- combinef(allf, Smatrix(y))

References

RJ Hyndman, RA Ahmed, G Athanasopoulos,and HL Shang (2011). “Optimal combinationforecasts for hierarchical time series”.Computational Statistics and Data Analysis55(9), 2579–2589

RJ Hyndman, RA Ahmed, and HL Shang (2013).hts: Hierarchical time series.cran.r-project.org/package=hts.

RJ Hyndman and G Athanasopoulos (2013).Forecasting: principles and practice. OTexts.OTexts.com/fpp/.Forecasting without forecasters Hierarchical and grouped time series 48

Outline

1 Motivation

3 ARIMA modelling

Forecasting without forecasters Functional time series 49

Fertility rates

Functional data modelLet ft,x be the observed data in period t at age x,t = 1, . . . ,n.

ft(x) = µ(x) +K∑

βt,k φk(x) + et(x)

Decomposition separates time and age to allowforecasting.

Estimate µ(x) as mean ft(x) across years.

Estimate βt,k and φk(x) using functional (weighted)principal components.

Univariate models used for automatic forecasting ofscores {βt,k}.

ft(x) = µ(x) +K∑

Fertility application

15 20 25 30 35 40 45 50

Australia fertility rates (1921−2006)

tility

Fertility model

15 20 25 30 35 40 45 50

1920 1960 2000

15 20 25 30 35 40 45 50

1920 1960 2000

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

80% prediction intervals

R code

15 20 25 30 35 40 45 50

tility

library(demography)plot(aus.fert)fit <- fdm(aus.fert)fc <- forecast(fit)

References

RJ Hyndman and S Ullah (2007). “Robustforecasting of mortality and fertility rates: Afunctional data approach”. ComputationalStatistics and Data Analysis 51(10), 4942–4956

RJ Hyndman and HL Shang (2009). “Forecastingfunctional time series (with discussion)”.Journal of the Korean Statistical Society 38(3),199–221

RJ Hyndman (2012). demography: Forecastingmortality, fertility, migration and populationdata.cran.r-project.org/package=demography.

For further information

robjhyndman.com

Slides and references for this talk.

Links to all papers and books.

Links to R packages.

A blog about forecasting research.

Forecasting without forecasters

Technology