107
Forecasting: principles and practice 1 Forecasting: principles and practice Rob J Hyndman 3.3 Hierarchical forecasting
Forecasting: principles and practice 1
Forecasting: principlesand practice
Rob J Hyndman
3.3 Hierarchical forecasting
Outline
1 Hierarchical and grouped time series
2 Lab session 15
3 Temporal hierarchies
4 Lab session 16
Forecasting: principles and practice Hierarchical and grouped time series 2
Australian tourism demand
Forecasting: principles and practice Hierarchical and grouped time series 3
Australian tourism demand
Forecasting: principles and practice Hierarchical and grouped time series 3
Quarterly data on visitor night from 1998:Q1 –2013:Q4From: National Visitor Survey, based on annualinterviews of 120,000 Australians aged 15+,collected by Tourism Research Australia.Split by 7 states, 27 zones and 76 regions (ageographical hierarchy)Also split by purpose of travel
HolidayVisiting friends and relatives (VFR)BusinessOther
304 bottom-level series
Spectacle sales
Forecasting: principles and practice Hierarchical and grouped time series 4
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range (6),materials (4), and stores (600)About 1 million bottom-level series
Spectacle sales
Forecasting: principles and practice Hierarchical and grouped time series 4
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range (6),materials (4), and stores (600)About 1 million bottom-level series
Spectacle sales
Forecasting: principles and practice Hierarchical and grouped time series 4
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range (6),materials (4), and stores (600)About 1 million bottom-level series
Spectacle sales
Forecasting: principles and practice Hierarchical and grouped time series 4
Monthly UK sales data from 2000 – 2014Provided by a large spectacle manufacturerSplit by brand (26), gender (3), price range (6),materials (4), and stores (600)About 1 million bottom-level series
Hierarchical time seriesA hierarchical time series is a collection of several timeseries that are linked together in a hierarchical structure.
Total
A
AA AB AC
B
BA BB BC
C
CA CB CC
ExamplesTourism by state and region
Forecasting: principles and practice Hierarchical and grouped time series 5
Hierarchical time seriesA hierarchical time series is a collection of several timeseries that are linked together in a hierarchical structure.
Total
A
AA AB AC
B
BA BB BC
C
CA CB CC
ExamplesTourism by state and region
Forecasting: principles and practice Hierarchical and grouped time series 5
Grouped time seriesA grouped time series is a collection of time series thatcan be grouped together in a number of non-hierarchicalways.
Total
A
AX AY
B
BX BY
Total
X
AX BX
Y
AY BY
ExamplesLabour turnover by occupation and stateSpectacle sales by brand, gender, stores, etc.
Forecasting: principles and practice Hierarchical and grouped time series 6
Grouped time seriesA grouped time series is a collection of time series thatcan be grouped together in a number of non-hierarchicalways.
Total
A
AX AY
B
BX BY
Total
X
AX BX
Y
AY BY
ExamplesLabour turnover by occupation and stateSpectacle sales by brand, gender, stores, etc.
Forecasting: principles and practice Hierarchical and grouped time series 6
Grouped time seriesA grouped time series is a collection of time series thatcan be grouped together in a number of non-hierarchicalways.
Total
A
AX AY
B
BX BY
Total
X
AX BX
Y
AY BY
ExamplesLabour turnover by occupation and stateSpectacle sales by brand, gender, stores, etc.
Forecasting: principles and practice Hierarchical and grouped time series 6
The problem
1 How to forecast time series at all nodessuch that the forecasts add up in the sameway as the original data?
2 Can we exploit relationships between theseries to improve the forecasts?
Forecasting: principles and practice Hierarchical and grouped time series 7
The problem
1 How to forecast time series at all nodessuch that the forecasts add up in the sameway as the original data?
2 Can we exploit relationships between theseries to improve the forecasts?
Forecasting: principles and practice Hierarchical and grouped time series 7
The solution
1 Forecast all series at all levels of aggregationusing an automatic forecasting algorithm(e.g., ets, auto.arima, . . . )
2 Reconcile the resulting forecasts so they addup correctly using least squaresoptimization (i.e., find closest reconciledforecasts to the original forecasts).
3 This is all available in the hts package in R.
Forecasting: principles and practice Hierarchical and grouped time series 8
The solution
1 Forecast all series at all levels of aggregationusing an automatic forecasting algorithm(e.g., ets, auto.arima, . . . )
2 Reconcile the resulting forecasts so they addup correctly using least squaresoptimization (i.e., find closest reconciledforecasts to the original forecasts).
3 This is all available in the hts package in R.
Forecasting: principles and practice Hierarchical and grouped time series 8
The solution
1 Forecast all series at all levels of aggregationusing an automatic forecasting algorithm(e.g., ets, auto.arima, . . . )
2 Reconcile the resulting forecasts so they addup correctly using least squaresoptimization (i.e., find closest reconciledforecasts to the original forecasts).
3 This is all available in the hts package in R.
Forecasting: principles and practice Hierarchical and grouped time series 8
hts package for R
Forecasting: principles and practice Hierarchical and grouped time series 9
hts: Hierarchical and Grouped Time SeriesMethods for analysing and forecasting hierarchical and grouped time seriesVersion: 5.0Depends: R (≥ 3.0.2), forecast (≥ 5.0), SparseM, Matrix, matrixcalcImports: parallel, utils, methods, graphics, grDevices, statsLinkingTo: Rcpp (≥ 0.11.0), RcppEigenSuggests: testthatPublished: 2016-04-06Author: Rob J Hyndman, Earo Wang, Alan Lee, Shanika WickramasuriyaMaintainer: Rob J Hyndman<Rob.Hyndman at monash.edu>BugReports: https://github.com/robjhyndman/hts/issuesLicense: GPL (≥ 2)
Example using Rlibrary(hts)
# bts is a matrix containing the bottom level time series# nodes describes the hierarchical structurey <- hts(bts, nodes=list(2, c(3,2)))
Forecasting: principles and practice Hierarchical and grouped time series 10
Example using Rlibrary(hts)
# bts is a matrix containing the bottom level time series# nodes describes the hierarchical structurey <- hts(bts, nodes=list(2, c(3,2)))
Forecasting: principles and practice Hierarchical and grouped time series 10
Total
A
AX AY AZ
B
BX BY
Example using Rlibrary(hts)
# bts is a matrix containing the bottom level time series# nodes describes the hierarchical structurey <- hts(bts, nodes=list(2, c(3,2)))
# Forecast 10-step-ahead using WLS combination method# ETS used for each series by defaultfc <- forecast(y, h=10)
Forecasting: principles and practice Hierarchical and grouped time series 11
Total
A
AX AY AZ
B
BX BY
gts functionUsagegts(y, characters)
Argumentsy Multivariate time series containing the bottom
level seriescharacters Vector of integers, or list of vectors, showing
how column names indicate group structure.Examplebnames <-c("VIC1F","VIC1M","VIC2F","VIC2M","VIC3F","VIC3M","NSW1F","NSW1M","NSW2F","NSW2M","NSW3F","NSW3M")
bts <- matrix(ts(rnorm(120)), ncol = 12)colnames(bts) <- bnamesx <- gts(bts, characters = c(3, 1, 1))
Forecasting: principles and practice Hierarchical and grouped time series 12
gts function
Example 2
bnames <-
c("VICMelbAA","VICMelbAB",
"VICGeelAA","VICGeelAB",
"VICMelbBA","VICMelbBB",
"VICGeelBA","VICGeelBB",
"NSWSyndAA","NSWSyndAB",
"NSWWollAA","NSWWollAB",
"NSWSyndBA","NSWSyndBB",
"NSWWollBA","NSWWollBB")
bts <- matrix(ts(rnorm(160)), ncol = 16)
colnames(bts) <- bnames
x <- gts(bts, characters = list(c(3, 4), c(1, 1)))Forecasting: principles and practice Hierarchical and grouped time series 13
forecast.gts functionUsageforecast(object, h,method = c("comb", "bu", "mo","tdgsa", "tdgsf", "tdfp"),weights = c("wls", "ols", "mint", "nseries"),fmethod = c("ets", "arima", "rw"),algorithms = c("lu", "cg", "chol", "recursive", "slm"),covariance = c("shr", "sam"),positive = FALSE,parallel = FALSE, num.cores = 2, ...)
Argumentsobject Hierarchical time series object of class gts.h Forecast horizonmethod Method for distributing forecasts within the hierarchy.weights Weights used for “optimal combination" method. When weights =
“sd”, it takes account of the standard deviation of forecasts.fmethod Forecasting method to usealgorithm Method for solving regression equationspositive If TRUE, forecasts are forced to be strictly positiveparallel If TRUE, allow parallel processingnum.cores If parallel = TRUE, specify how many cores are going to be used
Forecasting: principles and practice Hierarchical and grouped time series 14
Example: Australian tourism
Forecasting: principles and practice Hierarchical and grouped time series 15
Example: Australian tourism
Forecasting: principles and practice Hierarchical and grouped time series 15
Hierarchy:States (7)Zones (27)Regions (82)
Example: Australian tourism
Forecasting: principles and practice Hierarchical and grouped time series 15
Hierarchy:States (7)Zones (27)Regions (82)
Base forecastsETS (exponential smoothing)models
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: Total
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
6000
065
000
7000
075
000
8000
085
000
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: NSW
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
1800
022
000
2600
030
000
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: VIC
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
1000
012
000
1400
016
000
1800
0
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: Nth.Coast.NSW
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
5000
6000
7000
8000
9000
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: Metro.QLD
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
8000
9000
1100
013
000
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: Sth.WA
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
400
600
800
1000
1200
1400
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: X201.Melbourne
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
4000
4500
5000
5500
6000
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: X402.Murraylands
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
010
020
030
0
Base forecasts
Forecasting: principles and practice Hierarchical and grouped time series 16
Domestic tourism forecasts: X809.Daly
Year
Vis
itor
nigh
ts
1998 2000 2002 2004 2006 2008
020
4060
8010
0
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 1● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 2● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 3● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 4● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 5● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Forecast evaluation
Forecasting: principles and practice Hierarchical and grouped time series 17
Training sets Test sets h = 6● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
time
Hierarchy: states, zones, regionsForecast horizon
RMSE h = 1 h = 2 h = 3 h = 4 h = 5 h = 6 Ave
AustraliaBase 1762.04 1770.29 1766.02 1818.82 1705.35 1721.17 1757.28Bottom 1736.92 1742.69 1722.79 1752.74 1666.73 1687.43 1718.22OLS 1747.60 1757.68 1751.77 1800.67 1686.00 1706.45 1741.69WLS 1705.21 1715.87 1703.75 1729.56 1627.79 1661.24 1690.57GLS 1704.64 1715.60 1705.31 1729.04 1626.36 1661.64 1690.43
StatesBase 399.77 404.16 401.92 407.26 395.38 401.17 401.61Bottom 404.29 406.95 404.96 409.02 399.80 401.55 404.43OLS 404.47 407.62 405.43 413.79 401.10 404.90 406.22WLS 398.84 402.12 400.71 405.03 394.76 398.23 399.95GLS 398.84 402.16 400.86 405.03 394.59 398.22 399.95
RegionsBase 93.15 93.38 93.45 93.79 93.50 93.56 93.47Bottom 93.15 93.38 93.45 93.79 93.50 93.56 93.47OLS 93.28 93.53 93.64 94.17 93.78 93.88 93.71WLS 93.02 93.32 93.38 93.72 93.39 93.53 93.39GLS 92.98 93.27 93.34 93.66 93.34 93.46 93.34
Forecasting: principles and practice Hierarchical and grouped time series 18
Hierarchical time series
Total
A B C
Forecasting: principles and practice Hierarchical and grouped time series 19
yt : observed aggregate of allseries at time t.
yX,t : observation on series X at timet.
bt : vector of all series at bottomlevel in time t.
Hierarchical time series
Total
A B C
Forecasting: principles and practice Hierarchical and grouped time series 19
yt : observed aggregate of allseries at time t.
yX,t : observation on series X at timet.
bt : vector of all series at bottomlevel in time t.
Hierarchical time series
Total
A B C
yt = [yt, yA,t, yB,t, yC,t]′ =
1 1 11 0 00 1 00 0 1
yA,tyB,tyC,t
Forecasting: principles and practice Hierarchical and grouped time series 19
yt : observed aggregate of allseries at time t.
yX,t : observation on series X at timet.
bt : vector of all series at bottomlevel in time t.
Hierarchical time series
Total
A B C
yt = [yt, yA,t, yB,t, yC,t]′ =
1 1 11 0 00 1 00 0 1
︸ ︷︷ ︸
S
yA,tyB,tyC,t
Forecasting: principles and practice Hierarchical and grouped time series 19
yt : observed aggregate of allseries at time t.
yX,t : observation on series X at timet.
bt : vector of all series at bottomlevel in time t.
Hierarchical time series
Total
A B C
yt = [yt, yA,t, yB,t, yC,t]′ =
1 1 11 0 00 1 00 0 1
︸ ︷︷ ︸
S
yA,tyB,tyC,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 19
yt : observed aggregate of allseries at time t.
yX,t : observation on series X at timet.
bt : vector of all series at bottomlevel in time t.
Hierarchical time series
Total
A B C
yt = [yt, yA,t, yB,t, yC,t]′ =
1 1 11 0 00 1 00 0 1
︸ ︷︷ ︸
S
yA,tyB,tyC,t
︸ ︷︷ ︸
btyt = Sbt
Forecasting: principles and practice Hierarchical and grouped time series 19
yt : observed aggregate of allseries at time t.
yX,t : observation on series X at timet.
bt : vector of all series at bottomlevel in time t.
Hierarchical time seriesTotal
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
ytyA,tyB,tyC,tyAX,tyAY,tyAZ,tyBX,tyBY,tyBZ,tyCX,tyCY,tyCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
yAX,tyAY,tyAZ,tyBX,tyBY,tyBZ,tyCX,tyCY,tyCZ,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 20
Hierarchical time seriesTotal
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
ytyA,tyB,tyC,tyAX,tyAY,tyAZ,tyBX,tyBY,tyBZ,tyCX,tyCY,tyCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
yAX,tyAY,tyAZ,tyBX,tyBY,tyBZ,tyCX,tyCY,tyCZ,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 20
Hierarchical time seriesTotal
A
AX AY AZ
B
BX BY BZ
C
CX CY CZ
yt =
ytyA,tyB,tyC,tyAX,tyAY,tyAZ,tyBX,tyBY,tyBZ,tyCX,tyCY,tyCZ,t
=
1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1
︸ ︷︷ ︸
S
yAX,tyAY,tyAZ,tyBX,tyBY,tyBZ,tyCX,tyCY,tyCZ,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 20
yt = Sbt
Grouped dataAX AY A
BX BY B
X Y Total
yt =
ytyA,tyB,tyX,tyY,tyAX,tyAY,tyBX,tyBY,t
=
1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1
︸ ︷︷ ︸
S
yAX,tyAY,tyBX,tyBY,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 21
Grouped dataAX AY A
BX BY B
X Y Total
yt =
ytyA,tyB,tyX,tyY,tyAX,tyAY,tyBX,tyBY,t
=
1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1
︸ ︷︷ ︸
S
yAX,tyAY,tyBX,tyBY,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 21
Grouped dataAX AY A
BX BY B
X Y Total
yt =
ytyA,tyB,tyX,tyY,tyAX,tyAY,tyBX,tyBY,t
=
1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1
︸ ︷︷ ︸
S
yAX,tyAY,tyBX,tyBY,t
︸ ︷︷ ︸
bt
Forecasting: principles and practice Hierarchical and grouped time series 21
yt = Sbt
Hierarchical and grouped time series
Every collection of time series with aggregationconstraints can be written as
yt = Sbt
whereyt is a vector of all series at time tbt is a vector of the most disaggregated series at timetS is a “summing matrix” containing the aggregationconstraints.
Forecasting: principles and practice Hierarchical and grouped time series 22
Forecasting notation
Let yn(h) be vector of initial h-step forecasts, made attime n, stacked in same order as yt.(In general, they will not “add up”.)
Reconciled forecasts must be of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecasts yn(h) to getbottom-level forecasts.S adds them up
Forecasting: principles and practice Hierarchical and grouped time series 23
Forecasting notation
Let yn(h) be vector of initial h-step forecasts, made attime n, stacked in same order as yt.(In general, they will not “add up”.)
Reconciled forecasts must be of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecasts yn(h) to getbottom-level forecasts.S adds them up
Forecasting: principles and practice Hierarchical and grouped time series 23
Forecasting notation
Let yn(h) be vector of initial h-step forecasts, made attime n, stacked in same order as yt.(In general, they will not “add up”.)
Reconciled forecasts must be of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecasts yn(h) to getbottom-level forecasts.S adds them up
Forecasting: principles and practice Hierarchical and grouped time series 23
Forecasting notation
Let yn(h) be vector of initial h-step forecasts, made attime n, stacked in same order as yt.(In general, they will not “add up”.)
Reconciled forecasts must be of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecasts yn(h) to getbottom-level forecasts.S adds them up
Forecasting: principles and practice Hierarchical and grouped time series 23
Forecasting notation
Let yn(h) be vector of initial h-step forecasts, made attime n, stacked in same order as yt.(In general, they will not “add up”.)
Reconciled forecasts must be of the form:yn(h) = SPyn(h)
for some matrix P.
P extracts and combines base forecasts yn(h) to getbottom-level forecasts.S adds them up
Forecasting: principles and practice Hierarchical and grouped time series 23
Optimal combination forecastsMain resultThe best (minimum sum of variances) unbiased forecastsare obtained when P = (S′Σ−1h S)−1S′Σ−1h , whereΣh isthe h-step base forecast error covariance matrix.
yn(h) = S(S′Σ−1h S)−1S′Σ−1h yn(h)
Reconciled forecasts Base forecastsProblem: Σh hard to estimate, especially for h > 1.Solutions:
IgnoreΣh (OLS)AssumeΣh diagonal (WLS) [Default in hts]Try to estimateΣh (GLS)Forecasting: principles and practice Hierarchical and grouped time series 24
Optimal combination forecastsMain resultThe best (minimum sum of variances) unbiased forecastsare obtained when P = (S′Σ−1h S)−1S′Σ−1h , whereΣh isthe h-step base forecast error covariance matrix.
yn(h) = S(S′Σ−1h S)−1S′Σ−1h yn(h)
Reconciled forecasts Base forecastsProblem: Σh hard to estimate, especially for h > 1.Solutions:
IgnoreΣh (OLS)AssumeΣh diagonal (WLS) [Default in hts]Try to estimateΣh (GLS)Forecasting: principles and practice Hierarchical and grouped time series 24
Optimal combination forecastsMain resultThe best (minimum sum of variances) unbiased forecastsare obtained when P = (S′Σ−1h S)−1S′Σ−1h , whereΣh isthe h-step base forecast error covariance matrix.
yn(h) = S(S′Σ−1h S)−1S′Σ−1h yn(h)
Reconciled forecasts Base forecastsProblem: Σh hard to estimate, especially for h > 1.Solutions:
IgnoreΣh (OLS)AssumeΣh diagonal (WLS) [Default in hts]Try to estimateΣh (GLS)Forecasting: principles and practice Hierarchical and grouped time series 24
Outline
1 Hierarchical and grouped time series
2 Lab session 15
3 Temporal hierarchies
4 Lab session 16
Forecasting: principles and practice Lab session 15 25
Lab Session 15
Forecasting: principles and practice Lab session 15 26
Outline
1 Hierarchical and grouped time series
2 Lab session 15
3 Temporal hierarchies
4 Lab session 16
Forecasting: principles and practice Temporal hierarchies 27
Temporal hierarchies
Annual
Semi-Annual1
Q1 Q2
Semi-Annual2
Q3 Q4
Basic idea:å Forecast series at each available frequency.å Optimally reconcile forecasts within the same year.
Forecasting: principles and practice Temporal hierarchies 28
Temporal hierarchies
Annual
Semi-Annual1
Q1 Q2
Semi-Annual2
Q3 Q4
Basic idea:å Forecast series at each available frequency.å Optimally reconcile forecasts within the same year.
Forecasting: principles and practice Temporal hierarchies 28
Monthly seriesAnnual
Semi-Annual1
Q1
M1 M2 M3
Q2
M4 M5 M6
Semi-Annual2
Q3
M7 M8 M9
Q4
M10 M11 M12
k = 2, 4, 12 nodesk = 3, 6, 12 nodesWhy not k = 2, 3, 4, 6, 12 nodes?Forecasting: principles and practice Temporal hierarchies 29
Monthly seriesAnnual
FourM1
BiM1
M1 M2
BiM2
M3 M4
FourM2
BiM3
M5 M6
BiM4
M7 M8
FourM3
BiM5
M9 M10
BiM6
M11 M12
k = 2, 4, 12 nodesk = 3, 6, 12 nodesWhy not k = 2, 3, 4, 6, 12 nodes?Forecasting: principles and practice Temporal hierarchies 29
Monthly seriesAnnual
FourM1
BiM1
M1 M2
BiM2
M3 M4
FourM2
BiM3
M5 M6
BiM4
M7 M8
FourM3
BiM5
M9 M10
BiM6
M11 M12
k = 2, 4, 12 nodesk = 3, 6, 12 nodesWhy not k = 2, 3, 4, 6, 12 nodes?Forecasting: principles and practice Temporal hierarchies 29
Monthly data
ASemiA1SemiA2FourM1FourM2FourM3Q1...Q4BiM1...
BiM6M1...
M12
︸ ︷︷ ︸
(28×1)
=
1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 0 0 0 0 0 00 0 0 0 0 0 1 1 1 1 1 11 1 1 1 0 0 0 0 0 0 0 00 0 0 0 1 1 1 1 0 0 0 00 0 0 0 0 0 0 0 1 1 1 11 1 1 0 0 0 0 0 0 0 0 0
...0 0 0 0 0 0 0 0 0 1 1 11 1 0 0 0 0 0 0 0 0 0 0
...0 0 0 0 0 0 0 0 0 0 1 1
I12
︸ ︷︷ ︸
S
M1
M2
M3
M4
M5
M6
M7
M8
M9
M10
M11
M12
︸ ︷︷ ︸
Bt
Forecasting: principles and practice Temporal hierarchies 30
In general
For a time series y1, . . . , yT, observed at frequencym, wegenerate aggregate series
y[k]j =
jk∑t=1+(j−1)k
yt, for j = 1, . . . , bT/kc
k ∈ F(m) = {factors ofm}.A single unique hierarchy is only possible when thereare no coprime pairs in F(m).Mk = m/k is seasonal period of aggregated series.
Forecasting: principles and practice Temporal hierarchies 31
In general
For a time series y1, . . . , yT, observed at frequencym, wegenerate aggregate series
y[k]j =
jk∑t=1+(j−1)k
yt, for j = 1, . . . , bT/kc
k ∈ F(m) = {factors ofm}.A single unique hierarchy is only possible when thereare no coprime pairs in F(m).Mk = m/k is seasonal period of aggregated series.
Forecasting: principles and practice Temporal hierarchies 31
In general
For a time series y1, . . . , yT, observed at frequencym, wegenerate aggregate series
y[k]j =
jk∑t=1+(j−1)k
yt, for j = 1, . . . , bT/kc
k ∈ F(m) = {factors ofm}.A single unique hierarchy is only possible when thereare no coprime pairs in F(m).Mk = m/k is seasonal period of aggregated series.
Forecasting: principles and practice Temporal hierarchies 31
UK Accidents and Emergency Demand
Forecasting: principles and practice Temporal hierarchies 32
1 2 3 4 5 6
5100
5300
5500
Annual (k=52)
Forecast
2 4 6 8 10 1225
0026
0027
0028
0029
00
Semi−annual (k=26)
Forecast
5 10 15 20 25
1250
1350
1450
Quarterly (k=13)
Forecast
20 40 60 80
360
380
400
420
440
460
Monthly (k=4)
Forecast
50 100 150
180
190
200
210
220
230
Bi−weekly (k=2)
Forecast
50 100 150 200 250 300
9095
100
105
110
Weekly (k=1)
Forecast
– – – – base reconciled
UK Accidents and Emergency Demand
1 Type 1 Departments —Major A&E2 Type 2 Departments — Single Specialty3 Type 3 Departments — Other A&E/Minor Injury4 Total Attendances5 Type 1 Departments —Major A&E > 4 hrs6 Type 2 Departments — Single Specialty > 4 hrs7 Type 3 Departments — Other A&E/Minor Injury > 4 hrs8 Total Attendances > 4 hrs9 Emergency Admissions via Type 1 A&E10 Total Emergency Admissions via A&E11 Other Emergency Admissions (i.e., not via A&E)12 Total Emergency Admissions13 Number of patients spending > 4 hrs from decision to
admissionForecasting: principles and practice Temporal hierarchies 33
UK Accidents and Emergency Demand
Minimum training set: all data except the last yearBase forecasts using auto.arima().Mean Absolute Scaled Errors for 1, 4 and 13 weeksahead using a rolling origin.
Aggr. Level h Base Reconciled Change
Weekly 1 1.6 1.3 −17.2%Weekly 4 1.9 1.5 −18.6%Weekly 13 2.3 1.9 −16.2%Weekly 1–52 2.0 1.9 −5.0%Annual 1 3.4 1.9 −42.9%
Forecasting: principles and practice Temporal hierarchies 34
UK Accidents and Emergency Demand
Minimum training set: all data except the last yearBase forecasts using auto.arima().Mean Absolute Scaled Errors for 1, 4 and 13 weeksahead using a rolling origin.
Aggr. Level h Base Reconciled Change
Weekly 1 1.6 1.3 −17.2%Weekly 4 1.9 1.5 −18.6%Weekly 13 2.3 1.9 −16.2%Weekly 1–52 2.0 1.9 −5.0%Annual 1 3.4 1.9 −42.9%
Forecasting: principles and practice Temporal hierarchies 34
UK Accidents and Emergency Demand
Minimum training set: all data except the last yearBase forecasts using auto.arima().Mean Absolute Scaled Errors for 1, 4 and 13 weeksahead using a rolling origin.
Aggr. Level h Base Reconciled Change
Weekly 1 1.6 1.3 −17.2%Weekly 4 1.9 1.5 −18.6%Weekly 13 2.3 1.9 −16.2%Weekly 1–52 2.0 1.9 −5.0%Annual 1 3.4 1.9 −42.9%
Forecasting: principles and practice Temporal hierarchies 34
UK Accidents and Emergency Demand
Minimum training set: all data except the last yearBase forecasts using auto.arima().Mean Absolute Scaled Errors for 1, 4 and 13 weeksahead using a rolling origin.
Aggr. Level h Base Reconciled Change
Weekly 1 1.6 1.3 −17.2%Weekly 4 1.9 1.5 −18.6%Weekly 13 2.3 1.9 −16.2%Weekly 1–52 2.0 1.9 −5.0%Annual 1 3.4 1.9 −42.9%
Forecasting: principles and practice Temporal hierarchies 34
UK Accidents and Emergency Demand
Minimum training set: all data except the last yearBase forecasts using auto.arima().Mean Absolute Scaled Errors for 1, 4 and 13 weeksahead using a rolling origin.
Aggr. Level h Base Reconciled Change
Weekly 1 1.6 1.3 −17.2%Weekly 4 1.9 1.5 −18.6%Weekly 13 2.3 1.9 −16.2%Weekly 1–52 2.0 1.9 −5.0%Annual 1 3.4 1.9 −42.9%
Forecasting: principles and practice Temporal hierarchies 34
thief package for R
thief: Temporal HIErarchical Forecasting
Install from CRANinstall.packages("thief")
Usagelibrary(thief)thief(y)
Forecasting: principles and practice Temporal hierarchies 35
thief package for R
thief: Temporal HIErarchical Forecasting
Install from CRANinstall.packages("thief")
Usagelibrary(thief)thief(y)
Forecasting: principles and practice Temporal hierarchies 35
Outline
1 Hierarchical and grouped time series
2 Lab session 15
3 Temporal hierarchies
4 Lab session 16
Forecasting: principles and practice Lab session 16 36
Lab Session 16
Forecasting: principles and practice Lab session 16 37
![sales forecasting[1]](https://static.documents.pub/doc/80x56/54bf4f244a7959885b8b4574/sales-forecasting1.jpg)
