7/27/2019 Forecasting for manufacturing system

1/19

1

Forecasting

Lecturer: Prof. Duane S. Boning

Rev 8

7/27/2019 Forecasting for manufacturing system

2/19

2

Regression Review & Extensions

Single Model Coefficient: Linear Dependence

Slope and Intercept (or Offset):

Polynomial and Higher Order Models:

Multiple Parameters

Key point: linear regression can be used as long as the model islinear in the coefficients (doesnt matter the dependence in theindependent variable)

Time dependencies

Explicit

Implicit

7/27/2019 Forecasting for manufacturing system

3/19

3

Agenda

1. Regression Polynomial regression

Example (using Excel)

2. Time Series Data & Time Series Regression

Autocorrelation

ACF

Example: white noise sequences

Example: autoregressive sequences

Example: moving average

ARIMA modeling and regression

3. Forecasting Examples

7/27/2019 Forecasting for manufacturing system

4/19

4

Time Series Time as an Implicit Parameter

Data is oftencollected with a

t ime-order

An underlyingdynamic process

(e.g. due to physics

of a manufacturing

process) may create

0 10 20 30 40 50-10

-5

0

5

time

x

autocorrelated

autocorrelat ion inthe data

0 10 20 30 40 50-2

0

2

4

time

x

uncorrelated

7/27/2019 Forecasting for manufacturing system

5/195

Intuition: Where Does Autocorrelation Come From?

Consider a chamber with volume V, and with gas flow in and

gas flow out at rate f. We are interested in the concentrationxatthe output, in relation to a known input concentration w.

7/27/2019 Forecasting for manufacturing system

6/196

Key Tool: Autocorrelation Function (ACF)

Time series data: time indexi

CCF: cross-correlation function

ACF: auto-correlation function

) ACF shows the similarity of a signal

to a lagged version of same signal

0 20 40 60 80 100-4

-2

0

2

4

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

7/27/2019 Forecasting for manufacturing system

7/197

Stationary vs. Non-Stationary

Stationary series:

Process has a fixed mean

0 100 200 300 400 500-10

-5

0

5

10

time

x

0 100 200 300 400 500-10

0

10

20

30

time

7/27/2019 Forecasting for manufacturing system

8/198

Data drawn from IID gaussian

ACF: We also plot the 3 limits

values within these not significant

Note that r(0) = 1 always (a

signal is always equal to itself

with zero lag perfectlyautocorrelated at k = 0)

Sample mean

Sample variance

0 50 100 150 200-4

-2

0

2

4

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

White Noise An Uncorrelated Series

7/27/2019 Forecasting for manufacturing system

9/199

Autoregressive Disturbances

0 100 200 300 400 500-10

-5

0

5

10

time

x

0 5 10 15 20 25 30 35 40

-1

-0.5

0

0.5

1

lags

r(k)

Generated by:

Mean

Variance

So AR (autoregressive) behavior

increases variance of signal.

Slow drop in ACF with large

7/27/2019 Forecasting for manufacturing system

10/1910

Another Autoregressive Series

Generated by:

Slow drop in ACF with large

0 100 200 300 400 500-10

-5

0

5

10

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

Slow drop in ACF with large

But now ACF alternates in sign

High negative autocorrelation:

7/27/2019 Forecasting for manufacturing system

11/1911

Random Walk Disturbances

0 100 200 300 400 500-10

0

10

20

30

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

Veryslow drop in ACF for = 1

Generated by:

Mean

Variance

7/27/2019 Forecasting for manufacturing system

12/1912

Moving Average Sequence

0 100 200 300 400 500-4

-2

0

2

4

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

Generated by:

Mean

Variance

So MA (moving average) behavior

also increases variance of signal.

r(1)

Jump in ACF at specific lag

7/27/2019 Forecasting for manufacturing system

13/1913

ARMA Sequence

0 100 200 300 400 500-10

-5

0

5

10

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

Generated by:

Both AR & MA behavior

Slow drop in ACF with large

7/27/2019 Forecasting for manufacturing system

14/1914

0 100 200 300 400 500-200

0

200

400

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

ARIMA Sequence

Start with ARMA sequence:

Add Integrated (I) behavior

Slow drop in ACF with large

random walk (integrative) action

7/27/2019 Forecasting for manufacturing system

15/19

15

Periodic Signal with Autoregressive Noise

0 50 100 150 200 250 300 350 400-10

0

10

20

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

Original Signal

0 50 100 150 200 250 300 350 400-5

0

5

time

x

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

r(k)

After Differencing

See underlying signal with period = 5

7/27/2019 Forecasting for manufacturing system

16/19

16

Cross-Correlation: A Leading Indicator

Now we have two series:

An input or explanatoryvariable x

An output variable y

CCF indicates both AR and lag:

0 100 200 300 400 500-10

-5

0

5

10

time

x

0 100 200 300 400 500-10

-5

0

5

10

time

y

0 5 10 15 20 25 30 35 40-1

-0.5

0

0.5

1

lags

rx

y(k

)

7/27/2019 Forecasting for manufacturing system

17/19

17

Regression & Time Series Modeling

The ACF or CCF are helpful tools in selecting anappropriate model structure

Autoregressive terms?

xi = xi-1

Lag terms?

yi = xi-k

One can structure data and perform regressions

Estimate model coefficientvalues, significance, and

confidence intervals

Determine confidence intervals on output Check residuals

7/27/2019 Forecasting for manufacturing system

18/19

18

Statistical Modeling Summary

1. Statistical Fundamentals

Sampling distributions Point and interval estimation

Hypothesis testing

2. Regression

ANOVA

Nominal data: modeling of treatment effects (mean differences)

Continuous data: least square regression

3. Time Series Data & Forecasting Autoregressive, moving average, and integrative behavior

Auto- and Cross-correlation functions

Regression and time-series modeling

7/27/2019 Forecasting for manufacturing system

19/19

MIT OpenCourseWarehttp://ocw.mit.edu

2.854 / 2.853 Introduction to Manufacturing Systems

Fall 2010

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

http://ocw.mit.edu/http://ocw.mit.edu/termshttp://ocw.mit.edu/termshttp://ocw.mit.edu/