39
Time series analysis Matlab tutorial Joachim Gross
Time series analysis Matlab tutorial
Joachim Gross
Outline Terminology Sampling theorem Plotting Baseline correction Detrending Smoothing Filtering Decimation
Remarks Focus on practical aspects, exercises, getting
experience (not on equations, theory) Focus on How to do Learn some basic skills for TS analysis
Note: Usually there is not a single perfectly correct way of doing a TS operation! => learn the limitations!
What is a time series?A sequence of measurements over time
0 2 4 6 8 10-1
-0.5
0
0.5
1
1.5
Terminology Continuous TS: continuous observations Discrete TS: observations at specific times usually equally spaced
Deterministic TS: future values can be exactly predicted from past values
Stochastic TS: exact prediction not possible
Objectives of TS analysis Description Explanation Prediction Control
Simple descriptive analysisSummary statistics (mean, std) is not always meaningful for
TS
0 50 100 150 200 250 300 350 400 450 5000
2
4
6
8
10
12
Sampling Converting a continuous signal into a discrete time series Reconstruction is possible if sampling frequency is greater than twice
the signal bandwidth
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
Time (s)0 0.2 0.4 0.6 0.8 1
-1
-0.5
0
0.5
1
75 Hz sampling
Sampling Nyquist frequency: half of sampling frequency
10 Hz sampling 10 Hz reconstruction
0 0.2 0.4 0.6 0.8 1-3
-2
-1
0
1
2
3
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
Sampling
8 Hz sampling
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
Aliasing: Frequencies above Nyquist frequency are reconstructed below Nyquist frequency
Sampling
8 Hz sampling
Aliasing: Frequencies above Nyquist frequency are reconstructed below Nyquist frequency
0 1 2 3 4-140
-120
-100
-80
-60
-40
-20
0
20
Frequency
P
o
w
e
r
S
p
e
c
t
r
u
m
M
a
g
n
i
t
u
d
e
(
d
B
)
0 5 10 15 20-140
-120
-100
-80
-60
-40
-20
0
20
Frequency
P
o
w
e
r
S
p
e
c
t
r
u
m
M
a
g
n
i
t
u
d
e
(
d
B
)
40 Hz sampling
Simple operations on TS Plotting Removing a baseline Removing a trend Smoothing Filtering Decimation
Plotting in Matlab For visual inspection of TS For publications/talks
plot sptool
Data preprocessing I Removing offset ts=ts-mean(ts);
0 0.2 0.4 0.6 0.8 10
2
4
6
8
10
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
Data preprocessing II Removing a trend ts=detrend(ts); subtracts best fitting line detrend can be used to subtract mean: detrend(ts,constant)
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6-2
-1
0
1
2
3
4
5
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6-3
-2
-1
0
1
2
3
4
5
data 1 lineardata 2
Data preprocessing III Smoothing ts=filter(ones(1,30)/30,1,ts); %mean filter, moving average uses zeros at beginning! => baseline correction or do not use first 30 samples
-0.2 0 0.2 0.4 0.60
1
2
3
4
5
6
7
Data preprocessing III introduces a shift! => either correct for it or ts=filtfilt(ones(1,15)/15,1,ts); %mean filter, forward and reverse no shift! filter can take any smoothing kernel (gaussian, etc)
-0.2 0 0.2 0.4 0.6-3
-2
-1
0
1
2
3
4
-0.2 0 0.2 0.4 0.60
1
2
3
4
5
6
7shifted by 15 samples filtfilt
Data preprocessing III Smoothing ts=medfilt1(ts,30); %median filter, takes into account the shift uses 0 at beginning and end !
-0.2 -0.18 -0.16 -0.14 -0.12 -0.11
1.5
2
2.5
3
3.5
4
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.60
1
2
3
4
5
6
7
Data preprocessing III Smoothing ts=sgolayfilt(ts,3,41); %Savitzky-Golay filter fits 3rd order polynomial to frames of size 41 good at preserving high frequencies in the data
-0.2 0 0.2 0.4 0.60
1
2
3
4
5
6
7
Data preprocessing III Smoothing compare unsmoothed and smoothed data check for shift check beginning (and end) of the smoothed time series
Exercise 1
Data preprocessing IV Filtering FIR-Filter (finite impulse
response) stable high filter order usually have linear phase(phase change is proportional to
frequency)
IIR-Filter (infinite impulse response)
potentially unstable low filter order non-linear phase distortion computationally efficient
Data preprocessing IV IIR-Filter:
Butterworth Elliptic Chebychev Typ 1 Chebychev Typ 2 Bessel
FIR-Filter: fir1
Data preprocessing IV lowpass highpass bandpass bandstop
0 5 10 15 20 25 30 35 40 45
-400
-350
-300
-250
-200
-150
-100
-50
0
Frequency (Hz)
M
a
g
n
i
t
u
d
e
(
d
B
)
Magnitude Response (dB)
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
5 Hz lowpass
Data preprocessing IV lowpass highpass bandpass bandstop
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
0 5 10 15 20 25 30 35 40 45
-100
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Frequency (Hz)
M
a
g
n
i
t
u
d
e
(
d
B
)
Magnitude Response (dB)
30 Hz highpass
Data preprocessing IV lowpass highpass bandpass bandstop
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
2-30 Hz bandpass
0 5 10 15 20 25 30 35 40 45
-40
-35
-30
-25
-20
-15
-10
-5
0
Frequency (Hz)
M
a
g
n
i
t
u
d
e
(
d
B
)
Magnitude Response (dB)
Data preprocessing IV lowpass highpass bandpass bandstop
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
30-40 Hz bandstop
0 5 10 15 20 25 30 35 40 45
-25
-20
-15
-10
-5
0
Frequency (Hz)
M
a
g
n
i
t
u
d
e
(
d
B
)
Magnitude Response (dB)
Simple design: FIR [b]=fir1(4,2*4/sf); %4 Hz lowpass [b]=fir1(4,2*4/sf,high); %4 Hz highpass [b]=fir1(4,2*[4 10]/sf); %4-10 Hz bandpass [b]=fir1(4,2*[4 10]/sf,stop); %4-10 Hz bandstop
tsf=filter(b,1,ts); tsf=filtfilt(b,1,ts); %forward and reverse
Simple design: IIR [b,a]=butter(4,2*4/sf); %4 Hz lowpass [b,a]=butter(4,2*4/sf,high); %4 Hz highpass [b,a]=butter(4,2*[4 10]/sf); %4-10 Hz bandpass [b,a]=butter(4,2*[4 10]/sf,stop); %4-10 Hz bandstop
tsf=filter(b,a,ts); tsf=filtfilt(b,a,ts); %forward and reverse
Simple Inspection
0 5 10 15 20 25 30 35 40 45 50-400
-300
-200
-100
0
Frequency (Hz)
P
h
a
s
e
(
d
e
g
r
e
e
s
)
0 5 10 15 20 25 30 35 40 45 50-300
-200
-100
0
Frequency (Hz)
M
a
g
n
i
t
u
d
e
(
d
B
)
freqz(b,a,100,100);sf
number of frequencies
Complex design fdatool
magnitude response phase response impulse response compare filters effect of changing filter order
Filter artifacts onset transients
0 2 4 6 8-6
-5
-4
-3
-2
-1
0
1
2x 10-3 1 Hz lowpass (4th order Butterworth)
0 2 4 6 8-6
-5
-4
-3
-2
-1
0
1
2x 10-3
filtfiltfilter
Filter artifacts ringing
with artifact
3.5 4 4.5-5
0
5
10
15
20x 10-3
3.5 4 4.5-5
0
5
10
15
20x 10-3
without artifact
Filter artifacts ringing
filter, 20 Hz lowpass (12th order Butterworth)
3.5 4 4.5-5
0
5
10
15
20x 10-3
3.5 4 4.5
-2
-1
0
1
2
3
4x 10-3
filtfilt
Filter artifacts beginning and end of filtered ts is distorted filtering artifacts is dangerous filtering may change the latency of effects! filtering may change the phase
Suggestions be careful with low frequencies use low order butterworth forward and reverse (to avoid
phase distortions) carefully check beginning and end of filtered ts make sure you dont have artifacts in the data use surrogate data (filtered noise)
Data preprocessing V Decimation ts=decimate(ts,4); decimate uses a lowpass filter to avoid aliasing artifacts
Exercises 2-4
Time series analysisMatlab tutorialOutlineRemarksWhat is a time series?TerminologyObjectives of TS analysisSimple descriptive analysisSamplingSamplingSamplingSamplingSimple operations on TSPlotting in MatlabData preprocessing IData preprocessing IData preprocessing IIData preprocessing IIIData preprocessing IIIData preprocessing IIIData preprocessing IIIData preprocessing IIIExercise 1Data preprocessing IVData preprocessing IVData preprocessing IVData preprocessing IVData preprocessing IVData preprocessing IVSimple design: FIRSimple design: IIRSimple InspectionComplex designFilter artifactsFilter artifactsFilter artifactsFilter artifactsSuggestionsData preprocessing VExercises 2-4
