Trend and System Identification With Orthogonal Basis Function
OSE SEMINAR 2013
ÄMIR SHIRDEL
CENTER OF EXCELLENCE IN
OPTIMIZATION AND SYSTEMS ENGINEERING
AT ÅBO AKADEMI UNIVERSITY
ÅBO NOVEMBER 15 2013
Background
– System identification is difficult when process measurements are
corrupted by structured disturbances, such as trends, outliers, level
shifts
– Standard approach: removal by data preprocessing but difficult to
separate between the effects of known system inputs and unknown
disturbances (trends, etc.)
– Orthonormal basis function models are categorized as output-error
(ballistic simulation) models
2|N
Background
Amir Shirdel: Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
2|22
Present contribution
– Identification of system model parameters and disturbances
simultaneously
– Sparse optimization used in system identification problem
– Applying the method on simulated and real example
3|N
Present contribution
Amir Shirdel: Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
3|22
4
4 4
Present contribution
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
4|22
Present contribution
SYSTEN
(model) U Y
0 100 200 300 400 500 600 700 800 900 1000-10
-8
-6
-4
-2
0
2
4
6
8
d
0 100 200 300 400 500 600 700 800 900 1000-25
-20
-15
-10
-5
0
5
10
15
20
0 100 200 300 400 500 600 700 800 900 1000-4
-3
-2
-1
0
1
2
3
4
5
Orthogonal basis functions has some advantages:
• The corresponding approximation (representation) has simple and direct
solution.
• It corresponds to allpass filters which is robust to implement and use in
numerical computation.
• It is popular because a few parameters can describe the system.
• It is a kind of output-error model, and can be insensitive to noise.
5 5
Orthogonal basis function (Fixed-pole model)
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
5|22
Orthogonal basis function
6 6 6
Orthogonal basis function (Fixed-pole model)
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
6|22
Orthogonal basis function
FIR network:
Laguerre function:
Kautz function:
7
Model:
where
The parameters can be estimated using standard least squares method:
7 7
System Identification
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
7|22
System identification
2)ˆ)(( θ(k)ky T
d(k)θ(k)y(k) T
System u
d
y
T
n
T
n aaakukukuk ],...,,[,)](),...,(),([)( 2121
)()()(or)()()( kuqkukuqLku nnnn
8 8 8
Problem formulation
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
8|22
Problem formulation
We consider the linear system model:
It is assumed the measured output is given by
where d(k) is a structured disturbance:
• outlier signal,
• level shifts,
• piecewise constant trends
)()()( kdkyky L
),(...)()()( 2211 kuakuakuaky nnL
9 9 9
Disturbance models
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
9|22
Disturbance models
Sequence of outliers:
Level shifts:
Sequence of trends:
10 10 10
Sparse representation of disturbance
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
10|22
Sparse representation of disturbance
For the structured disturbances, the vectors are sparse,
where and , depends on disturbances
Outliers:
Level shifts:
Trends:
dDi
ID 0
iD
11
Identification by sparse optimization:
subject to
where
This is an intractable combinatorial optimization problem.
Instead we use and solve the convex problem
11 11 11 11
Sparse optimization approach
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
11|22
Sparse optimization approach
1
2ˆ,ˆ
ˆ))(ˆ)((min dDλkyky i
kd
2ˆ,ˆ
))(ˆ)((min k
dkyky
elements nonzero ofnumber 0
MdDi 0
relaxation1 l
12
Algorithm
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
12|22
Algorithm
1. Make the basis function expansion based on given prior knowledge of system (pole and system order).
2. Solution of sparse optimization problem by iterative reweighting:
Minimize the weighted cost to give the estimates
where i = 0, 1 or 2 (user selected).
3. Calculate new weights W and go to step 2.
4. Continue until convergence.
5. Use model order reduction to get lower order model.
1
2ˆ,ˆ
ˆ))(ˆˆ)((min dDWλkdky ii
k
T
d
We apply the proposed identification detrending method to the ARX model
with parameter vector
given by
u(k) and e(k) are normally distributed signals with variances 1 and 0.1, and
d(k) is unknown structured disturbance ,
For making the Kautz basis function we used N=4 as order of system and pole
is
13 13
Example1
13|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
4.07.0
14
Example1
14|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0.9340ˆ 9125.0 ˆ RMSE :without
5.2937RMSE 0.7697- 1.3465 2.8084 1.5214 ˆ
0.9102RMSE 0.8412- 1.3152 2.7305 1.5428 ˆ
LS
LS
RMSEd(k)
0 100 200 300 400 500 600 700 800 900 1000 -20
-10
0
10
20
K data points
Output
Output (model)
Output (LS)
0 100 200 300 400 500 600 700 800 900 1000
-5
0
5
10
K data points
Disturbance (Identified)
Disturbance
15
Example1
15|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 1 2 3 4 50
1
2
3
4
5
6
7
8Hankel Singular Values (State Contributions)
State
Sta
te E
nerg
y
Stable modes
16
Example1
16|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0.9118RMSE 0.4933- 0.6946- ˆ
0.9102RMSE 0.8412- 1.3152 2.7305 1.5428 ˆ
R
0 100 200 300 400 500 600 700 800 900 1000-20
-10
0
10
K data points
Output (model)
Output (Reduced model)
0 100 200 300 400 500 600 700 800 900 1000
-5
0
5
10
K data points
Disturbance (Identified)
Disturbance
17 17
Example2: Pilot-scale distillation column data
17|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 200 400 600 800 1000 1200 1400 1600 1800 2000 91
91.5
K data points
Top column output
0 200 400 600 800 1000 1200 1400 1600 1800 2000 0
2
K data points
Bottom column output
0 200 400 600 800 1000 1200 1400 1600 1800 2000 156
158
160
162
K data points
Reflux (L)
0 200 400 600 800 1000 1200 1400 1600 1800 2000 87
88
89
90
K data points
Reboiling flow (V)
1
18 18
Example2: Identification results
18|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 200 400 600 800 1000 1200 1400 1600 1800 2000-1
-0.5
0
0.5
1
1.5
K data points
Output (real)
Output+dd (model)
0 200 400 600 800 1000 1200 1400 1600 1800 2000-0.2
-0.1
0
0.1
0.2
0.3
K data points
Disturbance (identified)
0.0933RMSE 0.0456- 0.0353- ˆ
0.0826 RMSE 12) ...(total 0.0147- 0.0338- 0.0285- 0.0022- ˆ
0.0643RMSE 12) (total ... 0.0151- 0.0346- 0.0280- 0.0031- ˆ
RED
LS
19 19
Example2: Hankel singular value for reduction
19|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0 2 4 6 8 10 120
0.1
0.2
0.3
0.4
0.5
0.6
0.7Hankel Singular Values (State Contributions)
State
Sta
te E
nerg
y
Stable modes
20 20
Example2: Validation data with estimation of d(k)
20|22
Example
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
0.0891RMSE 0.0456- 0.0353- ˆ RED
0 200 400 600 800 1000 1200-1
0
1
Output (real)
Output(Test Model+Disturbance)
0 200 400 600 800 1000 1200
-0.2
0
0.2
Disturbance
0 200 400 600 800 1000 1200
-0.5
0
0.5
0 200 400 600 800 1000 1200-1
0
1
Summary:
• Presented a method for identification of linear systems in the presence of
structured disturbances (outliers, level shifts and trends) by using sparse
optimization and orthogonal basis function as the system model
• Gives acceptable results for simulated example
• Gives acceptable results for distillation column example
• The orthogonal basis model improves the robustness and more insensitive
to noise
Future work:
• Nonlinear system identification and trends and more general disturbances
21 21
Discussion and future work
21|22
Discussion and future work
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University
22 22
Thank you for your attention!
Questions?
22 22
22|22
Amir Shirdel : Trend and System Identification With Orthogonal Basis Function Center of Excellence in Optimization and Systems Engineering at Åbo Akademi University