Basics of Systems and ControlTheory for pyMORJens Saak
September 30, 2020
pyMOR Online Course 2020
Σu y
E x(t) = A x(t) + B u(t)
y(t) = C x(t) + D u(t)
MOR
E
˙x(t) =A
x(t) +B
u(t)
y(t) = C x(t) + D u(t)
Outline
1 Linear Time-Invariant (LTI) Systems
2 Transfer Function and Realizations
3 System Analysis
4 A Selection of MOR Methods
Restrictions for this lecture
• Only continuous-time systemsDiscrete-time is treated in [1]
• No differential-algebraic systemsFor DAE aspects see [6, 3, 4, 5]
• No non-linearities
• No parameter dependencies
Outline
1 Linear Time-Invariant (LTI) Systems
• Setting for this course
• Examples
2 Transfer Function and Realizations
3 System Analysis
4 A Selection of MOR Methods
Linear Time-Invariant (LTI) Systems | Setting for this course
First-order State-space Systems ( : LTIModel)
E
x(t) = Ax(t) + Bu(t),
y(t) = Cx(t) + Du(t).(Σ)
Here
• x(t) ∈ Rn is called the state,
• u(t) ∈ Rm is called the input,
• y(t) ∈ Rp is called the output
of the LTI system. Correspondingly, we have
E,
A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n and D ∈ Rp×m.
We assume t ∈ [0,∞), x(0) = 0.
3 / 37
Linear Time-Invariant (LTI) Systems | Setting for this course
First-order State-space Systems ( : LTIModel)
E
x(t) = Ax(t) + Bu(t),
y(t) = Cx(t).(Σ)
Here
• x(t) ∈ Rn is called the state,
• u(t) ∈ Rm is called the input,
• y(t) ∈ Rp is called the output
of the LTI system. Correspondingly, we have
E,
A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n.
We assume t ∈ [0,∞), x(0) = 0.
3 / 37
Linear Time-Invariant (LTI) Systems | Setting for this course
First-order State-space Systems ( : LTIModel)
Ex(t) = Ax(t) + Bu(t),
y(t) = Cx(t).(Σ)
Here
• x(t) ∈ Rn is called the state,
• u(t) ∈ Rm is called the input,
• y(t) ∈ Rp is called the output
of the LTI system. Correspondingly, we have
E, A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n.
We assume t ∈ [0,∞), x(0) = 0 andE invertible.
3 / 37
Linear Time-Invariant (LTI) Systems | Setting for this course
Second-order State-space Systems ( : SecondOrderModel)
Mx(t) + Ex(t) + Kx(t) = Bu(t),
y(t) = Cvx(t) + Cpx(t).
Here
• x(t) ∈ Rn is called the position,
• x(t) ∈ Rn is called the velocity,
• u(t) ∈ Rm is called the input,
• y(t) ∈ Rp is called the output
of the LTI system. Correspondingly, we have
M, E, K ∈ Rn×n, B ∈ Rn×m, Cv, Cp ∈ Rp×n.
4 / 37
Linear Time-Invariant (LTI) Systems | Examples
Heat Equation [MORWiki thermal block] I
For t ∈ (0, T ), ξ ∈ Ω and initial values
θ(0, ξ) = 0, for ξ ∈ Ω,
consider
∂tθ(t, ξ) +∇ · (−σ(ξ)∇θ(t, ξ)) = 0,
with boundary conditions
σ(ξ)∇θ(t, ξ) · n(ξ) = u(t) t ∈ (0, T ), ξ ∈ Γin,
σ(ξ)∇θ(t, ξ) · n(ξ) = 0 t ∈ (0, T ), ξ ∈ ΓN ,
θ(t, ξ) = 0 t ∈ (0, T ), ξ ∈ ΓD.
0.2
0.2
0.4
0.4
0.6
0.6
0.8
0.8
(0,0)
(0,1)
(1,0)
(1,1)
Ω0
Ω1 Ω2
Ω3Ω4
Γin
ΓN
ΓN
ΓD
5 / 37
Linear Time-Invariant (LTI) Systems | Examples
Heat Equation [MORWiki thermal block] II
Finite element semi-discretization in space
• pairwise inner products of ansatz functions E
• discretized spatial operator + Dirichlet boundary condition A
• discretized non-zero Neumann boundary condition B
• average temperatures on the inclusions C
• n = 7 488
• m = 1
• p = 4
6 / 37
Linear Time-Invariant (LTI) Systems | Examples
An Artificial Fishtail [MORWiki Artificial Fishtail] I
Construction: Fluid Elastomer Actuation:
no pressure
under pressure
7 / 37
Linear Time-Invariant (LTI) Systems | Examples
An Artificial Fishtail [MORWiki Artificial Fishtail] II
Variables:
• displacement ~s(t, ~z)
• strain ~ε(~s(t, ~z))
• stress ~σ(~s(t, ~z))
Material parameters:
• density ρ
• Lamé parameters λ, µ
Basic principle:
~ε(~s(t, ~z)) =1
2
(∇~s(t, ~z) +∇T~s(t, ~z)
)(kinematic equation)
~σ(~s(t, ~z)) = λ tr((~ε(~s(t, ~z))) I + 2µ~ε(~s(t, ~z))) (material equation)
ρ∂2~s(t, ~z)
∂t2= ∇ · ~σ(~s(t, ~z)) + ~f(t, ~z) (equation of motion)
+ initial and boundary conditions
8 / 37
Linear Time-Invariant (LTI) Systems | Examples
An Artificial Fishtail [MORWiki Artificial Fishtail] III
FEM semi-discretization:
Mx(t) + Ex(t) + Kx(t) = Bu(t),
y(t) = Cpx(t),
with
• M,E,K > 0,Cv = 0,
• n = 779 232,m = 1, p = 3.
10−3 10−1 101 10310−12
10−7
10−2
frequency (Hz)
Mag
nitu
de
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
0
1
2
3·10−2
time (s)
disp
lace
men
t(m
)
9 / 37
Outline
1 Linear Time-Invariant (LTI) Systems
2 Transfer Function and Realizations
• Laplace Transform
• Transfer Function
• Realizations
• Projection-based MOR
3 System Analysis
4 A Selection of MOR Methods
Transfer Function and Realizations | Laplace Transform
Definition
Let f : [0,∞)→ Rn be exponentially bounded with bounding exponent α. Then
Lf (s) :=∫ ∞
0
f(τ)e−sτdτ
for Re(s) > α is called the Laplace transform of f . The process of forming theLaplace transform is called Laplace transformation.
It can be shown that the integral converges uniformly in a domain with Re(s) ≥ β forall β > α.
Allows us to map time signals to frequency signals.
10 / 37
Transfer Function and Realizations | Laplace Transform
Definition
Let f : [0,∞)→ Rn be exponentially bounded with bounding exponent α. Then
Lf (s) :=∫ ∞
0
f(τ)e−sτdτ
for Re(s) > α is called the Laplace transform of f . The process of forming theLaplace transform is called Laplace transformation.
It can be shown that the integral converges uniformly in a domain with Re(s) ≥ β forall β > α.
Allows us to map time signals to frequency signals.
10 / 37
Transfer Function and Realizations | Laplace Transform
Theorem
Let f, g, h : [0,∞)→ Rn be given. Then the following two statements hold true:
a) The Laplace transformation is linear, i. e., if f and g are exponentially bounded, thenh := γf + δg is also exponentially bounded and
Lh = γLf+ δLg
holds for all γ, δ ∈ C.b) If f ∈ PC1([0,∞),Rn) and f is exponentially bounded, then f is exponentially
bounded andLf
(s) = sLf(s)− f(0).
• X(s) := Lx(s), U(s) := Lu(s), and Y (s) := Ly(s)• Ax(t) + Bu(t) AX(s) + BU(s)
• y(t) = Cx(t) Y (s) = CX(s)
• sX(s) := Lx(s) since x(0) = 0
11 / 37
Transfer Function and Realizations | Laplace Transform
Theorem
Let f, g, h : [0,∞)→ Rn be given. Then the following two statements hold true:
a) The Laplace transformation is linear, i. e., if f and g are exponentially bounded, thenh := γf + δg is also exponentially bounded and
Lh = γLf+ δLg
holds for all γ, δ ∈ C.b) If f ∈ PC1([0,∞),Rn) and f is exponentially bounded, then f is exponentially
bounded andLf
(s) = sLf(s)− f(0).
• X(s) := Lx(s), U(s) := Lu(s), and Y (s) := Ly(s)• Ax(t) + Bu(t) AX(s) + BU(s)
• y(t) = Cx(t) Y (s) = CX(s)
• sX(s) := Lx(s) since x(0) = 0
11 / 37
Transfer Function and Realizations | Transfer Function
Rational Matrix Function Representation
In summary we have:
• sEX(s) = AX(s) + BU(s)
• Y (s) = CX(s)
Thus the mapping from inputs to outputs in frequency domain can be expressed as
H(s) = C(sE−A)−1B.
Analogously, for second-order systems we get
H(s) = (sCv +Cp)(s2M+ sE+K
)−1B.
H is analytic inC \ Λ(E,A), orC \ Λ(M,E,K), respectively
12 / 37
Transfer Function and Realizations | Transfer Function
Rational Matrix Function Representation
In summary we have:
• sEX(s) = AX(s) + BU(s)
• Y (s) = CX(s)
Thus the mapping from inputs to outputs in frequency domain can be expressed as
H(s) = C(sE−A)−1B.
Analogously, for second-order systems we get
H(s) = (sCv +Cp)(s2M+ sE+K
)−1B.
H is analytic inC \ Λ(E,A), orC \ Λ(M,E,K), respectively
12 / 37
Transfer Function and Realizations | Transfer Function
Rational Matrix Function Representation
In summary we have:
• sEX(s) = AX(s) + BU(s)
• Y (s) = CX(s)
Thus the mapping from inputs to outputs in frequency domain can be expressed as
H(s) = C(sE−A)−1B.
Analogously, for second-order systems we get
H(s) = (sCv +Cp)(s2M+ sE+K
)−1B.
H is analytic inC \ Λ(E,A), orC \ Λ(M,E,K), respectively
12 / 37
Transfer Function and Realizations | Transfer Function
Important Representations of H(s)
(Laurent) series expansion
H(s) =
∞∑
k=0
(s− s0)kMk(s0) H(s) =
∞∑
k=0
s−kMk(∞)
The matricesMk(s0) are calledmoments ofH. At infinity they are also referred toasMarkov parameters.
Pole Residue Form
Let (λi, wi, vi) be the eigentriplets of the pair (A,E) with no degenerateeigenspaces. Then we have
H(s) =
n∑
i=1
Ris− λi
,
whereRi = (Cvi)(wHi B), assumingwH
i vi = 1.
13 / 37
Transfer Function and Realizations | Realizations
The representation ofH using (E,A,B,C) is not unique.
In fact for any invertible matrixT ∈ Rn×n, we have
H(s) = C(sE−A)−1
B
= CT−1T(sE−A)−1
T−1TB
= CT−1(sTET−1 −TAT−1
)−1TB
and thus a system given, by (TET−1,TAT−1,TB,CT−1) realizes the exact sameinput/output behavior.
Definition
• All sets of matrices leading to the same functionH are called its realizations.
• The matrixT above is called state-space transformation.
14 / 37
Transfer Function and Realizations | Realizations
Important Realizations
• Minimal RealizationsCan we realizeH with less equations?
• Truncated RealizationsCan we introduce a small error to get even less equations?
• Balanced Realizations see here
Can we find state coordinates that allow us to decide what is important?
15 / 37
Transfer Function and Realizations | Realizations
Important Realizations
• Minimal RealizationsCan we realizeH with less equations?
• Truncated RealizationsCan we introduce a small error to get even less equations?
• Balanced Realizations see here
Can we find state coordinates that allow us to decide what is important?
15 / 37
Transfer Function and Realizations | Realizations
Important Realizations
• Minimal RealizationsCan we realizeH with less equations?
• Truncated RealizationsCan we introduce a small error to get even less equations?
• Balanced Realizations see here
Can we find state coordinates that allow us to decide what is important?
15 / 37
Transfer Function and Realizations | Realizations
McMillan Degree and Minimal Realization
ExampleRealizations can even be of different dimensions. Take for example:
E = I the identity,A =
[−11 0
0 −5
],B =
[11
]andC =
[1 0
].
Truncating the second state component does not changeH.
Definition
There exists a minimum number of equations necessary to describeH. The statedimension n of this minimal set of equations is calledMcMillan degree of thesystem. A realization ofH with this dimension is calledminimal realization.
16 / 37
Transfer Function and Realizations | Realizations
Truncated Realizations via Ritz/Petrov-Galerkin Projection
Ex(t)−Ax(t)−Bu(t) = 0,
y(t)−Cx(t)−Du(t) = 0.
Step I: Use truncated state transformationReplace
x(t) ≈ Vx(t)
withV ∈ Rn×r and x(t) ∈ Rr .
Step II: Mitigate transformation errorSuppress truncation residual through left projection.
• one-sided method: useV again.
• two-sided method: findW ∈ Rn×r .
17 / 37
Transfer Function and Realizations | Realizations
Truncated Realizations via Ritz/Petrov-Galerkin Projection
EV ˙x(t)−AVx(t)−Bu(t) = eres(t),
y(t)−CVx(t)−Du(t) = eoutput(t).
Step I: Use truncated state transformationReplace
x(t) ≈ Vx(t)
withV ∈ Rn×r and x(t) ∈ Rr .
Step II: Mitigate transformation errorSuppress truncation residual through left projection.
• one-sided method: useV again.
• two-sided method: findW ∈ Rn×r .
17 / 37
Transfer Function and Realizations | Realizations
Truncated Realizations via Ritz/Petrov-Galerkin Projection
VTEV ˙x(t)−VTAVx(t)−VTBu(t) = 0,
y(t)−CVx(t)−Du(t) = eoutput(t).
Step I: Use truncated state transformationReplace
x(t) ≈ Vx(t)
withV ∈ Rn×r and x(t) ∈ Rr .
Step II: Mitigate transformation errorSuppress truncation residual through left projection.
• one-sided method: useV again.
• two-sided method: findW ∈ Rn×r .
17 / 37
Transfer Function and Realizations | Realizations
Truncated Realizations via Ritz/Petrov-Galerkin Projection
WTEV ˙x(t)−WTAVx(t)−WTBu(t) = 0,
y(t)−CVx(t)−Du(t) = eoutput(t).
Step I: Use truncated state transformationReplace
x(t) ≈ Vx(t)
withV ∈ Rn×r and x(t) ∈ Rr .
Step II: Mitigate transformation errorSuppress truncation residual through left projection.
• one-sided method: useV again.
• two-sided method: findW ∈ Rn×r .
17 / 37
A ≈
WT
A V
Transfer Function and Realizations | Projection-based MOR
Reduced order model (ROM) ( : LTIPGReductor)
Define E = WTEV, A = WTAV ∈ Rr×r , B = WTB ∈ Rr×m andC = CV ∈ Rp×r . Then
E ˙x(t) = Ax(t) + Bu(t),
y(t) = Cx(t) + Du(t)(ROM)
approximates the dynamics of the full-order model (Σ) with output error
y(t)− y(t) = eoutput(t).
• We call the corresponding transfer function H.
• Model order reduction (MOR) FindW,V ∈ Rn×r such that eoutput(t) issmall in a suitable sense.
• We will see energy-based and interpolation-based methods today and tomorrow.
19 / 37
Outline
1 Linear Time-Invariant (LTI) Systems
2 Transfer Function and Realizations
3 System Analysis
• System Norms and Hardy Spaces
• Frequency-Domain Analysis
4 A Selection of MOR Methods
System Analysis | System Norms and Hardy Spaces
We haveY (s) = H(s)U(s)
andY (s) = H(s)U(s).
Question
What are suitable norms such that
‖y − y‖ ≤∥∥∥H− H
∥∥∥ ‖u‖?
20 / 37
System Analysis | System Norms and Hardy Spaces
The Banach SpaceHp×m∞
Hp×m∞ :=
G : C+ → Cp×m : G is analytic inC+ and sup
s∈C+
‖G(s)‖2 <∞.
Hp×m∞ is a Banach space equipped with theH∞-norm
‖G‖H∞ := supω∈R‖G(iω)‖2 .
Can show: ‖y − y‖L2≤∥∥∥H− H
∥∥∥H∞‖u‖L2
.
This bound can even be shown to be sharp.
21 / 37
System Analysis | System Norms and Hardy Spaces
The Banach SpaceHp×m∞
Hp×m∞ :=
G : C+ → Cp×m : G is analytic inC+ and sup
s∈C+
‖G(s)‖2 <∞.
Hp×m∞ is a Banach space equipped with theH∞-norm
‖G‖H∞ := supω∈R‖G(iω)‖2 .
Can show: ‖y − y‖L2≤∥∥∥H− H
∥∥∥H∞‖u‖L2
.
This bound can even be shown to be sharp.
21 / 37
System Analysis | System Norms and Hardy Spaces
The Banach SpaceHp×m∞
Hp×m∞ :=
G : C+ → Cp×m : G is analytic inC+ and sup
s∈C+
‖G(s)‖2 <∞.
Hp×m∞ is a Banach space equipped with theH∞-norm
‖G‖H∞ := supω∈R‖G(iω)‖2 .
Can show: ‖y − y‖L2≤∥∥∥H− H
∥∥∥H∞‖u‖L2
.
This bound can even be shown to be sharp.
21 / 37
System Analysis | System Norms and Hardy Spaces
The Hilbert SpaceHp×m2
Hp×m2 :=
G : C+ → Cp×m : G is analytic inC+ and
supξ>0
∫ ∞
−∞‖G(ξ + iω)‖2F dω <∞
.
Hp×m2 is a Hilbert space with the inner product
〈F,G〉H2:=
1
2π
∫ ∞
−∞tr(F (iω)
HG(iω)
)dω
and induced norm
‖G‖H2:= 〈G,G〉1/2H2
=
(1
2π
∫ ∞
−∞‖G(iω)‖2F dω
)1/2
.
Can show: ‖y − y‖L∞ ≤∥∥∥H− H
∥∥∥H2
‖u‖L2.
22 / 37
System Analysis | System Norms and Hardy Spaces
The Hilbert SpaceHp×m2
Hp×m2 :=
G : C+ → Cp×m : G is analytic inC+ and
supξ>0
∫ ∞
−∞‖G(ξ + iω)‖2F dω <∞
.
Hp×m2 is a Hilbert space with the inner product
〈F,G〉H2:=
1
2π
∫ ∞
−∞tr(F (iω)
HG(iω)
)dω
and induced norm
‖G‖H2:= 〈G,G〉1/2H2
=
(1
2π
∫ ∞
−∞‖G(iω)‖2F dω
)1/2
.
Can show: ‖y − y‖L∞ ≤∥∥∥H− H
∥∥∥H2
‖u‖L2.
22 / 37
System Analysis | System Norms and Hardy Spaces
The Hilbert SpaceHp×m2
Hp×m2 :=
G : C+ → Cp×m : G is analytic inC+ and
supξ>0
∫ ∞
−∞‖G(ξ + iω)‖2F dω <∞
.
Hp×m2 is a Hilbert space with the inner product
〈F,G〉H2:=
1
2π
∫ ∞
−∞tr(F (iω)
HG(iω)
)dω
and induced norm
‖G‖H2:= 〈G,G〉1/2H2
=
(1
2π
∫ ∞
−∞‖G(iω)‖2F dω
)1/2
.
Can show: ‖y − y‖L∞ ≤∥∥∥H− H
∥∥∥H2
‖u‖L2.
22 / 37
System Analysis | System Norms and Hardy Spaces
System Gramians andH2-trace-formula
A system (Σ) with Λ(E,A) ⊂ C− is called asymptotically stable. Then, all statetrajectories decay exponentially as t→∞ and
a) the infinite controllability and observability Gramians exist:
P =
∫ ∞
0
eE−1AtE−1BBTE−TeA
TE−Ttdt
ETQE =
∫ ∞
0
eATE−TtCTCeE
−1Atdt.
b) P,Q solve the two Lyapunov equations
APET + EPAT = −BBT, ATQE + ETQA = −CTC
c) theH2-norm can be expressed as
‖H‖2H2= tr
(CPCT
)= tr
(BTQB
).
23 / 37
System Analysis | Frequency-Domain Analysis
Bode Plots
The Bode plot forH consists of amagnitude plot and a phase plot.
Bode magnitude plot
• component-wise graph of the function |H(iω)| for frequenciesω ∈ [ωmin, ωmax] ⊂ R.• ω-axis is logarithmic.
• magnitude is given in decibels, i.e., |H(i.)| is plotted as 20 log10(|H(i.)|).
Bode phase plot
• component-wise graph of the function argH(iω) for frequenciesω ∈ [ωmin, ωmax] ⊂ R.• ω-axis is logarithmic.
• phase is given in degrees on a linear scale.
24 / 37
System Analysis | Frequency-Domain Analysis
Bode Plot for the Thermal Block Example
−400
−200
0
Magnitude
(dB)
10−3 10−2 10−1 100 101 102 103
−2,000
−1,000
0
Frequency (Hz)
Phase
25 / 37
System Analysis | Frequency-Domain Analysis
(Sigma) Magnitude Plots
Sigma magnitude plot
• 2-norm-wise graph of the functionH(iω) for frequenciesω ∈ [ωmin, ωmax] ⊂ R.• ω-axis is logarithmic.
The name is due to the fact that for a given matrixM the norm ‖M‖2 is given by itslargest singular value.
The real sigma magnitude plot depicts all singular values as functions of ω.
26 / 37
System Analysis | Frequency-Domain Analysis
Sigma Magnitude Plot for the Artificial Fishtail
10−4 10−3 10−2 10−1 100 101 102 103 10410−12
10−9
10−6
10−3
frequency (Hz)
Mag
nitu
de
27 / 37
Outline
1 Linear Time-Invariant (LTI) Systems
2 Transfer Function and Realizations
3 System Analysis
4 A Selection of MOR Methods
• Balancing Based MOR
• Moments and Interpolation
A Selection of MOR Methods | Balancing Based MOR
Balanced Truncation aka. Lyapunov Balancing
Idea:
• The system (Σ), in realization (E = I,A,B,C), is called balanced, if thesolutionsP, Q of the Lyapunov equations
AP + PAT + BBT = 0, ATQ + QA + CTC = 0,
satisfy: P = Q = diag(σ1, . . . , σn) where σ1 ≥ σ2 ≥ · · · ≥ σn > 0.
• σ1, . . . , σn are the Hankel singular values (HSVs) of Σ.• A balanced realization is computed via state space transformation
T : (I,A,B,C) 7→ (I,TAT−1,TB,CT−1)
=
([A11 A12
A21 A22
],
[B1
B2
],[C1 C2
]).
• Truncation reduced order model: (I, A, B, C) = (I,A11,B1,C1).
28 / 37
A Selection of MOR Methods | Balancing Based MOR
Balanced Truncation aka. Lyapunov Balancing
Idea:
• The system (Σ), in realization (E = I,A,B,C), is called balanced, if thesolutionsP, Q of the Lyapunov equations
AP + PAT + BBT = 0, ATQ + QA + CTC = 0,
satisfy: P = Q = diag(σ1, . . . , σn) where σ1 ≥ σ2 ≥ · · · ≥ σn > 0.
• σ1, . . . , σn are the Hankel singular values (HSVs) of Σ.
• A balanced realization is computed via state space transformation
T : (I,A,B,C) 7→ (I,TAT−1,TB,CT−1)
=
([A11 A12
A21 A22
],
[B1
B2
],[C1 C2
]).
• Truncation reduced order model: (I, A, B, C) = (I,A11,B1,C1).
28 / 37
A Selection of MOR Methods | Balancing Based MOR
Balanced Truncation aka. Lyapunov Balancing
Idea:
• The system (Σ), in realization (E = I,A,B,C), is called balanced, if thesolutionsP, Q of the Lyapunov equations
AP + PAT + BBT = 0, ATQ + QA + CTC = 0,
satisfy: P = Q = diag(σ1, . . . , σn) where σ1 ≥ σ2 ≥ · · · ≥ σn > 0.
• σ1, . . . , σn are the Hankel singular values (HSVs) of Σ.• A balanced realization is computed via state space transformation
T : (I,A,B,C) 7→ (I,TAT−1,TB,CT−1)
=
([A11 A12
A21 A22
],
[B1
B2
],[C1 C2
]).
• Truncation reduced order model: (I, A, B, C) = (I,A11,B1,C1).
28 / 37
A Selection of MOR Methods | Balancing Based MOR
Balanced Truncation aka. Lyapunov Balancing
Idea:
• The system (Σ), in realization (E = I,A,B,C), is called balanced, if thesolutionsP, Q of the Lyapunov equations
AP + PAT + BBT = 0, ATQ + QA + CTC = 0,
satisfy: P = Q = diag(σ1, . . . , σn) where σ1 ≥ σ2 ≥ · · · ≥ σn > 0.
• σ1, . . . , σn are the Hankel singular values (HSVs) of Σ.• A balanced realization is computed via state space transformation
T : (I,A,B,C) 7→ (I,TAT−1,TB,CT−1)
=
([A11 A12
A21 A22
],
[B1
B2
],[C1 C2
]).
• Truncation reduced order model: (I, A, B, C) = (I,A11,B1,C1).
28 / 37
A Selection of MOR Methods | Balancing Based MOR
Implementation: The Square Root Method
The SR Method ( : BTReductor)
1. Compute (Cholesky) factors of the solutions to the Lyapunov equation,
P = STS, Q = RTR.
2. Compute singular value decomposition
SRT = [U1, U2 ]
[Σ1
Σ2
] [VT
1
VT2
].
3. DefineW := RTV1Σ
−1/21 , V := STU1Σ
−1/21 .
4. Then the reduced order model is (WTAV,WTB,CV).
29 / 37
A Selection of MOR Methods | Balancing Based MOR
Implementation: The Square Root Method
The SR Method ( : BTReductor)
1. Compute (Cholesky) factors of the solutions to the Lyapunov equation,
P = STS, Q = RTR.
2. Compute singular value decomposition
SRT = [U1, U2 ]
[Σ1
Σ2
] [VT
1
VT2
].
3. DefineW := RTV1Σ
−1/21 , V := STU1Σ
−1/21 .
4. Then the reduced order model is (WTAV,WTB,CV).
29 / 37
A Selection of MOR Methods | Balancing Based MOR
Implementation: The Square Root Method
The SR Method ( : BTReductor)
1. Compute (Cholesky) factors of the solutions to the Lyapunov equation,
P = STS, Q = RTR.
2. Compute singular value decomposition
SRT = [U1, U2 ]
[Σ1
Σ2
] [VT
1
VT2
].
3. DefineW := RTV1Σ
−1/21 , V := STU1Σ
−1/21 .
4. Then the reduced order model is (WTAV,WTB,CV).
29 / 37
A Selection of MOR Methods | Balancing Based MOR
Properties
• Lyapunov balancing preserves asymptotic stability.
• We have the a priori error bound:∥∥∥H− H
∥∥∥H∞≤ 2
n∑k=r+1
σk
Variants ( : BRBTReductor, LQGBTReductor)Other versions for special classes of systems or applications exist, such as
• positive-real balancing, (passivity-preserving)
• bounded-real balancing, (contractivity-preserving)
• linear-quadratic Gaussian balancing. (stability preserving)(aims at low-order output feedback controllers)
The given ones all computeP, Q as solutions of algebraic Riccati equations of theform:
0 = APET + EPAT + BBT ± EPCTCPET
0 = ATQE + ETQA+ CTC ± ETQBBTQE.
30 / 37
A Selection of MOR Methods |Moments and Interpolation
Tools I
Lemma (Neumann series)
LetA ∈ Cn×n with spectral radius ρ(A) < 1 be given. Then I−A is invertible and itholds that
(I−A)−1
=
∞∑
k=0
Ak.
Will be important to identify the actual shape of Markov parameters and systemmoments.
31 / 37
A Selection of MOR Methods |Moments and Interpolation
Tools II
Definition ((polynomial) Krylov subpace)
Given an invertible matrixA ∈ Rn×n and a vector b ∈ Rn the k-dimensional(polynomial) Krylov subspace is defined as
Kk(A,b) := spanb,Ab,A2b, . . . ,Ak−1b
.
Definition (rational Krylov subpace)
Given an invertible matrixA ∈ Rn×n a vector b ∈ Rn and a vector of shifts s ∈ Rkthe k-dimensional rational Krylov subspace is defined as
Kk(A,b, s) := span
(s1I−A)−1
b, (s2I−A)−1
b, . . . , (skI−A)−1
b.
Orthonormal bases of these spaces should be computed via the Arnoldi iteration.
32 / 37
A Selection of MOR Methods |Moments and Interpolation
Padé-type approximations
Goal
Match the coefficientsMk(s0) orMk(∞) in
H(s) =
∞∑
k=0
(s− s0)kMk(s0) H(s) =
∞∑
k=0
s−kMk(∞)
Motivation (assume:m = p = 1, s large enough)
H(s) = C(sE−A)−1
B = 1sC
(I− 1
sE−1A
)−1︸ ︷︷ ︸=∑∞
k=01
sk(E−1A)k
E−1B
=
∞∑
k=1
C(E−1A)k−1
E−1B 1sk.
33 / 37
A Selection of MOR Methods |Moments and Interpolation
Padé-type approximations
Motivation (assume:m = p = 1, s large enough)
H(s) = C(sE−A)−1
B = 1sC
(I− 1
sE−1A
)−1︸ ︷︷ ︸=∑∞
k=01
sk(E−1A)k
E−1B
=
∞∑
k=1
C(E−1A)k−1
E−1B 1sk.
Therefore, we have
Mk(∞) =
0, if k = 0,
C(E−1A)k−1
E−1B, if k ≥ 1. useV = Kr(E−1A,E−1B)
33 / 37
A Selection of MOR Methods |Moments and Interpolation
Padé-type approximations
Approximation at∞
V = Kr(E−1A,E−1B), W = V orW = Kr(ATE−T,CT)
Approximation at s0 = 0
V = Kr(A−1E,A−1B), W = V orW = Kr(ETA−T,CT)
Approximation at s0 ∈ (0,∞)
V = Kr((s0E−A)−1
E, (s0E−A)−1
B), W = V
orW = Kr(ET(s0E
T −AT)−1,CT)
34 / 37
A Selection of MOR Methods |Moments and Interpolation
Multi-point Moment Matching, Interpolation and IRKA/TSIA
Approximation at s1, . . . , sr
V = Kr(s,E−1A,E−1B), W = V orW = Kr(s,ATE−T,CT).
• W = V as above matches first r moments of (Σ).• W 6= V as above matches first 2r moments of (Σ).• W 6= V as above actually achieves Hermite interpolation ofH, see, e.g., [2].
How do we choose s1, . . . , sr?H2-optimal MOR
Find s = [s1, . . . , sr]T, such that
∥∥∥H− H∥∥∥H2
is minimized.
IRKA iterative improvement of s using Λ(Ej , Aj).( : IRKAReductor)
TSIA run a fixed point iteration on the first order necessary conditions.( : TSIAReductor)
35 / 37
References
[1] A. C. Antoulas, Approximation of Large-Scale Dynamical Systems, vol. 6 of Adv. Des. Control,SIAM Publications, Philadelphia, PA, 2005,https://doi.org/10.1137/1.9780898718713.
[2] A. C. Antoulas, C. A. Beattie, and S. Gugercin, Interpolatory Methods for Model Reduction,Computational Science & Engineering, Society for Industrial and Applied Mathematics,Philadelphia, PA, 2020, https://doi.org/10.1137/1.9781611976083.
[3] S. Gugercin, T. Stykel, and S. Wyatt, Model reduction of descriptor systems by interpolatoryprojection methods, SIAM J. Sci. Comput., 35 (2013), pp. B1010–B1033,https://doi.org/10.1137/130906635.
[4] V. Mehrmann and T. Stykel, Balanced truncation model reduction for large-scale systems indescriptor form, in Dimension Reduction of Large-Scale Systems, P. Benner, V. Mehrmann, andD. C. Sorensen, eds., vol. 45 of Lect. Notes Comput. Sci. Eng., Springer-Verlag,Berlin/Heidelberg, Germany, 2005, pp. 83–115,https://doi.org/10.1007/3-540-27909-1_3.
[5] T. Stykel, Gramian-based model reduction for descriptor systems, Math. Control Signals Systems,16 (2004), pp. 297–319, https://doi.org/10.1007/s00498-004-0141-4.
[6] M. Voigt, Model reduction, Lecture Notes, Uni Hamburg, 2019,https://www.math.uni-hamburg.de/home/voigt/Modellreduktion_SoSe19/Notes_ModelReduction.pdf.
Questions?