Lecture Lecture Lecture Lecture –––– 11111111
Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) Linear Quadratic Regulator (LQR) –––– IIIIIIII
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Optimal Control, Guidance and Estimation
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore2
Outline
� Summary of LQR design
� Stability of closed-loop system in LQR
� Optimum value of the cost function
� Extension of LQR design• For cross-product term in cost function
• Rate of state minimization
• Rate of control minimization
• LQR design with prescribed degree of stability
� LQR for command tracking
� LQR for inhomogeneous systems
Summary of LQR DesignSummary of LQR DesignSummary of LQR DesignSummary of LQR Design
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore4
LQR Design:
Problem Statement
� Performance Index (to minimize):
� Path Constraint:
� Boundary Conditions:
( )( )
( )( )
0
,
1 1
2 2
f
f
t
T T T
f f f
t
L X UX
J X S X X Q X U RU dt
ϕ
= + +∫������� ���������
X A X BU= +ɺ
( )
( )00 :Specified
: Fixed, : Freef f
X X
t X t
=
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore5
LQR Design:Necessary Conditions of Optimality
� Terminal penalty:
� Hamiltonian:
� State Equation:
� Costate Equation:
� Optimal Control Eq.:
� Boundary Condition:
( ) ( )1
2
T T TH X Q X U RU AX BUλ= + + +
( ) ( )1
2
T
f f f fX X S Xϕ =
X AX BU= +ɺ
( ) ( )/T
H X QX Aλ λ= − ∂ ∂ = − +ɺ
( ) 1/ 0 TH U U R B λ−∂ ∂ = ⇒ = −
( )/f f f fX S Xλ ϕ= ∂ ∂ =
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore6
LQR Design:
Derivation of Riccati Equation
Guess ( ) ( ) ( )t P t X tλ =
( )
( )
( )
1
1
T
T
PX PX
PX P AX BU
PX P AX BR B
PX P AX BR B PX
λ
λ−
−
= +
= + +
= + −
= + −
ɺ ɺ ɺ
ɺ
ɺ
ɺ
( ) ( )
( )
1
1 0
T T
T T
QX A PX P PA PBR B P X
P PA A P PBR B P Q X
−
−
− + = + −
+ + − + =
ɺ
ɺ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore7
LQR Design:
Derivation of Riccati Equation
� Riccati equation
� Boundary condition
1 0T TP PA A P PBR B P Q
−+ + − + =ɺ
( ) ( ) is freef f f f f
P t X S X X=
( )f fP t S=
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore8
LQR Design:
Infinite Time Regulator Problem
Theorem (By Kalman)
Algebraic Riccati Equation (ARE)
Final Control Solution:
1 0T TPA A P PBR B P Q
−+ − + =
As , for constant and matrices, 0ft Q R P t→ ∞ → ∀ɺ
( )1 TU R B P X K X
−= − = −
Stability of closedStability of closedStability of closedStability of closed----loop system in LQRloop system in LQRloop system in LQRloop system in LQR
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore10
LQR Design:
Stability of Closed Loop System
� Closed loop system
� Lyapunov function
( )X AX BU A BK X= + = −ɺ
( ) TV X X PX=
( ) ( )
( ) ( )
( )
1 1
1 1
1
T T
T T
TT T T
T T T T
T T
V X PX X PX
A BK X PX X P A BK X
X A BR B P P P A BR B P X
X PA A P PBR B P Q Q PBR B P X
X Q PBR B P X
− −
− −
−
= +
= − + −
= − + −
= + − + − −
= − −
ɺ ɺ ɺ
00
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore11
LQR Design:
Stability of Closed Loop System
( )
( )
1
-1
-1
For 0, 0. Also 0
So 0
Also 0.
Hence, 0
0
Hence, the closed loop system is
always asymptotically stable!
T
T
R R P
PBR B P
Q
PBR B P Q
V X
−> > >
>
≥
+ >
∴ <ɺ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore12
LQR Design:
Minimum value of cost function
( )
( ) ( )
( )
( ) [ ]
( )
0
0
0
0 0
0
1 1
1
0 0 0 0
1
2
1
2
1
2
1 1 1
2 2 2
1 1
2 2
T T
t
TT T T
t
T T
t
T
t tt
T T T
J X Q X U RU dt
X Q X R B PX R R B PX dt
X Q PBR B P X dt
V dt V X PX
X PX X PX X PX
∞
∞
− −
∞
−
∞∞∞
∞ ∞
= +
= + − −
= +
= − = − = −
= − =
∫
∫
∫
∫ ɺ
Extensions of LQR DesignExtensions of LQR DesignExtensions of LQR DesignExtensions of LQR Design
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore14
LQR Extensions:
1. Cross Product Term in P.I.
( )0
12
2
T T T
tJ X QX X WU U RU dt
∞
= + +∫
Let us consider the expression:
( ) ( ) ( )
( )
1 1 1
2
TT T T T
T T T T T
T T T
X Q WR W X U R W X R U R W X
X QX U RU U W X X WU
X QX X WU U RU
− − −− + + +
= + + +
= + +
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore15
( ) ( ) ( )
( )
0
1 1
0
1 1 1
1 1 1
1
2
1
2
TT T T T
t
Q U
T T
t
J X Q WR W X U R W X R U R W X dt
X Q X U RU dt
∞− − −
∞
= − + + +
= +
∫
∫
������� �������
( )
( )
1
1
1
1
1 1
T
T
X AX BU
AX B U R W X
A BR W X BU
A X BU
−
−
= +
= + −
= − +
= +
ɺ
1
Control Solution
U K X= −
( )
1
1
1
T
T
U U R W X
K R W X
−
−
= −
= − +
LQR Extensions:
1. Cross Product Term in P.I.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore16
LQR Extensions:
2. Weightage on Rate of State
( )
( ) ( )
( ) ( ) ( )
0
0
0
1 1
0
1 1
1
2
1
2
1
2
12
2
12
2
T T T
t
TT T
t
T T T T T T
T T T Tt
Q R W
T T T T T T
t
T T
J X QX U RU X SX dt
X QX U RU AX BU S AX BU dt
X QX U RU X A S A X X A SBUdt
U B S A X U B SBU
X Q A SA X U R B SB U X A SB U dt
X Q X U RU
∞
∞
∞
∞
= + +
= + + + +
+ + +=
+ +
= + + + +
= + +
∫
∫
∫
∫
ɺ ɺ
����� ����� �����
( )0
T
tX WU dt
∞
∫Leads to a cross product case
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore17
LQR Extensions:
3. Weightage on Rate of Control
( )
( )
0
0
0
1 ˆ2
Let ,
01 ˆ02
1 ˆ ˆ2
T T T
T T
T T
J X QX U RU U RU dt
XV U
U
QJ V RV dt
R
J Q V RV dt
∞
∞
∞
= + +
= =
= +
= +
∫
∫
∫
X
X X
X X
ɺ ɺ
ɺ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore18
LQR Extensions:
3. Weightage on Rate of Control
( )
{ }
0
ˆˆ
, 0
0ˆ ˆ
0 0
(1) The dimension of the problem has increased
from to ( )
(2) If , is controllable, it can be shown that
the new syste
BA
X AX BU X X
U V
A BV A BV
I
n n m
A B
= + =
=
= + = +
+
X X X
Note :
ɺ
ɺ
ɺ
�����
m is also controllable.
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore19
[ ]
1
1
11 121 1
12 22
12 22
1 1
12 22
ˆ ˆ ˆ
ˆwhere is the solution of
ˆ ˆˆ ˆ ˆ ˆ ˆ ˆ ˆ 0
Hence
ˆ ˆˆ ˆ ˆ ˆ0
ˆ ˆ
ˆ ˆ ˆ ˆ
T
T T
T
T
T
V U R B P
P
A P PA PBR B P Q
P P XU R I R P P
UP P
R P X R P U
−
−
− −
− −
= = −
+ − + =
= − = −
= − −
Solution :
X
X
ɺ
ɺ
LQR Extensions:
3. Weightage on Rate of Control
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore20
1 1
12 22ˆ ˆ ˆ ˆHowever, is a dynamic
equation in and hence is not easy for implementation.
For this reason, we want an expression in the RHS
only as a function of and operations on it.
State
TU R P X R P U
U
X
− −= − −ɺ
( )( )
equation:
This suggests:
This is only an approximate solution, unless
X AX BU
U B X AX
m n
+
= +
= −
≥Note :
ɺ
ɺ
LQR Extensions:
3. Weightage on Rate of Control
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore21
( )
( )
12
1 1 1
12 22 22
1 1
12 22 22
1 2
1 2 0
0Proportional Initial conditio
Itegral
ˆ ˆ ˆ ˆ ˆ ˆ
ˆ ˆ ˆ ˆ ˆ
Integrating this expression both sides,
T
T
KK
t
U R P X R P B X R P B AX
R P P B A X R P B X
K X K X
U K X K X z dz U
− − + − +
− + − +
= − − −
= − + −
= − −
= − − +∫
ɺ ɺ ɺ
ɺ��������������
ɺ
�����
n
0 can be obtained using a performance index
without the term
U
U
Note :
ɺ
LQR Extensions:
3. Weightage on Rate of Control
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore22
LQR Extensions:
4. Prescribed Degree of Stability
Condition: All the Eiganvalues of the closed loop system
should lie to the left of line AB
( )( )
0
0
0
21 where, 0
2
1
2
1
2
Let
t T T
t
T Tt t t t
t
T T
t
t
t
J e X QX U RU dt
e X Q e X e U R e U dt
X QX U RU dt
X e X
U e U
α
α α α α
α
α
α∞
∞
∞
= + ≥
= +
= +
=
=
∫
∫
∫ ɶ ɶ ɶ ɶ
ɶ
ɶ
Co-ordinate
transformationα
A
B
jω
σ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore23
( )
( ) ( ) ( )
( )
t
t t
t
t t t
X e X e X
e AX BU e X
A e X B e U e X
X A I X BU
α
α α
α
α α α
α
α
α
α
= +
= + +
= + +
= + +
ɺɶ ɺ
ɺɶ ɶ ɶ
LQR Extensions:
4. Prescribed Degree of Stability
t t
U K X
e U K e X
U K X
α α
= −
= −
= −
ɶ ɶControl Solution:
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore24
LQR Extensions:
4. Prescribed Degree of Stability
( )
Modified System:
U K X
X A BK I Xα
= −
= − +
ɶ ɶ
ɺɶ ɶ ( )
Actual System:
U K X
X A BK X
= −
= −ɺ
( )
is designed in such a way that eigenvalues of
will lie in the left-half plane.
K
A BK Iα − +
( )Hence, eigenvalues of will lie to the left
of a line parallel to the imaginary axis, which is located
away by distance from the imaginary axis.
A BK
α
−
LQR Design for Command TrackingLQR Design for Command TrackingLQR Design for Command TrackingLQR Design for Command Tracking
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore26
LQR Design for
Command Tracking
Problem:
To design such that a part of the state vector of the
linear system tracks a commanded reference signal.
i.e. , where TT c
N
U
X AX BU
XX r X
X
= +
→ =
ɺ
Solution:
1) Formulate a standard LQR problem.
However, select the matrix properly.
2) Implement the controller as T c
N
Q
X rU K
X
−= −
0Typically
0 0
TTQ
Q
=
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore27
LQR Design for
Command Tracking
B
A
- K
+ Xɺ
+∫
LQ Regulation
XU
-
B
A
- K
+ Xɺ∫
+
T
N
X
X
( )
0
cr t
+
LQ Tracking
XU
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore28
LQR Design for Command
Tracking with Integral Feedback
Solution (with integral controller):
1) Augment the system dynamics with integral states
0
0
0 0 0
2) Select the matrix properly
(should penalize
T TT TN T T
N NT NN N N
I I
X A A X B
X A A X B U
X I X
Q
= +
ɺ
ɺ
ɺ
( ) ( )0
only and states)
3) Control solution
T I
TT
tT T
T c N T c
X X
U K X r X X r dt = − − −
∫
LQR Design for Inhomogeneous SystemsLQR Design for Inhomogeneous SystemsLQR Design for Inhomogeneous SystemsLQR Design for Inhomogeneous Systems
Prof. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant PadhiProf. Radhakant Padhi
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore30
LQR Design for
Inhomogeneous Systems
Reference
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore31
LQR Design for
Inhomogeneous Systems
� To derive the state of a linear (rather linearized) system to the origin by minimizing the following quadratic
performance index (cost function)
( ) ( )0
1 1
2 2
where
, 0 (psdf), 0 (pdf)
ft
T T T
f f f
t
f
J X S X X Q X U RU dt
S Q R
= + +
≥ >
∫
X A X BU C= + +ɺX
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore32
LQR Design for
Inhomogeneous Systems
� Performance Index (to minimize):
� Path Constraint:
� Boundary Conditions:
( ) ( )0
1 1
2 2
ft
T T T
f f f
t
J X S X X Q X U RU dt= + +∫
X A X BU C= + +ɺ
( )
( )00 :Specified
: Fixed, : Freef f
X X
t X t
=
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore33
LQR Design for
Inhomogeneous Systems
� Terminal penalty:
� Hamiltonian:
� State Equation:
� Costate Equation:
� Optimal Control Eq.:
� Boundary Condition:
( ) ( )1
2
T T TH X Q X U RU AX BU Cλ= + + + +
( ) ( )1
2
T
f f f fX X S Xϕ =
X AX BU C= + +ɺ
( ) ( )/ TH X QX Aλ λ= − ∂ ∂ = − +ɺ
( ) 1/ 0 TH U U R B λ−∂ ∂ = ⇒ = −
( )/f f f fX S Xλ ϕ= ∂ ∂ =
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore34
LQR Design for
Inhomogeneous Systems
Guess ( ) ( ) ( ) ( )t P t X t K tλ = +
( )
( )( )( ) ( )( )
1
1
T
T T
PX PX K
PX P AX BU C K
PX P AX BR B PC K
QX A PX K PX P AX BR B PX K PC K
λ
λ−
−
= + +
= + + + +
= + − + +
− + + = + − + + +
ɺ ɺ ɺ ɺ
ɺ ɺ
ɺ ɺ
ɺ ɺ
( )
( )
1
1 0
T T
T T
P PA A P PBR B P Q X
K A K PBR B P PC
−
−
+ + − +
+ + − + =
ɺ
ɺ
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore35
LQR Design for
Inhomogeneous Systems
� Riccati equation
� Auxiliary equation
� Boundary conditions
1 0T TP PA A P PBR B P Q
−+ + − + =ɺ
( ) ( ) ( ) is freef f f f f f
P t X K t S X X+ =
( )f fP t S=
( )1 0T TK A PBR B K PC
−+ − + =ɺ
( ) 0f
K t =
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore36
LQR Design for
Inhomogeneous Systems
Control Solution:
Note: There is a residual controller even after
This part of the controller offsets the continuous disturbance.
0.X →
( )
1
1
1 1
T
T
T T
U R B
R B PX K
R B PX R B K
λ−
−
− −
= −
= − +
= − −
OPTIMAL CONTROL, GUIDANCE AND ESTIMATION
Prof. Radhakant Padhi, AE Dept., IISc-Bangalore37
Thanks for the Attention….!!