Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
No-gap second-order optimality conditions foroptimal control problems with state constraints
Application to the shooting algorithm
Audrey Hermant
CMAP Ecole Polytechnique and INRIA Futurs, France
13th Czech-French-German Conference on OptimizationHeidelberg, September 19, 2007
Joined work with J. Frederic Bonnans
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 1/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Outline
Introduction
Definitions
Regularity
Second-order analysis
Shooting algorithm
Remarks & conclusion
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 2/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Optimal control problem
(P) min(u,y)
∫ T
0`(u(t), y(t))dt + φ(y(T ))
s.t. y(t) = f (u(t), y(t)) a.e. [0,T ], y(0) = y0 (1)
gi (y(t)) ≤ 0 on [0,T ], i = 1, . . . , r
ci (u(t), y(t)) ≤ 0 a.e. [0,T ], i = r + 1, . . . , r + s.
I Control u ∈ U := L∞(0,T ; Rm),state y ∈ Y := W 1,∞(0,T ; Rn).
I Assumption (A0) Data `, φ, f , g , c of class C∞, f Lipschitzcontinuous, gi (y0) < 0, ∀ i = 1, . . . , r .
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 3/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Problem 1: second-order optimality conditions
I Second-order sufficient conditions (SSC) are useful to: showlocal optimality of solutions, stability/sensitivity analysis,convergence of algorithms.
I SSC as weak as possible if as close as possible to thesecond-order necessary condition (’no-gap’).
I No-gap conditions known for control constraints, mixedcontrol-state constraints (Osmolovskii).
Pure state constraints: is there a gap?Cf Kawasaki-Pales-Zeidan (necessary cond., additional term),Malanowski-Maurer (SSC).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 4/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Problem 2: Well-posedness of the shooting algorithm
I Useful algorithm to obtain solutions with a high precision andlow complexity.
I Principle: reduce the problem to a multi-points boundaryvalue problem and solve the finite-dimensional shootingequation using a Newton method.
I Theoretical difficulties due to pure state constraints:reformulation of the optimality conditions, the algorithm takesinto account only a part of the optimality conditions.
Is this algorithm well-posed? (Jacobian of the shooting mappinginvertible).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 5/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Structure of the contact set
I Contact set : I (gi (y)) := {t ∈ [0,T ] : gi (y(t)) = 0}.I Junction points of gi : Ti = ∂I (gi (y)).
g(y(t))
t
boundary arc [τ ien, τ
iex ] touch point {τ i
to}
τ ien : entry point, τ i
ex : exit point.
I Similar definitions for mixed control-state constraints.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 6/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Order of a state constraint
Definition
The order of the state constraint gi , denoted by qi , is the smallestnumber of derivation of t → gi (y(t)), when y satisfies y = f (u, y),to have an explicit dependence in u.
I More precisely, the time derivatives of gi satisfy:
g(j)i (u, y) = g
(j−1)i ,y (y)f (u, y) = g
(j)i (y), j = 1, . . . , qi − 1
g(qi )i (u, y) = g
(qi−1)i ,y (y)f (u, y), g
(qi )i ,u 6≡ 0.
I For mixed control-state constraints, we set
qi := 0, g(qi )i (u, y) := ci (u, y), i = r + 1, . . . , r + s.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 7/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Linear Independence Condition
Set Iε(t) := {i : gi (y(t)) ≥ −ε, ci (u(t), y(t)) ≥ −ε} and
Γ(t) :=(∇ug
(qi )i (u(t), y(t))
)i∈Iε(t)
.
(LIC ) ∃ γ, ε > 0, γ|ξ| ≤ |Γ(t)ξ|, ∀ξ ∈ R|Iε(t)|, ∀t ∈ [0,T ] .
Proposition (Normal form)
Assume that (LIC) holds and u is continuous. Then there exists alocal change of variables z = Φ(y), v = Ψ(u, y) such that, in thenew coordinates, the dynamics and the constraints write locally{
z(qi )i = vi , i = 1, . . . , r
˙z = f (v , z)
zi ≤ 0, i = 1, . . . , r ,vi ≤ 0, i = r + 1, . . . , r + s.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 8/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
First-order optimality condition
Hamiltonian H : Rm × Rn × Rn∗ × Rs∗ → R,
H(u, y , p, λ) := `(u, y) + pf (u, y) + λc(u, y).
Definition
(u, y) stationary point of (P), if exist p ∈ BV ([0,T ]; Rn∗),dη ∈M([0,T ]; Rr∗) and λ ∈ L∞(0,T ; Rs∗) satisfying (1),
−dp = Hy (u, y , p, λ)dt + dηgy (y), p(T ) = φy (y(T ))
0 = Hu(u(t), y(t), p(t), λ(t)) a.a. t ∈ [0,T ]
0 ≥ g(y(t)), dη ≥ 0,∫ T0 dη(t)g(y(t)) = 0,
0 ≥ c(u(t), y(t)), λ ≥ 0,∫ T0 λ(t)c(u(t), y(t))dt = 0.
(u, y) local solution + (LIC) => (u, y) stationary point .
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 9/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Assumptions
(A1) The control u is continuous on [0,T ] and the strenghtenedLegendre-Clebsch condition holds: ∃ α > 0,
Huu(u(t), y(t), p(t), λ(t))(v , v) ≥ α|v |2, ∀ v ∈ Rm, ∀ t ∈ [0,T ].
(A2) The Linear Independance Condition (LIC) holds.
(A3) The set of junction times Ti is finite ∀i = 1, . . . , r + s(⇒ finitely many boundary arcs and touch points).
(A4) The junction times of state constraints do not coincide, i.e.i 6= j ⇒ Ti ∩ Tj = ∅, and gi (y(T )) < 0, ∀ i .
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 10/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Remark: Sufficient condition for the continuity of u
Proposition
Assume that
I The Hamiltonian is uniformly strongly convex w.r.t. u and themixed control-state constraints are convex w.r.t. u, i.e.
∃ α > 0, Huu(u, y(t), p(t), λ)(v , v) ≥ α|v |2,
∀ u, v ∈ Rm, ∀ λ ∈ Rs∗+ , ∀ t ∈ [0,T ].
I The linear independence condition holds for mixedcontrol-state constraints.
Then u is continuous on [0,T ].
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 11/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Regularity and junction conditions
Proposition
Let (u, y) be a stationary point of (P) satisfying (A1)-(A4). Then
I Outside the set of junction times, u, y , p, η, λ are C∞.
I The multipliers λi , ηi associated with constraints of orderqi = 0, 1 are continuous on [0,T ].
I Let τ ∈ Ti . If qi ≥ 3:
• the time derivatives of u are continuous until order qi − 2.
• If qi is odd, and τ is an entry/exit point, the timederivatives of u are continuous until order qi − 1.
Ref : Jacobson et al. (71) in the scalar case m = r = 1.Maurer (79) for the first item in the case when m = r .
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 12/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Regularity and junction conditions
Proposition
Let (u, y) be a stationary point of (P) satisfying (A1)-(A4). Then
I Outside the set of junction times, u, y , p, η, λ are C∞.
I The multipliers λi , ηi associated with constraints of orderqi = 0, 1 are continuous on [0,T ].
I Let τ ∈ Ti . If qi ≥ 3:
• the time derivatives of u are continuous until order qi − 2.
• If qi is odd, and τ is an entry/exit point, the timederivatives of u are continuous until order qi − 1.
Ref : Jacobson et al. (71) in the scalar case m = r = 1.Maurer (79) for the first item in the case when m = r .
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 12/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Regularity and junction conditions
Proposition
Let (u, y) be a stationary point of (P) satisfying (A1)-(A4). Then
I Outside the set of junction times, u, y , p, η, λ are C∞.
I The multipliers λi , ηi associated with constraints of orderqi = 0, 1 are continuous on [0,T ].
I Let τ ∈ Ti . If qi ≥ 3:
• the time derivatives of u are continuous until order qi − 2.
• If qi is odd, and τ is an entry/exit point, the timederivatives of u are continuous until order qi − 1.
Ref : Jacobson et al. (71) in the scalar case m = r = 1.Maurer (79) for the first item in the case when m = r .
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 12/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Definitions
Definition
A touch point of a state constraint τ ∈ Ti is essential, if[ηi (τ)] > 0. The set of essential touch points is denoted by T ess
i .
Remark : qi = 1 ⇒ no essential touch points (T essi = ∅).
Definition
A touch point τ of a state constraint of order qi ≥ 2 is reducible, if
g(2)i (u(τ), y(τ)) < 0.
We denote by T redi a finite set (possibly empty) of essential and
reducible touch points.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 13/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Assumptions
(A5) • For all entry/exit times τ ∈ Ti of state constraints:if qi is odd (resp. even), the derivative of order 2qi (resp.2qi − 1) of t → gi (y(t)) is discontinuous at τ .
• All essential touch points τ ∈ T essi (qi ≥ 2) are reducible,
i.e.g
(2)i (u(τ), y(τ)) < 0.
(A6) Strict complementarity on boundary arcs for state constraints:
dηi
dt> 0 a.e. on int I (gi (y)).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 14/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Abstract formulation
I For u ∈ U , let yu ∈ Y be the unique solution of the stateequation (1) yu = f (u, yu), yu(0) = y0, and let
J(u) :=∫ T0 `(u, yu)dt + φ(yu(T )), G (u) := g(yu),
G(u) := c(u, yu), K := C ([0,T ]; Rr−), K := L∞(0,T ; Rs
−).Then (P) writes
(P) minu∈U
J(u), G (u) ∈ K , G(u) ∈ K.
I Lagrangian
L(u; η, λ) := J(u) + 〈η, G (u)〉+ 〈λ,G(u)〉.
I Given v ∈ V := L2(0,T ; Rm), denote by zv the solution of
zv = fy (u, yu)zv + fu(u, yu)v a.e. [0,T ], zv (0) = 0.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 15/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Quadratic form and critical cone
I Set Q(v) := D2uuL(u; η, λ)(v , v) given by
Q(v) =
∫ T
0D2
(u,y)(u,y)H(u, y , p, λ)((v , zv ), (v , zv ))dt
+
∫ T
0gyy (y)(zv , zv )dη + φyy (y(T ))(zv (T ), zv (T )).
I Critical cone C (u): set of v ∈ V satisfying
gi ,y (y)zv = 0 on supp(dηi ),
gi ,y (y)zv ≤ 0 on I (gi (y)) \ supp(dηi ),
ci ,y (u, y)zv + ci ,u(u, y)v = 0 a.e. on supp(λi ),
ci ,y (u, y)zv + ci ,u(u, y)v ≤ 0 a.e. on I (ci (u, y)) \ supp(λi ).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 16/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Second-order necessary optimality condition
Theorem
Let (u, y) be a local solution of (P) satisfying (A1)-(A6). Then
Q(v)−r∑
i=1
∑τ∈T ess
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
≥ 0, ∀v ∈ C (u).
Idea of proof
I Computation of the curvature term by Kawasaki 88, 90 forstate constraints.
I No contribution of mixed constraints (polyhedricity).
I Using the normal form, we combine both arguments.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 17/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Second-order necessary optimality condition
Theorem
Let (u, y) be a local solution of (P) satisfying (A1)-(A6). Then
Q(v)−r∑
i=1
∑τ∈T ess
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
≥ 0, ∀v ∈ C (u).
Idea of proof
I Computation of the curvature term by Kawasaki 88, 90 forstate constraints.
I No contribution of mixed constraints (polyhedricity).
I Using the normal form, we combine both arguments.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 17/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Second-order sufficient optimality condition
Theorem
Let (u, y) be a stationary point satisfying (A1). If
Q(v)−r∑
i=1
∑τ∈T red
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
> 0, ∀v ∈ C (u) \ {0}
then (u, y) is a local solution of (P) satisfying the quadraticgrowth condition: ∃ β, r > 0 such that:
J(u) ≥ J(u) + β ‖u − u‖22 ∀ u ∈ U : G (u) ∈ K , ‖u − u‖∞ < r .
Idea of proof: Use of a reduction approach (cf semi-infiniteprogramming).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 18/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Second-order sufficient optimality condition
Theorem
Let (u, y) be a stationary point satisfying (A1). If
Q(v)−r∑
i=1
∑τ∈T red
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
> 0, ∀v ∈ C (u) \ {0}
then (u, y) is a local solution of (P) satisfying the quadraticgrowth condition: ∃ β, r > 0 such that:
J(u) ≥ J(u) + β ‖u − u‖22 ∀ u ∈ U : G (u) ∈ K , ‖u − u‖∞ < r .
Idea of proof: Use of a reduction approach (cf semi-infiniteprogramming).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 18/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Characterization of quadratic growth
Corollary
Let (u, y) be a stationary point satisfying (A1)-(A6). Then
Q(v)−r∑
i=1
∑τ∈T ess
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
> 0, ∀v ∈ C (u) \ {0}
iff (u, y) is a local solution of (P) satisfying the quadratic growthcondition.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 19/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
The shooting algorithm (unconstrained case)
I By (A1), Hu(u(t), y(t), p(t)) = 0 iff u(t) = Υ(y(t), p(t)).I The first-order optimality condition writes (two-points
boundary value problem):
y = f (Υ(y , p), y), y(0) = y0
−p = Hy (Υ(y , p), y , p), p(T ) = φy (y(T )).
I Shooting algorithm: Find a zero of the shooting mappingp0 → p(T )− φy (y(T )) with
y = f (Υ(y , p), y), y(0) = y0
−p = Hy (Υ(y , p), y , p), p(0) = p0.
I Constrainted case: when the structure of the trajectory isknown, introduce junction times as unknown of the shootingmapping, as well as jump parameters of the costate for stateconstraints (Bryson et al. 63).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 20/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
The shooting algorithm (unconstrained case)
I By (A1), Hu(u(t), y(t), p(t)) = 0 iff u(t) = Υ(y(t), p(t)).I The first-order optimality condition writes (two-points
boundary value problem):
y = f (Υ(y , p), y), y(0) = y0
−p = Hy (Υ(y , p), y , p), p(T ) = φy (y(T )).
I Shooting algorithm: Find a zero of the shooting mappingp0 → p(T )− φy (y(T )) with
y = f (Υ(y , p), y), y(0) = y0
−p = Hy (Υ(y , p), y , p), p(0) = p0.
I Constrainted case: when the structure of the trajectory isknown, introduce junction times as unknown of the shootingmapping, as well as jump parameters of the costate for stateconstraints (Bryson et al. 63).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 20/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Assumptions
(A7) • Mixed constraints have no touch points.
• For all mixed constraints, ddt ci (u, y) is discontinuous at
entry and exit points.
(A8) Strict complementarity:
supp(dηi ) = I (gi (y)), supp(λi ) = I (ci (u, y)).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 21/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Well-posedness of the shooting algorithm
Theorem
Let (u, y) be a local solution of (P) satisfying (A1)-(A8). Thenthe shooting algorithm is well-posed in the neighborhood of (u, y)(invertible Jacobian of the shooting mapping), iff:(i) State constraints of order qi ≥ 3 have no boundary arc;(ii) The no-gap sufficient second-order condition
Q(v)−r∑
i=1
∑τ∈T ess
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
> 0, ∀v ∈ C (u) \ {0}
holds, i.e. (u, y) satisfies the quadratic growth condition.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 22/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Verification of sufficient second-order optimality condition
Open problem!Using Riccati equations (cf Maurer), assuming (A1), we can checkthe stronger condition below:
Q(v)−r∑
i=1
∑τ∈T red
i
[ηi (τ)](g
(1)i ,y (y(τ))zv (τ))2
g(2)i (u(τ), y(τ))
> 0, ∀v ∈ C (u) \ {0}
where C (u) ⊃ C (u) is the set of v ∈ V satisfying
ci ,y (u, y)zv + ci ,u(u, y)v = 0 a.e. on boundary arcs,
g(qi )i ,y (u, y)zv + g
(qi )i ,u (u, y)v = 0 a.e. on boundary arcs.
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 23/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Constraints on the final state
Extension possible to finitely many equality and inequalityconstraints on the final state
Ψeq(y(T )) = 0, Ψin(y(T )) ≤ 0 (2)
if we assume in addition a strong controllability condition:for κ = 2,∞, for all ϕ ∈
∏i W
qi ,κ(0,T )× Lκ(0,T ; Rs) and allµ ∈ R|Ψac |, there exists v ∈ Lκ(0,T ; Rm) such that
gi ,y (y)zv = ϕi on a neighborhood of I (gi (y))
ci ,y (u, y)zv + ci ,u(u, y)v = ϕi on a neighborhood of I (ci (u, y))
DΨac(y(T ))zv (T ) = µ,
with Ψac the equality and active inequality components of (2).
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 24/25
Introduction Definitions Regularity Second-order analysis Shooting algorithm Remarks & conclusion
Conclusion & Outlook
I We obtain no-gap second-order optimality conditions for purestate constraints of arbitrary orders and mixed control-stateconstraints, and a characterization of the well-posedness ofthe shooting algorithm.
I Outlook: Numerical applications of the shooting algorithm,using homotopy/continuation methods to automaticallydetect the structure of the trajectory and initialize some of theshooting parameters.
Reference of this talk: J.F. Bonnans, A.H., Second-order analysis for
optimal control problems with pure and mixed state constraints, INRIA
Research Report 6199 (2007), submitted.
http://hal.inria.fr/inria-00148946
CFG 07 A. Hermant Second-order optimality conditions for optimal control problems with state constraints 25/25