Optimal Control

4SC000 Q2 2017-2018

Duarte Antunes

Recap

1

• Continuous-time optimal control problems

x(t) = f(x(t), u(t)), x(0) = x0, t 2 [0, T ]Z T

0g(x(t), u(t))dt+ gT (x(T ))

Dynamic model

Cost

• We have seen two simple versions of the PMP considering

• problems with no terminal constraints and final cost

• problems with terminal constraints and no final cost

gT (x(T ))

x(T ) = x

Recap

2

• In both problems, we have

(u⇤(t), x⇤(t))

x

⇤(t) = f(x⇤(t), u⇤(t))

�(t) = �( @

@x

f(x⇤(t), u⇤(t)))|�(t)� ( @

@x

g(x⇤(t), u⇤(t)))|

@@uf(x

⇤(t), u⇤(t))|�(t) + @@ug(x

⇤(t), u⇤(t))| = 0

�(T ) = @

@x

g

T

(x⇤(T ))

State eq.

Adjoint eq.

Control eq.

• The boundary conditions are different for the two problems

(no terminal state constraints)

(terminal constraints)

9�(t), t 2 [0, T ] is an optimal path candidate if s.t.

x

⇤(0) = x0

x

⇤(0) = x0 x

⇤(T ) = x

• It is also possible that only the terminal value of some variables are constrained, xi(T ) = xi, i 2 C

Outline

• Pontryagin’s maximum principle

• Examples

• Hamiltonian’s principle in mechanics

• Brachistochrone

• Minimum energy problems

• Minimal time problems for linear systems (bang-bang control)

3

Discussion

• Last lecture we used an informal discretization approach to derive a simplified version of the PMP.

• The Pontryagin’s maximum principle is more general and it allows to consider constraints and free terminal time problems. This requires a different mathematical framework in continuous-time.

CT PMP

CT DP

DT PMP

DT DP

Discretization, step ⌧ ⌧ ! 0

Taking the limit

Optimal path and

policy

Stage decision problem

CT control problem

Optimal path and

policy

The goals for today are

• State the general Pontryagin’s maximum principle.

• Solve minimal time problems with the PMP for linear system with input constraints.

4

Problem formulation

Control system

• Final time and final state can be free or fixed.

Cost functional to be minimized

• Initial time and initial state are fixed.

Assumptionsx(t) 2 Rn u 2 U ⇢ Rm

for all

• , control input may be constrained

• Lipschitz property: for every bounded set , s.t.9L

x0 = x(t0)

x(t) = f(x(t), u(t))

D 2 Rn ⇥ U

|f(x1, u)� f(x2, u)| L|x1 � x2|(x1, u), (x2, u) 2 D

• are continuous.

x(0) = x0

J(u) =R T0 g(x(t), u(t))dt

f, g, @f

@x

, @g

@x

t0 = 0

T xf := x(T )

5

Target set

• Depending on the control objective, the final time and final state can be free or fixed, or can belong to some set.

• All the possibilities are captured by introducing a closed target set and letting be the smallest time such that

• fixed-time, fixed end state for some fixed (previous lecture)

• fixed-time, free-end state (previous lecture)

• or in general for some surface in

• In particular only some of the state variables may be constrained.

• free-time, fixed end state

• or in general

• free-time, free end state (not so common)

S1 Rn

T (T, xf ) 2 S

S = {T}⇥ {xf} T, xf

S = {T}⇥ Rn

{T}⇥ S1

S ⇢ [0,1)⇥ Rn

S = [0,1)⇥ Rn

S = [0,1)⇥ {x1}

S = [0,1)⇥ S1

6

Target set

• We assume that , differentiable

• If the target set is never reached the cost is set to infinity.

S1 = {x 2 Rn : h1(x) = · · · = hn�k(x) = 0} hi : Rn ! R

• We can always consider since it captures not only free time-fixed end state problems ( ) but also fixed-time by considering time as a state

Then, for this new problem

• In particular,

• fixed-time, free end state

• fixed-time, fixed end state

xn+1 = t

x(t)

xn+1(t)

�=

f(x(t), u(t))

1

�

S = [0,1)⇥ S1

S = [0,1)⇥ {x1}

x(0)

xn+1(0)

�=

x0

0

�

S = [0,1)⇥ Rn ⇥ {t1}

S = [0,1)⇥ S1 ⇥ {t1}

S = [0,1)⇥ {x1}⇥ {t1}

7

Terminal cost• To consider the problem with terminal cost

note that

where

• Then

and so there is no loss of generality in our problem formulation.

x(0) = x0x(t) = f(x(t), u(t))

J(u) =R T0 g(x(t), u(t))dt+ gT (xf )

gT (xf ) = gT (0) +R T0

ddtgT (x(t))dt

d

dt

g

T

(x(t)) = @

@x

g

T

(x(t))x(t) = @

@x

g

T

(x(t))f(x(t), u(t))

J(u) =

R T0 g(x(t), u(t))dt+ constant

g(x, u) = g(x, u) + @

@x

g

T

(x)f(x, u)

8

Pontryagin’s maximum principle

Let be an optimal control (in the global sense) and let be the corresponding optimal state trajectory. Then there exist a function and a constant satisfying for all and having the following properties:

1) and satisfyx

⇤ p⇤

Consider an optimal control problem with target set .

H : Rn ⇥ U ⇥ Rn ⇥ R ! Rwith boundary conditions and , where is the Hamiltonian, defined as .

2) For each fixed , the function has a global maximum, i.e.,

hold for all and all .

3) for all

4) The vector is orthogonal to the tangent space to at

t

H(x⇤(t), u⇤(t), p⇤(t), p⇤0) � H(x⇤(t), u, p⇤(t), p⇤0)

(p⇤0(t), p⇤(t)) 6= (0, 0)

u ! H(x⇤(t), u, p⇤(t), p⇤0)

u 2 U

H(x⇤(t), u⇤(t), p⇤(t), p⇤0) = 0

S1

x⇤ =@

@pH(x⇤, u⇤, p⇤, p⇤0)

p⇤ = � @

@xH(x⇤, u⇤, p⇤, p⇤0)

p⇤0 0

S = [0,1)⇥ S1

x

⇤ : [0, T ] ! Rn

p⇤ : [0, T ] ! Rnu⇤ : [0, T ] ! U

t 2 [0, T ]

x

⇤(0) x

⇤(T ) 2 S1

H(x, u, p, p0) = p|f(x, u) + p0g(x, u)

t 2 [0, T ]

t 2 [0, T ]

p⇤(T ) x

⇤(T )

< p⇤(T ), d >= 0 8d, i :< d,

@

@x

hi(x⇤(T ))| >= 0

9

Discussion

• These are still only necessary conditions for optimality and not sufficient. However a local maximum or a local minimum cannot satisfy these conditions and thus these are necessary conditions for global optimality.

• The function should be interpreted as our previous costate, but multiplied by , . This is just a convention.

• The scalar is called the abnormal multiplier. We can replace except in degenerate cases which we will not address (in which case ).

• Accordingly, the Hamiltonian is defined in a slightly different manner

which is obtained by multiplying by our previous definition

p(t) = ��(t)�1

p0

�1

p⇤0 = �1p⇤0 = 0

H(x, u, p,�1) = p|f(x, u)� g(x, u)

H(x, u,�) = �|f(x, u) + g(x, u) = �p|f(x, u) + g(x, u) = �H(x, u, p,�1)

p(t) : [0, T ] ! Rn

10

Conditions 1 and 2

• The conditions stated in point 1) of the Theorem are nothing more that the equations for the state and the co-state that we have obtained before. In our previous notation

• The condition 2) states that has a (global) maximum. In our previous notation this would correspond to a minimum. If there are no input constraints and are differentiable with respect to (our previous setting) then we must have

Thus condition 2 is more general and shall allow us to handle input constraints

x⇤ =@

@pH(x⇤, u⇤, p⇤, p⇤0)

p⇤ = � @

@xH(x⇤, u⇤, p⇤, p⇤0)

u ! H(x⇤(t), u, p⇤(t), p⇤0)

x

⇤ = f(x⇤, u

⇤, p

⇤)

�

⇤ = � @

@x

f(x⇤, u

⇤,�

⇤)|�⇤ � @

@x

L(x⇤, u

⇤,�

⇤)|

u

@

@uH(x⇤, u, p⇤, p⇤0)|u=u⇤ = 0

@

@u

f(x⇤, u

⇤, p

⇤)|�⇤ +@

@u

g(x⇤, u

⇤, p

⇤)| = 0

(as in previous lectures)

(as in previous lectures)

f, g

11

Condition 3

• Condition 3 states that the Hamiltonian must be zero.

• For problems with fixed terminal time, recall the trick

where either or

• This trick leads to an extra variable for the costate and the Hamiltonian for the new problem, denoted by , is given by

• Thus is constant and is constant (as in previous lectures)

Thus condition 3 is more general and shall allow us to handle free time problems for which the Hamiltonian must be zero.

xn+1 = t

x(t)

xn+1(t)

�=

f(x(t), u(t))

1

�

pn+1(t)

pn+1(t) = � @

@xn+1H = 0

pn+1(t) H = 0 H

H

x(0)

xn+1(0)

�=

x0

0

�

S = [0,1)⇥ Rn ⇥ {t1} S = [0,1)⇥ {x1}⇥ {t1}

H =⇥p| pn+1

⇤ f1

�� g = p|f � g| {z }

H

+pn+1

12

Condition 4

• Condition 4 is called the transversality condition

• For fixed terminal time, fixed terminal state problems

free (as in previous lectures)

• For fixed terminal time, free terminal state problems there are no constraints so is free and

S1 = {x 2 Rn : h1(x) = · · · = hn�k(x) = 0} hi : Rn ! R

d

h(x(T )) = x(T )� xf = 0 =) d = 0 =) p

⇤(T )

p⇤(T ) = 0

< p⇤(T ), d >= 0 8d, i :< d,

@

@x

hi(x⇤(T ))| >= 0

13

Condition 4

• Recall that if

• Thus the Hamilton for the problem where we consider is

where can be shown to satisfy condition 1 of the theorem.

• Then (as in previous lectures)

Thus condition 4 is more general.

J(u) =

R T0 g(x(t), u(t))dt+ gT (xf ) =

R T0 g(x(t), u(t))dt+ constant

g(x, u) = g(x, u) + @

@x

g

T

(x)f(x, u)

p = p� @

@x

g

T

(x)|

H(x, p, u) = p|f(x, u)� (g(x, u) + @

@x

gT

(x)f(x, u))

p(T ) = 0 =) p(T ) = � @

@x

g

T

(x(T ))|

g

= p

|f(x, u)� g(x, u)

Historical note

The Pontryagin’s maximum principle was developed by the Pontryagin schools in the Soviet Union in the late 1950s. It was presented to the wider research community at the first IFAC World Congress in Moscow in 1960.

14

Lev Pontryagin (1908-1988) was a Soviet mathematician. He was born in Moscow and lost his eyesight due to a stove explosion when he was 14. Despite his blindness he was able to become one of the greatest mathematicians of the 20th century, partially with the help of his mother Tatyana Andreevna who read mathematical books and papers (source: wikipedia).

Outline

• Pontryagin’s maximum principle

• Examples

• Hamiltonian’s principle in mechanics

• Brachistochrone

• Minimum energy problems

• Minimal time problems for linear systems (bang-bang control)

15

Hamiltonian’s principle in Mechanics

The motion of a conservative system, from time to time is such that the integral

has a stationary value (typically a minimum), where

L = T (u, q)� V (q)

T

V

q

u = q

Lagrangian of the system

Principle of least action

Kinetic energy of the system

Potential energy of the system

Generalized velocity vector

Generalized coordinate vector (state of the system)

0 T

I =R T0 L(u, q)dt

16

Hamiltonian’s principle in Mechanics

The Hamiltonian is then

Consequently, the Euler-Lagrange equations are

� = �@H

@q= �@L

@qcostate eq.

control eq.

state eq. u = q

0 =@H

@u=

@L

@u+ � = 0

Combining the first two equations we obtain

d

dt(@L

@q)� @L

@q= 0

which are the Lagrange’s equations of motion for a conservative system.

H = L+ �u

17

Hamiltonian’s principle in Mechanics

If is not an explicit function of time, the Hamiltonian is constant

Interesting: nature governs the motion of bodies with optimal control!

L

H = L� @L

@uu = T � V � @T

@uu = const

One can show that the kinetic energy satisfies: (think of )T =1

2mv2

@T

@uu = 2T

Hence, we have

�H = T + V = const

that is, the kinetic plus potential energy is constant during the motion.

Outline

• Pontryagin’s maximum principle

• Examples

• Hamiltonian’s principle in mechanics

• Brachistochrone

• Minimum energy problems

• Minimal time problems for linear systems (bang-bang control)

18

Brachistochrone

Find a path between two points in a vertical plane such that a particle sliding without friction along this path takes theshortest possible time to travel from one point to the other.

x

y

a b

yb

Historical note

19

Johann Bernoulli posed the problem of the brachistochrone to the readers of Acta Eruditorum in June, 1696. Five mathematicians responded with solutions: Isaac Newton, Jakob Bernoulli (Johann's brother), Gottfried Leibniz, Ehrenfried Walther von Tschirnhaus and Guillaume de l'Hôpital. Four of the solutions (excluding l'Hôpital's) were published in the same edition of the journal as Johann Bernoulli's. In his paper Jakob Bernoulli showed that its solution is a cycloid. According to Newtonian scholar Tom Whiteside, Newton found the problem in his mail, in a letter from Johann Bernoulli, when he arrived home from the mint at 4 p.m., and stayed up all night to solve it and mailed the solution by the next post. This story gives some idea of Newton's power, since Johann Bernoulli took two weeks to solve it. In an attempt to outdo his brother, Jakob Bernoulli created a harder version of the brachistochrone problem. In solving it, he developed new methods that were refined by Leonhard Euler into what the latter called (in 1766) the calculus of variations. Joseph-Louis Lagrange did further work that resulted in modern infinitesimal calculus. (adapted from Wikipedia)

Johann Bernoulli  (1667 – 1748) was a Swiss mathematician, known for his contributions to infinitesimal calculus and educating Leonhard Euler. He was a professor of mathematics at the university of Groningen.

20

Problem formulationLet us relabel x ! t

y ! x

The initial kinetic and potential energy is zero, and we must have the following equation determining the velocity as a function of

Then the total time is the integral of the arc-length over the velocity

mv

2

2= mgx

x

Z b

a

p1 + (x(t))2p2gx(t)

dt

g = 9.8

21

Optimal controlZ b

a

p1 + u(t)2p2gx(t)

dt

x(t) = u(t)Dynamic model

Cost function

Hamiltonian H(x, u,�) = L(x, u) + �u

L(x, u)

Terminal constraints x(a) = 0 x(b) = xb

Optimality conditions

@H

@u= 0

H = constant

p1 + u

⇤(t)

2

p2gx

⇤(t)

� u

⇤(t)

2

p1 + u

⇤(t)

2p2gx

⇤(t)

= constant

22

Optimal solutionThe solutions to the differential equation

x

⇤(t) =

sC � x

⇤(t)

x

⇤(t)

are cycloids determined implicitly by

t(✓) = a+ c(✓ � sin(✓))

x(✓) = c(1� sin(✓))

These equations describe the curve traced by a point on a circle of radius as this circle rolls without slipping on the horizontal axis.

c

Outline

• Pontryagin’s maximum principle

• Examples

• Hamiltonian’s principle in mechanics

• Brachistochrone

• Minimum energy problems

• Minimal time problems for linear systems (bang-bang control)

23

Minimum energy control

How to move a motion system described by a linear equation from point A to point B with minimum energy

x(t) = Ax(t) +Bu(t)

x(0) = x0

x(T ) = xdesired

A

B

minuR T0 g(u(t))dt

Fx

Fy

yx

g(u) is convexAssumption: u 2 R

24

PMP

H(x, u,�) = g(u) + �|(Ax+Bu)

@

@uH(x, u,�) =

@

@ug(u) + �|B = 0

� = �[@

@xH(x, u,�)]| = �A|�

x = Ax+Bu

u = g�1u (��|B)

g�1u :=

@

@ug(u) is invertible since is convexg

�(t) = e�A|t�(0)

x(T ) = xdesiredimposing we obtain a linear system from which we can obtain �(0)

= g�1u (�B|�)

x(t) = e

Atx(0) +

Z t

0e

A(t�s)Bg

�1u (�B

|e

�A|s)ds�(0)

25

Example

A =

0 10 0

�B =

01

�(double integrator)

g(u) =1

4u4

x(0) =

00

�x(T ) =

10

�T = 1

solution:

g�1u (y) = sign(y)y1/3 sign(y) =

8><

>:

1 if y > 0

� 1 if y < 0

0 if y = 0

or equivalently

Substituting in the expression

10

�= x(1) = e

2

40 10 0

3

5

x(0)+

Z 1

0e

2

40 10 0

3

5(1�s) 01

�g

�1u (�

⇥1 0

⇤e

�

2

40 10 0

3

5s

�(0))ds

10

�=

Z 1

0

1� s1

�g�1u (s�1(0)� �2(0))ds

�1(0) = 2�2(0) = 203.54

26

Results

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

u

-5

-4

-3

-2

-1

0

1

2

3

4

5

abs((895185530450641 t)/4398046511104 - 895185530450641/8796093022208)1/3 (heaviside(t - 1) - heaviside(t - 1/2)) - (895185530450641/8796093022208 - (895185530450641 t)/4398046511104)1/3 (heaviside(t - 1/2) - heaviside(t))

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x 2 = d

p/dt

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(3 35807421218025641/3 heaviside(t - 1/2) (1 - 2 t)4/3)/262144 -...+ (3 35807421218025641/3 heaviside(t - 1) ((2 t - 1)4/3 - 1))/262144

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

x 1=p

0

0.2

0.4

0.6

0.8

1

(3 35807421218025641/3 heaviside(t) ((3 (1 - 2 t)7/3)/14 - 3/14))/262144 -...+ (3 35807421218025641/3 heaviside(t - 1) ((3 (2 t - 1)7/3)/14 - 3/14))/262144

time t time t

time t

27

Connection to controllability gramianSuppose that , the initial condition is zero and is large

x(t) = Ax(t) +Bu(t) x(T ) = xdesired

g(u) =1

2u2

x(0) = 0

min

Z T

0

1

2u(t)2dt

T (T ! 1)

Then is the identify, the optimal control input isg�1u

�(0)

u(t) = �B|e�A|t�(0)

and can be obtained from

x(T ) =

Z T

0e

A(T�s)BB

|e

�A|sds�(0) =

Z T

0eArBB|eA

|rdre�A|T�(0)

r = T � s

�(0) = e

A|TW (T )�1

x(T ) W (T ) =

Z T

0eArBB|eA

|rdr

28

Connection to controllability gramian

Z T

0

1

2u(t)2dt =

1

2x(T )|W (T )�1

Z T

0e

A(T�r)BB

|e

A|(T�r)dr

| {z }W (T )

W (T )�1x(T )

=1

2x(T )|W (T )�1

x(T )

W (T ) =

Z T

0eArBB|eA

|rdr

If is Hurwitz and is controllable, we can take the limitA (A,B) (T ! 1)

obtaining

which is the controllability gramian - solution to:

W1 =

Z 1

0eArBB|eA

|rdr

0 = AW1 +W1A| +BB|

Optimal cost

Outline

• Pontryagin’s maximum principle

• Examples

• Hamiltonian’s principle in mechanics

• Brachistochrone

• Minimum energy problems

• Minimal time problems for linear systems (bang-bang control)

Minimal time problemsConsider the problem

minT

x(t) = Ax(t) +Bu(t) t � 0x(0) = x0

x(T ) = x

(A,B)

subject to

• We assume that is controllable

• Without loss of generality we can assume x = 0

• We can write the cost as T =R T0 1dt

•For simplicity we assume u(t) 2 R

�c ui(t) c i 2 {1, . . . ,m} u(t) 2 Rm

•Without loss of generality we can assume c = 129

PMPMaximize

or equivalently, minimize

H(x(t), u(t),�(t)) = �(t)|(Ax(t) +Bu(t)) + 1

H(x(t), u(t), p(t),�1) = p(t)|(Ax(t) +Bu(t))� 1

�(t) = �p(t)

� = �@H@x

x = @H@�

terminal constraints ,

x(t) = Ax(t) +Bu(t)

x(0) = x0x(T ) = x

�(0),�(T ) free

the function

30

u ! H(x(t), u(t),�(t))is minimized when

In particularH(x(t), u(t),�(t)) = 0

t = T|�(T )|B| = 1

u(t) = �sign�(t)|B =

(� 1 if �(t)|B > 0

1 if �(t)|B 0u(t) = [�1, 1]

�(t) = �A|�(t)

31

Discussion

u(t) = �sign�(t)|B =

(� 1 if �(t)|B > 0

1 if �(t)|B 0

• The fact that

shows that the input always takes either the maximum or the minimum value.

• For a given initial condition, the control input will switch between the minimum and the maximum value until the state reaches the origin.

• For this reason, this is called bang-bang control.

• As explained in [Bryson,Hu, sec 3.9] there is a numerical procedure to find .

x(t) = Ax(t) +Bu(t)

u(t) = �sign�(t)|B =

(� 1 if �(t)|B > 0

1 if �(t)|B 0

• However, for two-dimensional systems one can typically find solutions by inspection.

• To obtain the optimal path we must find such that when we solve �(0)

we obtainx(T ) = 0

�(0)

�(t) = �A|�(t)

• We also show to solve by inspection a problem for a fourth dimensional system.

2D problem

32

Suppose that we wish to bring an object with a given initial position and velocity to rest at position zero in minimum time

subject to a constraint on the magnitude of the force

minT =

Z T

01dt

�1 u(t) 1 tfor all

(x1(0), x2(0)) x1(T ) = 0 x2(T ) = 0

x1 = 0x1

ux1(t) = x2(t)

x2(t) = u(t)

x(t) = Ax(t) +Bu(t) A =

0 10 0

�B =

01

�

Optimal solution

33

If is an optimal control trajectory, must minimize the Hamiltonian for each , i.e.,

{u⇤(t)|t 2 [0, T ]} u⇤(t)t

Therefore

u

⇤(t) = argmin�1u1[1 + �1(t)x⇤2(t) + �2(t)u]

u⇤(t) =

(1 if �2(t) < 0

�1 if �2(t) � 0

The equations describing the costate are

for some constants .

�2(t) = c2 � c1t�2(t) = ��1(t)

�1(t) = 0 �1(t) = c1

c1, c2

Thus there are only four possibilities.

and we must have . |�2(T )| = 1

Possibilities

34

u⇤(t) =

(1 if �2(t) < 0

�1 if �2(t) � 0�2(t) = c2 � c1t

0 T t

�2(t)

0 T t

�2(t)

0 T t

�2(t)

0 T t

�2(t)

0 T t

�1

1

u⇤(t)

0 T t

�1

1u⇤(t)

0 T t

�1

1

u⇤(t)

0 T t

�1

1u⇤(t)

|�2(T )| = 1

A B C D

Possibilities

35

Possibilities A and C correspond to cases where the origin is achieved by setting the actuation constant ( or ) for a given initial conditionu(t) = 1u(t) = �1

0 = e

ATx0 +

R T0 e

A(T�s)B(�1)ds 0 = e

ATx0 +

R T0 e

A(T�s)Bds

u(t) = �1

0 x1

x2

u(t) = 1

or

x0 =R 0T e

A(�s)B(�1)ds

x0 =R 0T e

A(�s)Bds

x0 =

12T

2

�T

�x0 =

� 1

2T2

T

�

Control trajectories

36

u(t) = ⇠ ⇠ 2 {+1,�1} x1(t) = x1(0) + x2(0)t+ ⇠

t

2

2x2(t) = x2(0) + ⇠t

x1(t)�1

2⇠

(x2(t))2= x1(0)�

1

2⇠

(x2(0))2= constant

0 x1

x2 u(t) = 1 u(t) = �1

0 x1

x2

Switching curve

u(t) = �10 x1

x2

u(t) = 1

(x1(0), x2(0)) trajectories switch when they hit the switching curve

For an initial condition not lying in this curve, we know (from PMP co-state) that the control input can only switch once.

37

Minimum time control for a quadcopter

How to move a (linearized model of a) quadcopter from one hovering position to another one in minimum time?

x(t) = Ax(t) +Bu(t)

x(0) = x0

x(T ) = xdesired

minT

x0 xT

|u(t)| L

38

Model

Linearized model (y-axis)

Thrust

Gravity

y =1

mT sin(✓)

z =

1

mT cos(✓)� gT

y

z ✓

mg

mass

✓ =1

I✓⌧✓

x =⇥y y ✓ ✓

⇤|

x(t) = Ax(t) +Bu(t)

B =

2

664

0001

3

775u =

⌧✓I✓

x(0) =⇥0 0 0 0

⇤|x(T ) =

⇥y 0 0 0

⇤|

|u(t)| L

⇡ g✓

T ⇡ mg

A =

2

664

0 1 0 00 0 g 00 0 0 10 0 0 0

3

775

39

Optimal control input

u(t) = �L sign( a3|{z}��1(0)

6I✓

t3 + a2|{z}�2(0)2I✓

t2 + a1|{z}��3(0)

t+ a0|{z}�4(0)

)

|u(t)| = �L sign(�(t)|B) = �L sign(�4(t))

Polynomial of order 3 has at most 3 roots so there are at most three sign changes in the optimal control input

�(t) = e�A|t�(0) =

2

664

1 0 0 0�t 1 0 0gt2

2 �gt 1 0

� gt3

6gt2

2 �t 1

3

775

2

664

�1(0)�2(0)�3(0)�4(0)

3

775

40

Optimal control input

0 TT1 T2 T3

0 TT1 T2 T3

time

time

u(t) = �L sign(a3t3 + a2t

2 + a1t+ a0)

Conclusion: search over is equivalent to search over switching times (and sign of the control input)

�(0)Ti

0 TT1 T2 T3

0 TT1 T2 T3

time

time

or

41

ResultsFor u =

⌧✓I✓

|u(t)| L I✓ = 0.002 L = 50 y = 1

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

u

-60

-40

-20

0

20

40

60100 heaviside(t - 941/2000) -...- 100 heaviside(t - 4964993749729043/36028797018963968)

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

d/dt3

-10

-8

-6

-4

-2

0

2

4

6

8(45215787882993825 heaviside(t - 1808631515319753/2251799813685248))/562949953421312 -...- 100 t heaviside(t - 1808631515319753/2251799813685248)

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

3

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

(885481 heaviside(t - 941/2000))/80000 -...+ (45215787882993825 t heaviside(t - 1808631515319753/2251799813685248))/562949953421312

T2 = 0.4705

T3 = 0.8032

T ⇤ = 0.941

T1 = 0.1378

time t time t

time t

42

Results

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

v

0

0.5

1

1.5

2

2.5

(483164605226563795468286475929144999878116613455 heaviside(t - 1808631515319753/2251799813685248))/5708990770823839524233143877797980545530986496 -...- (801431249760917353029788835347205 t heaviside(t - 1808631515319753/2251799813685248))/2535301200456458802993406410752

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

y0

0.2

0.4

0.6

0.8

1

(38419753466689 heaviside(t - 941/2000))/19200000000000 -...+ (483164605226563795468286475929144999878116613455 t heaviside(t - 1808631515319753/2251799813685248))/5708990770823839524233143877797980545530986496

time t time t

43

ResultsFor u =

⌧✓I✓

|u(t)| L I✓ = 0.002 y = 1L = 100

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

u

-100

-50

0

50

100

200 heaviside(t - 791/2000) -...- 200 heaviside(t - 6081307241734005/9007199254740992)

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

d/dt3

-15

-10

-5

0

5

10

(208677473753223875 heaviside(t - 8347098950128955/72057594037927936))/9007199254740992 -...- 200 t heaviside(t - 6081307241734005/9007199254740992)

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

3

-1.5

-1

-0.5

0

0.5

1

1.5

(625681 heaviside(t - 791/2000))/40000 -...- 100 t2 heaviside(t - 8347098950128955/72057594037927936)

T2 = 0.39558

T3 = 0.6752

T ⇤ = 0.7910

T1 = 0.1158

time t time t

time t

44

Results

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

v

0

0.5

1

1.5

2

2.5

3

(142486188710187787663341665948145434555802153049375 heaviside(t - 8347098950128955/72057594037927936))/280608314367533360295107487881526339773939048251392 -...- (980 t3 heaviside(t - 8347098950128955/72057594037927936))/3

t0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

y0

0.2

0.4

0.6

0.8

1

(19182358974289 heaviside(t - 791/2000))/9600000000000 -...- (245 t4 heaviside(t - 8347098950128955/72057594037927936))/3

time t time t

Concluding remarks

45

• Pontryagin’s maximum principal is a very elaborate theorem and the proof is far from trivial

• We can tackle minimum time problems with the Pontryagin’s maximum principle.

• For systems with linear dynamics this leads to bang-bang control.

• For 2D systems we can solve these by inspection by determining switching curves.

Summary

After this lecture you should be able to

• Solve simple minimum time-problems for linear systems with a two-dimensional state.