8/8/2019 SLafrica1

1/37

An Introduction to OptimalControl Applied to DiseaseModels

Suzanne Lenhart

University of Tennessee, Knoxville

Departments of Mathematics

Lecture1 p.1/3

8/8/2019 SLafrica1

2/37

Example

Number of cancer cells at time

(exponential growth) State

Drug concentration Control

known initial data

minimize

where the first term represents number of cancer

cells and the second term represents harmfuleffects of drug on body.

Lecture1 p.2/3

8/8/2019 SLafrica1

3/37

Optimal Control

Adjust controls in a system to achieve a goalSystem:

Ordinary differential equations

Partial differential equations

Discrete equations

Stochastic differential equations

Integro-difference equations

Lecture1 p.3/3

8/8/2019 SLafrica1

4/37

Deterministic Optimal Control

Control of Ordinary Differential Equations (DE)

control state

State function satisfies DE

Control affects DE

!

"

#

$

$

$

Goal (objective functional)

Lecture1 p.4/3

8/8/2019 SLafrica1

5/37

Basic Idea

System of ODEs modeling situation

Decide on format and bounds on the controlsDesign an appropriate objective functional

Derive necessary conditions for the optimal control

Compute the optimal control numerically

Lecture1 p.5/3

8/8/2019 SLafrica1

6/37

Design an appropriate objective functional

balancing opposing factors in functionalinclude (or not) terms at the final time

Lecture1 p.6/3

8/8/2019 SLafrica1

7/37

Big Idea

In optimal control theory, after formulating aproblem appropriate to the scenario, there areseveral basic problems :

(a) to prove the existence of an optimal control,

(b) to characterize the optimal control,

(c) to prove the uniqueness of the control,

(d) to compute the optimal control numerically,(e) to investigate how the optimal control

depends on various parameters in the model.

Lecture1 p.7/3

8/8/2019 SLafrica1

8/37

Deterministic Optimal Control- ODE

Find piecewise continuous control %&

'

(

andassociated state variable )

&

'

(

to maximize

0

1

2

3

4

&

'

5

)

&

'

(

5

%

&

'

( (

6

'

subject to

)

7

&

'

(

8

9

&

'

5

)

&

'

(

5

%

&

'

( (

)

&

@

(

8

)

4and )& (

AB B

Lecture1 p.8/3

8/8/2019 SLafrica1

9/37

Contd.

Optimal Control CD

E

F

G

achieves the maximum

PutC

D

E

F

G

into state DE and obtainH

D

E

F

G

H

D

E

F

G

corresponding optimal stateC

D

E

F

G

, HD

E

F

G

optimal pair

Lecture1 p.9/3

8/8/2019 SLafrica1

10/37

Necessary, Sufficient Conditions

Necessary ConditionsIf I

P

Q

R

S

, TP

Q

R

S

are optimal, then the following

conditions hold...

Sufficient ConditionsIf I

P

Q

R

S

, TP

Q

R

S

andU

(adjoint) satisfy the conditions...then I

P

Q

R

S

, TP

Q

R

S

are optimal.

Lecture1 p.10/3

8/8/2019 SLafrica1

11/37

Adjoint

like Lagrange multipliers to attach DE to objective func-

tional.Lecture1 p.11/3

8/8/2019 SLafrica1

12/37

Deterministic Optimal Control- ODE

Find piecewise continuous control VW

X

Y

andassociated state variable

W

X

Y

to maximize

a

b

c

d

e

W

X

f

W

X

Y

f

V

W

X

Y Y

g

X

subject to

h

W

X

Y

i

p

W

X

f

W

X

Y

f

V

W

X

Y Y

W

q

Y

i

eand W Y

rs s

Lecture1 p.12/3

8/8/2019 SLafrica1

13/37

Quick Derivation of Necessary Condition

Suppose tu

is an optimal control and vu

corresponding state.w

x

y

variation function,

.

t

u*(t)+ ah(t)

u*(t)

t

u

x

y

w

x

y

another control. xy

state corresponding to tu

w

,

x

y

y

x

y

x

y

x

t

u

w

x

y

Lecture1 p.13/3

8/8/2019 SLafrica1

14/37

Contd.

At

,

t

x * (t)

y(t,a)

x 0

all trajectories start at same position

when

control

Maximum of

w.r.t.

occurs at

.Lecture1 p.14/3

8/8/2019 SLafrica1

15/37

Contd.

j

k

m

k

n

n

n

n

j

j

m

z

j

{

k

m m

j

}

m

z

j

}

{

k

m

~

j

m

z

j

{

k

m

j

j

m

z

j

{

k

m m

j

m

z

j

{

k

m

~

j

}

m

z

j

}

{

k

m

Adding

to our

j

k

m

gives

Lecture1 p.15/3

8/8/2019 SLafrica1

16/37

Contd.

here we used product rule and

.

Lecture1 p.16/3

C d

8/8/2019 SLafrica1

17/37

Contd.

Arguments of

terms are

.

Arguments of

terms are

.

Lecture1 p.17/3

C d

8/8/2019 SLafrica1

18/37

Contd.

Choose

s.t.

adjoint equation

transversality condition

arbitrary variation

for all

Optimality condition.

Lecture1 p.18/3

U i H ilt i

8/8/2019 SLafrica1

19/37

Using Hamiltonian

Generate these Necessary conditions fromHamiltonian

integrand (adjoint) (RHS of DE)maximize w.r.t. at

optimality eq.

adjoint eq.

transversality conditionLecture1 p.19/3

8/8/2019 SLafrica1

20/37

Converted problem of finding control to maximizeobjective functional subject to DE, IC to using

Hamiltonian pointwise.For maximization

at

as a function of

For minimization

at

as a function of

Lecture1 p.20/3

8/8/2019 SLafrica1

21/37

Two unknowns

and

introduce adjoint

(like a Lagrange multiplier)

Three unknowns

,

and

nonlinear w.r.t.

Eliminate

by setting

and solve for

in terms of

and

Two unknowns

and

with 2 ODEs (2 point BVP)+ 2 boundary conditions.

Lecture1 p.21/3

Pontr agin Ma im m Principle

8/8/2019 SLafrica1

22/37

Pontryagin Maximum Principle

If

and

are optimal for above problem, then there

exists adjoint variable

s.t.

at each time, where Hamiltonian

is defined by

!

and

"

#

$

$

%

&

transversality condition

Lecture1 p.22/3

H ilt i

8/8/2019 SLafrica1

23/37

Hamiltonian

'

(

0

1

2

3

4

3

5

6

7

8

1

2

6

0

1

2

3

4

3

5

6

5

9

maximizes'

w.r.t. 5,'

is linear w.r.t. 5

'

(

@

1

2

3

4

3

8

6

5

1

2

6

7

A

1

2

3

4

3

8

6

bounded controls,B

C

5

1

2

6

C

D

.Bang-bang control or singular control

Example:

'

(

E

5

7

8

5

7

4

F

8

4

H

I

'

I

5

(

E

7

8

P

(

Q

cannot solve for 5

'

is nonlinear w.r.t.5

, set'

R

(

Q

and solve for5

9

optimality equation.Lecture1 p.23/3

Example 1

8/8/2019 SLafrica1

24/37

Example 1

T

U

V

X

Y

a

b

c

e

f

g

c

subject to ip

b

c

e

q

i

b

c

e

a

b

c

e

r

i

b

s

e

q

u

What optimal control is expected?

Lecture1 p.24/3

Example 1 worked

8/8/2019 SLafrica1

25/37

Example 1 worked

v

w

x

integrand

RHS of DE

at

j

k

l

j

m

m

k

Lecture1 p.25/3

Example 2

8/8/2019 SLafrica1

26/37

Example 2

o

z

{

|

subject to }

~

{

~

z

What optimal control is expected?

Lecture1 p.26/3

Hamiltonian

8/8/2019 SLafrica1

27/37

Hamiltonian

The adjoint equation and transversality conditiongive

Lecture1 p.27/3

Example 2 continued

8/8/2019 SLafrica1

28/37

Example 2 continued

and the optimality condition leads to

The associated state is

Lecture1 p.28/3

Graphs Example 2

8/8/2019 SLafrica1

29/37

Graphs, Example 2

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 21

0

1

2

3

Time

State

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

2

4

6

8

Time

Adjoint

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 28

6

4

2

0

Time

Control

Example 2.2

Lecture1 p.29/3

Example 3

8/8/2019 SLafrica1

30/37

Example 3

at

Lecture1 p.30/3

Contd

8/8/2019 SLafrica1

31/37

Contd.

Solve for

and then get

.Do numerically with Matlab or by hand

Lecture1 p.31/3

Exercise

8/8/2019 SLafrica1

32/37

Exercise

control

Lecture1 p.32/3

Exercise completed

8/8/2019 SLafrica1

33/37

Exercise completed

control

!

Lecture1 p.33/3

Contd.

8/8/2019 SLafrica1

34/37

Contd.

"

#

$

%

'

(

)

1

2

'

(

)

13

4

)

5

There is not an Optimal Control" in this case.

Want finite maximum.

Here unbounded optimal state

unbounded OC

Lecture1 p.34/3

Opening Example

8/8/2019 SLafrica1

35/37

Ope g a p e

6

7

8

9

Number of cancer cells at time8

(exponential growth) State@

7

8

9

Drug concentrationControl

A

6

A

8

B

C 6

7

8

9

D

@

7

8

9

6

7

E

9

B

6

G known initial data

minimize

6

7 9

I

G

@

P

7

8

9

A

8

See need for bounds on the control. See salvage

term.Lecture1 p.35/3

Further topics to be covered

8/8/2019 SLafrica1

36/37

p

Interpretation of the adjointSalvage term

Numerical algorithmsSystems caseLinear in the control case

Discrete models

Lecture1 p.36/3

more info

8/8/2019 SLafrica1

37/37

more info

See my homepage www.math.utk.edu Q R lenhart

Optimal Control Theory in Application to Biology

short course lectures and lab notes

Book: Optimal Control applied to Biological Models

CRC Press, 2007, Lenhart and J. Workman

Lecture1 p.37/3