1 Successive Approx AUTO

8/10/2019 1 Successive Approx AUTO

1/15

Automotlcq Vol. 3 pp. 135-149. Pegamon Pra 1966. Printed ia Great Britain.

PAPER II

SUCCESSIVE APPROXIMATION METHODS FOR THESOLUTION OF OPTIMAL CONTROL PROBLEMS

S. K. MIX

1. INTRODUCTION

IN THIS paper we present some successive approximation methods for the solution of ageneral class of optimal control problems. The class of problems considered is known as

the Bolxa Problem in the Calculus of Variations [l]. The algorithms considered are exten-sions of the gradient methods due to KELLEY 2] and BRYSON 3] and similar to the methodsproposed by MIRIAM [4, s]. MERRIAM approaches the problem from the Hamilton-Jacobi viewpoint and restricts himself to the simplified Bolxa problem. The algorithmpresented is formally equivalent to Newtons Method in Function Space [6, 73 and indeedin some problems it would be better to use Newtons Method.

The development in this paper is formal and indicates how we solve these problemson a digital computer. However, under the assumptions we have made a rigorous treatmentof these successive approximation methods can be given. We shall do this elsewhere.

The paper may be divided into 8 sections. In Section 3 we formulate the problem andstate the assumptions we have made. In Section 4 we state the first-order necessaryconditions of optimality. These are the Euler-Lagrange equations and the transversalitycondition.

Section 5 is devoted to Second Variation Successive Approximation Methods andcertain modifications to it.

In Section 6 we show how the second variation method is formally equivalent toNewtons Method and also indicate how the linear two point boundary value problemarising in Newtons Method can be solved in essentially the same way as in the SecondVariation Method.

In Section 7 we point out certain advantages and disadvantages of the Second VariationMethod.

[l] G. A. BLISS: L.ectur es on fke Calculus of Vari ations. University of Chicago Press, Chicago (1946).[21 H. J. KELLEY: Method of Gradients, in: Optimi sari on Techniques, Chap. 6. ed_ by G. w.

Academic Pmss, New York (1962).[3] A. E. BRYSON nd W. F. Dm: A steepest ascent method for solvhg optimum prow

problems. J. Appt. Meek. 241-257 (1962).[4] C. W. m, III:

New York (1964).Optimisation Theory and he Design of Feedback Control Systems. McGraw Hill

[s] C. W. MEruubf, III: An algorithm for the iterative solution of a class of two point boundary valueproblems. S.Z.LiLK . Corm. A2 l-10 (1964).

[q R. H. MOORB: Newtons Method and Variations, in: Nonhear Zntegral E u ~. ed. by P. M.ANsarnNE University of Wisconsin Press (1964).

m M. L. Sm: On methods for obtaining solutions of i d end point problems in the calculxu of varia-tions. J. Res. Nat. Bur . Stand. 50, May (1953).

135


2/15

13 6 S . K . M r r r ~

I n S e c t i o n 8 w e i n d i c a te t h a t a l g o r i t h m s a n d p r o b l e m s p r e s e n t e d in t h is p a p e r m a yb e c o n s i d e r e d t o b e s p e c i a l c a s e s o f a m o r e g e n e r a l c l as s o f p r o b l e m .

S o m e n u m e r i c a l w o r k u s i n g t h e s e m e t h o d s h a s b e e n d o n e . D e t a i l e d r e su l ts w ill b ep r e s e n t e d e l s e w h e r e .

2 . N O TAT I O N

T h r o u g h o u t w e s ha l l u s e v e c t o r m a t r i x n o t a t i o n . A l l v e c t o r s a r e c o l u m n v e c t o rs .C o m p o n e n t s o f a v e c t o r w il l b e d e n o t e d b y s u b s cr ip t s. S u p e r s c ri p t T d e n o t e s t r a n s p o s e dm a t r i x . T h e s y m b o l < . , . > e n d e n o t e s i n n e r - p r o d u c t i n E u c l i d e a n n - sp a c e. U s u a l l yw e s h a l l o n l y w r i t e < . , . > . F o r a s c a l a r - v a l u e d f u n c t i o n F X l , x 2 . . . . . x ~).

f F T'

F x ~ ) =

a r e e v a l u a te d a t x - - ~ .

F o r a v e c t o r v a lu e d f u n c t i o nf ( x l . . . . x n) ,w h e r e j r i s a n m - v e c t o r.

f o : . . . . . 0 : _~ ~X O Xn

L ~ ) -- l e f . 0 f .I, ax l ax .

a n d t h e p a r t i a l d e r i v a t i v e s a r e a g a i n e v a l u a t e d a t x = ~ .

S i m i l a r ly f o r t h e s c a l a r - v a l u e d f u n c t i o n F

6~2F t~ZF

Ox~ ax~ax

F x x ~ ) =

a 2 F

~xnt~x l

w h e r e t h e p a r t i a l d e r i v a t i v e s

a n m x n m a t r i x

Ox x Ox,,

ozax~

D o t i n d i c a t e s d i f f e r e n t i a t i o n

3. F O R M U L AT I O N O F T H E P R O B L E M

W e c o n s i d e r t h e f o ll o w i n g B o l z a p r o b l e m . F i n d t h e o p t i m a l c o n t r o l fu n c t i o n u a n dt h e c o r r e s p o n d i n g o p t i m a l t r a j e c t o r y x s o th a t t h e p e r f o r m a n c e f u n c t i o n a l

f[ x ( t o ) , u ] = F [ x ( t ) , t ] + L x t ) , u( t ) , t )d t 1)

to


3/15

Successive approx imation metho ds for the solution of optimal control problem s 137

is min imised , sub jec t to the cons t ra in t s

d xd t = f i x ( t ) , u ( t ) , t'] ; X( to ) = c 2)

G[ x( t ) , t f ] = 0 3)H e r e x ( t ) e E n , u ( t ) e E m ,f i s a f u n c t i o n m a p p i n g E n+m + 1 t o e n a n d G i s a f u n c t io n m a p p i n gE n+t to E r , p < _ n . T h et ime t f may be exp l ic i t ly o r impl ic i t ly spec i f i ed .

A s s u m p t i o n s .i . A l l f u n c t i o n s a r e a s s u m e d t o h a v e c o n t i n u o u s s e c o n d d e r i va t iv e s .i i. T h e p - t e r m i n a l c o n s t r a i n t s a r e a s s u m e d t o b e i n d e p e n d e n t .

i i i . T h e s y s t e m i s a s s u m e d t o b e l o c a l l y c o m p l e t e l y c o n t r o l l a b l e u n i -f o r m l y i n t o, t y] a l o n g a n y t r a j e c t o r y c o r r e s p o n d i n g t o a nadmiss ib le* co n t ro l f i i .e . fo r the l inea r ized sys tem

w e h a v e

fY c = f ~ ( t ) f x + f ~ ( t ) f u ; f iX ( to ) = 0

t t o ~ t x)f~( , )fr~(x)~PT(t ,z ) d * > 0 4 )

f o r a l l t~ to , . r ] n d w h e r e t , ) i s t h e s o l u t i o n o f

d ~ t-~ -~ (, t o ) f f , ( t ) @ ( t ,to); eP( to , t o )= l 5)

T h e l o c a l c o n t r o ll a b i l it y a s s u m p t i o n e n s u r e s t h a t t h e s o l u t i o n o f th e a c c e s s o r y m i n i m i z a t i o np r o b l e m i s n o r m a l i n th e s e n se o f C l a ss i ca l C a l c u lu s o f Va r i a ti o n s ) .

4 . F I R S T O R D E R N E C E S S A RY C O N D I T I O N S

F o r t h e p r o b l e m f o r m u l a t e d in S e c t i o n 3 t h e E u l e r - L a g r a n g e e q u a t i o n s a n d Tr a n s -v e r s a l it y c o n d i t io n s m a y b e d e r i v e d i n th e u s u a l w a y.

L e t u a n d x b e t h e o p t i m a l c o n t r o l a n d o p t i m a l tr a j e c t o r y a n d l e t ~ t ) b e a n n - v e c t o ro f L a g r a n g e m u l t ip l i e r f u n c t i o n s a n d / ~ o b e a p - v e c t o r o f c o n s t a n t s w h i c h a r e t h e m u l t i p li e r sc o r r e s p o n d i n g t o t h e t e r m i n a l c o n s t r a i n t s .

D e f i n e

H = H [x ( t ) , u ( t ) , ,~ (t) , t ]= L[ x ( t ) , u ( t ) ,t ]+ < :~q t ) , f [x t ) , u / ) , t ]> 6 )

~o = [xO ( -r),ts) =F [xO( -r), t :] +< g o , a[xO ( -r), t ~]> 7)

T h e n u a n d x s a t i sf y

E u l e r - L a g r a n g e E q u a t i o n s

Yc( t ) f fi f [x (t ) , u ( t ) , t ] f H ~ ( t ) ; X( to ) f f i c

A( t ) = - H x ( t ) ; ~ t f ) f fi ~ : c t f )

H . O = O

~[x( t : ) ,t : ] = 0

It is atmumed h at u belongst o a bounded OlaCnset ~ c E .

s )

9)

lO)

11)


4/15

138

an d Transversality Condition

S. K. M rrrta

o t : ) = n t : ) + , t : ) - - o 02

5 . S E C O N D VA R I AT I O N S U C C E S S I V E A P P R O X I M AT I O N M E T H O D

W e sha l l cons ider th ree d i fferen t cases o f the pro b lem form ula ted in Sec t ion 3 .Case (i) .W e assum e tha t t e rmina l cons t ra in t s a re abse n t and the te rmina l t ime t s i s

f ixed. This is the s implif ied Bolza prob lem of the Calculus o f Variat ions.Le t u s a s su me t h a t we have c hosen a nomina l c on t ro l f unc t i on u and ob t a i ned t he

cor responding nomina l t ra jec tory x by in tegra t ing the sys tem dynamic equa t ions in thefo r wa rd d i rec ti on . W e can now i n teg r a te t he Eu l e r-Lag range equa ti o n 4 = - / / ~ ; w i tht he bo und a ry cond i t i on 2 ( t / ) = / r x ( t : ) whe re t he ba r i nd ica te s t ha t H and F a r e eva l ua t eda t t h e n omina l con t ro l and nom ina l t ra j e cto r y. The pe r fo rmance f unc t iona l m ay nowbe re-wri t ten as

P[x( to ) , u ] = F( t f )+ < 1( t) , ~ t ) > ]d t (13)

Expand ing the per form ance func t iona l P in a Tay lor s Ser ies and re ta in ing te rms u p tothe second order we ob ta in the fo l lowing express ions for the f i r s t and second var ia t ionso f P

[ < / / . t ) ,6u t )> ]d t 14)

f r : [ < Fi . . ( t ) tu ( t ) ,62P = au( t ) > + < Flx~(t )6x( t) , 6x( t ) >+ 2 < F l ,= ( t) 6 x (t ), 6 u ( O > ] d t + < P x ~ ( t : )6 x ( t ; ), 6 x ( t f ) >(15)

In ob ta in ing the above express ions we have per formed the usua l in tegra t ion by pans .At th i s po in t we have to in t roduce the fo l lowinga s s u m p t i o n : T h ematr ix o f par t ia l

der ivat ives H . . is posi t ive def ini te ; F,= an dH x , - H ~ r t ~ I H . =are positive semi-definite.This implies that there a re no poin ts con juga te to t ffi s in the interval [ to ,tf).

The improvement in con t ro l ~u i s ob ta ined by min imis ing

6 P + 62P = < arx~,( t: )tx( t, ), 6x ( t , ) > + f i [ < H .( t ) ,6 u ( t ) > ] d t

+ + dt 1 0

w he r e 6u a n d 6 x are re la ted by the l inear ized system,

6~( t) =L ( t )6x (O +L (O 6~ t ) ; ,~X(to) - -o (17)

This i s a new var ia t iona l p rob lem. In v iew of the assump t ion we have jus t m ade th i sprob lem has a weak re la tive min imum . The E ule r-Lagrange E qua t ions of th i s aux i li a rymin imiza t ion prob lem are

6Yfft ) - - f~( t ) tx( t ) + f , ( t ) tu ( t ) ; 6X(to)--0 (18)


5/15

Successive approximation methods f or the solution of optimal control problems

A)t( t ) = - I : lzx (t~x (t) - Fl~(t) tSu(t)- ]T~(t)A).(t); A ~(t f ) ff i Fxx( ty)

~ , ( t ) = - ( r t . . ( t ) ) - [ n . ( o + l q . . (o ~ x ( t ) + f Y ( o ~ o 3

i s t h e m u l t i p l i e r f o r th e a u x i l i a ry m i n i m i z a t i o n p r o b l e m .S u b s t i t u t i n g ( 2 0 ) i n t o ( 1 9 ) a n d ( 1 8 ) , w e o b t a i n

6:~(t) ---A(t)tSx(t) + B(t)A 3.(t) +o(t ) ;

A,~(t) = -C ( t ) 6 x ( t ) - A r ( t ) A R ( t ) -w(t ) ;

w h e r e

B e f o r e p r o c e e d i n g f u r t h e r i t

139

09

20 )

6 x ( t .) = 0 (21)

~ t / ) = F = 2 2)

,4ffiL-;dt;.qq.~ 7n = - f ~ ; . F .cfH =-~ q~ t ; .~ rt=

v = - f ~ . - _ . l l q ,w = - / / , , : 7 ~ , 1 / / ,

(23)

i s n e c e s s a ry t o s h o w t h a t t h i s c h o i c e o f 6 u r e d u c e s t h e v a l u eo f t h e p e r f o r m a n c e f u n c t i o n a l ( a s s u m i n g t h a t t h e l in e a r iz a t io n o f t h e s y s t e m d y n a m i c sa n d t h e T a y l o r s s e ri e s e x p a n s i o n a r e v a l i d) i .e . w e h a v e t o s h o w6 P + 6 2 Pi s nega t ive .U s i n g ( 1 8 ) , ( 1 9 ) a n d ( 2 0 ) a n d s u b s t i t u t i n g i n ( 1 6 ) , i t m a y b e s h o w n t h a t t h e v a l u e o f6 P + 5 2 Pc o r r e s p o n d i n g t o t h e c h o i c e o f h u i s g i v e h b y

6p + 62p ffi _ I { < FI , ( t) + f~( t )A ~( t ) ,~ / ~ l ( t ) [ / / , ( t ) + f r ( t ) A 2 ( t ) ] > } d tto

f < C ( t ) ~ x ( t ) , 6 x ( t ) > a t (24)tw h i c h i s n e g a t i v e .

I n g e n e r al , t h e li n e a ri z a ti o n a n d s e c o n d o r d e r e x p a n s i o n o f t h e p e r f o r m a n c e f u n c t i o n a lwi l l no t be va l id an d i t i s nece: ~sa ry to in t ro du ce a pa ram ete r 8 , 0 < 8 ~< 1 in t he fo l low ingw a y t o r e d u c e t h e s t e p s i z e .

~ u = - 8 / / , , ~ ) -l F l , t )+ L r O a ~ t) ) -F l 1 F l ,~ O ~ x O 2 5 )

Wi t h t h i s c h o i c e o f6u,

6 P + 6 P - - - ~ 2 i [ < F l .( t) + r. (t )A ; ,( t ,Fl~ , ' ( tXF l . ( t )+ f [ ( t )AR( t ) ) > ]d t

< C ( t )6x( t ) , ~x( t ) ~ , d r (26)t

w h i c h is n e g a t i v e f o r 0 < 8 ~ 1 .

T h e l i n e a r t w o - p o i n t b o u n d a r y v a l u e p r o b l e m ( 2 1 ) - - ( 2 2 ) m a y b e s o l v e d i n v a r i o u s w a y s .

T h e m o s t a d v a n t a g e o u s w a y a p p e a r s t o b e t o i n tr o d u c e t h e li n ea r t r a n s fo r m a t i o n

A/l(t) ffi (t) +K( t )6x ( t ) (27)


6/15

140 S. K. MrrngR

where l ( t ) i s an n-vec tor andK t ) is an n x n sym me tr ic matr ix . Different ia t ing (27) andequa t ing wi th the r igh t hand s ide of (22) we ge t

l ) + [K t)B t) + A r )] l t ) + K t)v t ) + w )

+ [/~(t) +K t )B t )K t ) + K t )A t ) + A r t )K t ) + C t ) ]6x t )= 0 (28)

Since (28) is t rue for arbi t rary 6x we have

l t ) + [K t )B t ) + A r t ) ] l t ) + K t )v t )+ w(t) = 0; l t l )=0 (29)

K t) + K t) B t ) K t ) K t) A t ) + A r t) K t ) + C t ) = 0 ; K t f ) = F x x (30)

Equ at ion (30) is a ma t r ix Ricca t i Equ a t ion and i ts p roper t ies have been ex tensive ly s tud iedin the l it e ra ture . In par t i cu la r the equa t ion i s s tab le when in tegra ted in the backw ardsdirect ion. The solut ion o f equ at ion (30) is def ined everywh ere in [to,t f] i n v i ew o f o u rconjuga te po in t assum pt ion [8].

The com pu t ing a lgo ri thm fo r so lv ing the L ag range p rob l em m ay n ow b e su mmar i s ed

as fo l lows:i . Gue ss the cont ro l func t ion u and in tegra te the sys tem equa t ion ~ ( t )~ f Ix( t ) , u ( t) , t]

fo rwards wi th X( to)=e .S tore u and the cor responding t ra jec tory x .

ii. I n t eg r a te t he Eu l e r- Lag range equa t i on ~= - H x backwards w i th2 tT ) ff iF~.Ca lcu la t e H . , H , , H~ , and H= , and a l so eva lua te (H ~ - 1 a long t he tr a je c to ry. S i mu l tane -ous ly in tegra te the d i ffe ren t ia l equa t ions for l ( t ) andK t ) wi th t he p rope r boun da rycondi t ions. Storer t ) = H ~ l t ) [ H ~ t ) + f f ~ t ) l t )]and the feedback ga in mat r ix

M t )- -- H ~ t t ) [ H u x t ) + j r t ) K t ) Jwh ere 0


7/15

We ob ta in

Su__ece~___ive pproxim ation methods for the so lutio n of op tim al control problem s 141

~ p = < p ~ t r ) , ~ x t r )> + 6 u ( t ) > + < / : ~ ( t ) ,~ x t ) > ) d r

6 Z p = < F x ~ ( t f ) 6 x ( t f ) , 6 x ( t f ) >+ f ( < E ~ ( t ) 6 u ( t ) , ~ u ( t ) > + ] [ t o

+ 2 < / : , ~ 0 ~ x t ) , ~ u 0 > ) d r

The cont ro l improvement ~u i s ob ta ined by minimis ing~ p + ~ 2 psubjec t to the con s t ra in t

~ =j r~(0~x( t )+ f . t ) ~ u t ) ; ~ X t o ) = o

In v iew of our assumpt ions o n F and L the s t rengthened L egendre condi t ion a nd theconjugate poin t condi t ion a re au tomat ica l ly sa t i s f ied for th i s auxi l ia ry minimiza t ionproblem.

The Euler-Langrange eq uat ions of th i s problem are

~Y(t)= ] , t ( t )6x ( t ) + f , ( t )6u ( t ) ; ~X(to) 0 (31)

A ~ (t )_ ~ _ F ~ x ( t )_ x x ( t ) 6 x ( t ) _ ~ ( t ) 6 u ( t ) - f r x ( t ) A A ( t ) ; A ~ ( t f ) ~ . I ~ : (t : )+ F x x ( t ~ x ( t f )(32)

~ u 0 = - r - ~ t X l : , t ) + l : ,~ 0 6 x 0 + Y , r 0 ~ 0 3 3 )

Subst i tut ing (33) into (31) and (32) we get

~( t ) = A ( t )~x ( t ) + B( t )A~ . (t ) + v ( t ) (34)

Ave(t) = - C (t )~x ( t ) - .4r(t)A ~.(t)- w(t) (35)

where

A t )= f x t ) - f , 0 E 1 t ) E , ~ t )

8 ( 0 = - l , ( 0 r - 1 (0 ] , ~( t)

c o = E , , . , t) - L ~ , t ) rC , I O F ~ O

v t ) = - f , t ) r = . ~ t ) E , t )

w 0 - - Z x t ) -E ~ , t )r . 1 t ) r ., t)

This l inear two po in t b oun dary va lue problem is so lved in exac t ly the same Way as for thesecond var ia t ion case by assumingA A ( t ) = l ( t ) + K ( t ) 6 x ( t ) .The different ial equat ions forl( t ) and K ( t )are

] ( t ) + [ K ( t ) B ( t ) + A r ( t ) ] l (t ) + K ( t ) v ( t ) + w ( t ) -- O ; l ( t f ) f fi F z (Q ) (36)

l ~ ( t) + K ( t ) B ( t ) K ( t ) + K ( t ) A ( t ) + A r ( t) K ( t ) + C ~ -0 ; K ( t )ffiF~ (t ) (37)

N o te howeve r t ha t t he boun da ry cond i t ion o f the I equa t ion i s d if fe r en t f rom tha t i n t he

second var ia t ion case and there i s no ~ equat ion to in tegra te .


8/15

142 S . K . M l r l ~

I t is eas i ly sho wn tha t th is cho ice o f t g ives a va lue o f

aP 6zPf [S E ~ , t ( t )[ E , ( t ) + f , ( t )A 2 ( t ) ] >+ < ( E , , ~ ( t ) - E ~ ( t ) E ~ l ( t ) E , , , ( t )) f x ( t ) , 6 x ( t ) > ] d t

w h i c h is n e g a t iv e in v i e w o f t h e a s s u m p t i o n o n L . To r e d u c e s t e p s iz e t h e p a r a m e t e re , 0

A n o m i n a l c o n t r o l ~ is c h o s e n a n d t h e c o r r e s p o n d i n g t r a j e c t o r y ~ is o b t a i n e d b y i n t e g r a t i n gt h e s y s t e m e q u a t i o n s f o r w a r d . T h e m u l t ip l i e r v i s e s t i m a t e d a n d t h e E u l e r - L a g r a n g ee q u a t i o n ~ = - / 7 ~ , i s i n t e g r a t e d b a c k w a r d s w i t h b o u n d a r y c o n d i t i o n 2 ( t f ) = i ~ x ( t f ) 6 ui s n o w c h o s e n s o t h a t

eSP+ 6ZP = < ~:x ( t : )b x( t f ) , f ix ( t : ) > + [ ' s < i r l, (t ), 6u( t ) > d tJ to

t

+ [ (< N ~( t )~u ( t ) , 6u ( t ) > + < Rx~( t )~x ( t) , ~x (O > + 2 < 17 ,~ (06 x(0 ,6 u( O > ) d td to

i s min imized , sub jec t to the cons t ra in t s ,

35)

c~Yc(t)= f R t ) 6 x ( t )+ f , ( t ) 6 u ( t ) ; 6X(to) = 0 (39)

G( t : ) + Gx( t )&x(Q ) = 0 (40 )

The so lu t ion to th i s p r ob lem i s s imi la r to th a t o f Case ( i) the on ly d i ffe rence be ing in theb o u n d a r y c o n d i t io n o f A 2. T h e r e l e v a n t e q u a t io n s a r e

tS : c ( t )=A( t ) cSx( t )+B( t )A2( t )+o ( t ) ; Jx( t0 ) = 0 (41)

AJ.( t) = - C ( t ) 6 x( t ) - Ar( t )A A(t ) - w( t ) ;A2( t f ) = ~xx( t f )~x ( t r ) + G~( t f )Av(42)

w h e r e Av is a p - v e c t o r o f c o n s t a n t s , b e i n g t h e m u l t i p l ie r f o r t h e t e r m i n a l c o n s t r a i n t s , w h i l e

the de f in i t ions o f A , B e tc . a re the sam e as in eq ua t io n (23).T h e l i n e a r t w o - p o i n t b o u n d a r y v a l u e p r o b l e m w e h a v e t o s o l v e i s g i v e n b y e q u a t i o n s

(40), (41) an d (42) . Th e so lu t ion i s aga in an a log ou s to tha t o f Case ( i) .I n t r o d u c e t h e l i n e ar t r a n s f o r m a t i o n s

A 2 t ) = l (t ) K ( t ) x ( t ) N ( t ) A v

&G = r e (t ) + N r ( t ) J x ( t) + P ( t )A v

whe re I i s an n -vec to r, m a p -vec to r, K a n x n ma t r ix , N a n p m at r ix , P a p p ma t r ix .H e r e 6 G =G,, ( t )6x( t l )i s t h e a m o u n t b y w h i c h t h e t e r m i n a l c o n d i t i o n s h a v e b e e n m i s s e d.I f th i s q u a n t i t y i s la rg e w e m a y i n t r o d u c e a p a r a m e t e r 8 2 w h e r e 0


9/15

Su__~s___q~ve pproxlnm tion methods fo r the so lutio n of op tim al control problem s 143

In ex ac t ly the same w ay as fo r C ase ( i) we deduce , b y equat ing the coeff ic ien ts of ~xand Av to zero the fo l lowing d i ffe ren tia l equat ions

l ( t) + [K (t )B( t) + A r( t ) ] l( t ) + K (t )o( t) + w(t ) =O; l ( t : ) = O (45)

g( t ) + K( t )B( t )K( t ) + K( t )A( t ) + Ar ( t )K ( t )+ C(t)= 0; K ( t s ) = ~ ( t f ) (4 6)l~ l ( t )+(Ar( t )+ K( t )B( t )N( t )=O ; N( ty )=~f~( t ) (47)

rh(t ) + Nr( t ) [B ( t ) l ( t )+o( t ) ]=O ; m (t f )= O (48)

P ( t ) + N r ( t ) B ( t ) N ( t )= O ; P ( t ) = O (49)

To proceed w i th the im provem ent process it is necessary to de te rmine Av. Hav ing in te -gra ted eq uat ions (45 ) -(49) backw ards , we m ay de termine Av f rom

Av = P - 1 to)[/5G_m (to) - Nr(to)~X(to)] (50)

The co njugate poin t assum pt ion ensures tha t P - 1(to) exis ts . This procedu re for de te rminingAv is som ew hat s imilar to that of BRFAKWI~L, SPlrcmt and BRYsON [9] . They , ho we ver,so lve the l inear two-po in t boun dary va lue problem in a d i ffe ren t way. I t is thoug ht th a tou r me tho d has advan tages f rom the po in t o f v i ew o f numer i ca l s tab il it y.

Subst i tut ing (43) and (50) into the expression for 6u, we obtain

6u -- - H~ l( t)[ '81/~,( t ) +8 1 f r (t )l (t ) + f T ( t ) N ( t ) P -' ( t o ) [ t G -am(to) - Nr ( to) tX(to )]

- ll~.l(t)[I:l .=(t) + 8f r.(t) K (t)] 6x (t) (51)

W e have in t roduced 8~, 82, 0 d t ( 5 2 )

d t

[9] J. V. B Nw x~ , J. L. SpL~_.~ and A. E. I~ N : Opt ;m; ~io n and control of n o n systems

~ n g the second variation. S . I . A . ~ . . 1 . C o n t r . A I, 1963).


10/15

144

w h e r e

S. K. Mrrrrat

+ ~ , d ~ t : + 2 < ~ , , ~ ( t : ) d ~ : > ) + < / L ( t : ) , u ~ : ) d t : >

+ d [ ~ / ( 0 - < ; ~( 0, ( t ) ] , , ,~ t~

+ [ / / ( t ; ) - < ~ ( t f ) , t s . ) > ] d a t :

+f i[ + < R x x t ) 6 X t ) ,x ( t ) >+ 2 < ~ . , (t ) ~ x( t ) , u t ) > ] d t 5 3 )

a ~ x ( t : ) = ~ t : ) d ~ t f ~ ( t : ) d t ~ , + ~ ) d t : ( 5 4 )

T h e a u xi l ia ry i n i m i z a t i o n p r o b l e m t o b e s o l v e d is M i n i m i s e~ P + ~ . 6 a P,s u b j e c t o t h ec o n s t r a i n t s

6 :c t) = f= t )6x t ) + f . t )6u t ) ; SX to )= 0 (55)

G t ~ .) + G . t : )~x t f ) + Ox t ) : c f )d t f + Ot t f )d t I = 0 (56)

I f w e u s e ~ t l ) = ~ t l ) a n d n e g l ~ t t e r m s i nd 2 t f a n d s o l v e t h i s v a r i a t i o n a l p r o b l e m , w eo b t a i n t h e f o l l o w i n g E u l e r- L a g r a n g e E q u a t i o n s

w h e r e

6 : ~ t ) = A t ) 6 x t ) + B t ) A R t ) + v t ) ; a X t o )= O (57)

A ,~ O = - c O a x t ) -A (t)ea(O-w ) (58)~ t f ) = ~. .~ t .r )6 x t :) ~ t / ) A v + ~ , = t I ) ~ t f ) ~ , t f ) J ~ t f ) ~ . t I ) ) d t : ( 5 9 )

6 G = ~ x t ) 6 x t ) ~ t ) :c t ) O Z t ) ) d t 6 0 )

6 ~ 2 < ~ t : ) ~ t f ) + ~ ( t : ) + f ~ t / ) ~ t : ) , x t : )

+ < ~ x t : ) ~ t : ) + G , t / ) , v > + s t s ) d t : ( 6 1 )

s t f ) = [ < ~ x w : c ,: ~> + < ~ x , f x + f u f i + f t > + < L x , > - < L . , f i >L f ~ . 2 < ~ 2 . ~ > ] t = , /

T h e l i n ea r tw o - p o i n t b o u n d a r y v a l u e p r o b l e m 5 / - 6 0 is s ol v e d in e x a c t l y t h e s a m e w a y a sp r e v i o u sl y b y i n t r o d u c i n g

~ O = l t ) K t ) 6 x O N t ) A v + p t )d t :

6 G= m t )+ N T t ) 6 x t ) P ( t ) Av q t ) d t :

6 ~- -- n( t ) < p ( t ) , x ( t ) > + < q ( t ) , v > + s t ) d t :

where

6 G = - ~ 2 t f )

, s ~ - ~ 3 ( F l ( t ) ~ ; , ( t ) )

0


11/15

Su__r~ss_~iveapproximation methods for the solution of optimal control pmbkam 145

The d i ffe ren t ia l equa t ions for l , K , N, m and P a re the same as tha t g iven by equa t ions(45)-(49).

The equa t ion for p , q , n, and s a re ob ta ined in the sam e way as in the prev ious twocases. They a re :

~ t) 4. [AT t)-6K t ) B t ) ] p t )----0

~ t ) + N ~ t ) B t ) p O = o

~ t )+

= 0

~ t )+

=0

6 2 )

(63)

(64)

(65)

Av and d t : are de te rmined by in tegra t ing equa t ions (45)- (49) and eq ua t ions (62) - (68)backwards and so lv ing

3 G = r e t o ) + N T t o ~ X t o ) + P t o )AV + q t o ) d t s (69)

6 ~ = n t o ) + + +S(to)dt~r (70)

6. R E L AT I O N S H I P S W I T H N E W T O N S M E T H O D

For s impl ic i ty we cons ider the case when there a re no te rmina l cons t ra in t s p resen t .The m ethod and conc lus ions a re va l id fo r the genera l Bolza prob lem. Solv ing the var ia -t iona l p rob lem by N ew ton s M ethod mean s so lv ing the Eule r-Lagrange Equa t ions (8) ,(9) and (10) . The m ethod cons i st s in guessing a nom ina l con t ro l func t ion , a nom ina lt ra jectory and a nominal mult ipl ier funct ion and then l inear lz ing equat ions (8) , (9) and(10) roun d the guessed func t ions . A l inear two-p oin t bou nda ry va lue prob lem i s thenso lved which y ie lds cor rec t ions to the guessed func t ions . The l inear two-po in t bou nda ryva lue prob lem to be so lved is

Y c 6 : c ~ f t ) f ~ t ) c Sx t ) f ~ t ) 6 u t ) ; 6 X t o ) ~ 0

f~ 4- 6~. = - F ix t ) - H,~ , t )~Sx t ) - F l~ , t )6 u t ) - F lxa t )~5~ t ); 6~ t/ ) .=O

R , t ) + F / , t )~Su t ) + F /,~ , t)~x t ) + F I , ~ t~ t )= 0

But fo r the fac t tha t the sys tem equa t ions and the Eule r-Lagrange equa t ions a re no tsat isf ied by the ini t ia l ly guessed funct ions, the se equat io ns are precisely the sa m e as equat io ns(18) , (19) and (20) . Th us th e m etho ds we hav e used in so lving equ at ions (18) , (19) and (20)may be u se d i n so lv ing th e l i nea r two-po in t boun da ry va lue p rob l em in N ewto n s M e thod .As we have ind ica ted prev ious ly f rom the v iewpo in t o f numer ica l s tab i li ty i t is advantag eoust o so l ve t he two-po in t boun da r y v a lue p rob l em in t he way we have i nd i ca ted . I n p rob l em sw he r e t he r e i s a cons t ra in t o f t he fo r m x ( t l ) = a i t ma y be b e t te r t o u se New to n s M e tho ds ince we can guess the nom ina l t ra jec tory to sa t i s fy the bo und ary condi t ion .

7 . A D I S C U S S I O N O F VA R I O U S M E T H O D S O FS O LV I N G O P T I M A L C O N T R O L P R O B L E M S

A num ber o f me thods have been p roposed fo r t he so lu t ion o f tw o-po in t bou nda ryva lue prob lems a r i s ing in op t imal con t ro l p rob lems . These ma y be subdiv ided in to th reemain classes:

i. Bo un d a ry Con d i t ion I te r a t ion M e thodii . Co nt ro l func t ion I te ra t ion M ethod

ii i. N ew ton type I te ra t ion Meth ods


12/15

146 S.K . M_rrr~

The cho i c e o f t he m e th od t o be a dop t ed depe nds on t he p rob l em an d on t he n a t u r e o f t h eappl icat ion. Eac h probl em wil l have a cer ta in s t ructure and exh ibi t cer ta in s tabi li typropert ies , a l though in a non-l inear problem i t might be very diff icul t to isolate e i ther.Fur th er the na ture o f the cont ro l app l ica t ion m ay impose var ious cons t ra in t s . Fo r example ,i f on- l ine cont ro l i s env isaged , rap id ity o f convergence m ay over- r ide o ther fac tors . Fo rsome problems i t may be necessary to ob ta in ex t remely accura te t ra jec tor ies , whi le ino thers convergence of the per form ance func t iona l to w i th in a p re -ass igned to le rancem ay be suff ic ient. In spi te of this , cer ta in ad van tages an d disad vantag es o f each o f theseme t hods m ay be po in ted ou t and ce r t ai n r e comm enda t i ons mad e .

i . Bou nda ry cond i t ion i t e ra t ionIn th i s m ethod , typ ica lly the con t ro l func t ion u i s e l imina ted f rom the f ir s t two Eule r-

Lagrange equa t ions b y so lv ing H~ = 0 and the resu l ting f ir s t two Eule r-Lagrange equa t ionsare so lved by i te ra t ion on o ne o f the unk now n b oun dary va lues say, 2 (to ). A su i tab le

sca lar t e rmina l e r ror fun c t ionV { x [ t s 2(to)I, 2[ ts , 2( to)]} is then c onstruc ted. The bo un da ryvalue 2(to) is then adjus ted t il l the error funct io n goes to zero. The a djustm ent requiresthe com puta t ion of the grad ien t o f V. Sys temat ic m ethod s for do ing th i s a re ava i lab le [10] .These me thods have cer ta in com puter p rogramm ing advantages . Co m pute r log ic i s s impleand fas t s to rage requi rements a re smal l. In p rob lems where the m etho d i s successfu laccura te t ra jec tor ies a re ob ta ined . The m ain d i sadvantage i s the inheren t ins tab i li ty o fone of the Eule r-Lagrange equa t ions . To d e te rmine whether the metho d i s app l icab le apre l iminary ana lys i s o f the prob lem m ay po ss ib ly be car r ied ou t in the fo l lowing way:le t the unforced sys tem equa t ion b e l inear ized roun d the g iven in it ia l condi t ion . An e igen-va lue ana lys i s o f the linearized sys tem mat r ix co u ld now be made . I f the mat r ix tu rns ou tto be essent ia l ly se lf -ad jo in t bou nda ry i t e ra t ion m ethod s a re qu i te su itab le . I f no t an d i ft s - to is substant ia l ly greater than th e dom inan t sy stem t ime-constan t , severe instabi li t iesm a y b e encoun t er ed .

i i. Con t ro l fun c t ion i t e ra tionCon t ro l func t ion i t e rat ion me thods us ing bo th grad ien t t echniques and s teepes t descen t

technique have been propo sed in the l it e ra ture . In these m ethods the co nt ro l func t ion issuccess ive ly impro ved t il l I I~ , l l -.0 , where i s som e su i tab le norm o f the H~ func t ion .Th e p r im a ry adv an t age o f t hi s me thod i s t ha t com pu ta t i on s a r e a lways pe r fo rme d i n t hes tab le d i rec tion . Ho w ever convergence tends to be in to le rab ly s low in a ce rta in ne ighbou r-ho od o f the op t imum . To im prove convergence the s ize -step cann ot be increased s inceth i s l eads to ins tab i li ty. The i t e ra t ion m ethods w e have presen ted in th is pape r ma y becons idered to be d i rec t ex tens ions of g rad ien t o r s teepes t descen t t echniques . W e haves ta ted prev ious ly tha t the second var ia t ion m ethod i s fo rmal ly equiva len t to N ew ton ' sm ethod in func t ion space. In a su itab le ne ighb ourho od o f the op t imu m convergence isthere fore quadra t ic . Co m puta t ion s here a re a lso a lways per formed in the s tab le d i rec tion .In fac t in a su i tab le ne ighbou rhood of the op t im um, the inheren t s tab il i ty p roper t ies o fl inear feedba ck cont ro l sys tems inh ib i ts the propa ga t ion of numer ica l e r rors . As a by-p r oduc t we ob t a in l in ea r t im e -va rying f eedb ack gain s f o r ne ighbou r ing op t imu m feed backcontrol .

[10] M. I.ZVlNE: A steepest descent technique for synthesizing optima l control progranumm. Pa pe r 4,Conf. Advances In A utomatic Control.No ttingham, A pril (1965).


13/15

Successive approximation methods for the solution of optimal control problems 147

On the other hand the conditions that H be positive definite and that there be noconjugate point for the trajectories occurring in successive auxiliary minimization problemsmay be too strong. In such cases it may be necessary to get better estimates of the controlfunction by using gradient methods or use the alternative successive approximation methodwe have indicated in conjunction with the second variation method. Numerical di ultiesmay also be encountered in integrating the matrix Ricatti equations, specially if the dynamicsystem is unstable. It is also to be noted that the matrix H is to be inverted. Computerstorage requirements are also greater since the feedback gain matrices have to be stored.

Some computational effort may be saved. For example, it is not necessary to computeH ; at every iteration. In fact in practice this may be held constant after two or threeiterations. Convergence will necessarily be slower.

For ordinary minimization problem some very efficient computational algorithmshave recently been proposed [l 11 These algorithms may be considered to lie somewherebetween gradient and Newtons method. A distinctive feature of these methods is thatuse is made of information generated in previous iterations. Generalisations of these

methods to function spaces should be possible.In this paper we have not considered inequality constraints. The assumption was

made that these could be approximated by means of penalty functions. Extensions of thetechniques presented here to problems with inequality constraints on control and statevariables appear to be possible. The auxiliary minimization problem then has additionallinear inequality constraints. In this case the corresponding dual maximization problemcould be solved to obtain the improvement in control function.

iii. Newtons method

Newtons method was first proposed by HESTENES 12] to solve fixed end point problemsof the Calculus of Variations. A complete analysis of the method for this class of problemswas given by STEIN [13]. In the context of function space, the method dates back toKANTOROVICH [l4]. KALABA [15] has also used this method for a special class of problemsand called it quasi-linear ion. Recently the method has been applied to some optimalcontrol problems by OPP and MCGILL [18]. They eliminated the control function u fromthe first two Euler-Lagrange equations by using the equation H =O . The linear d Euler-Lagrange equations are then integrated for n-linearly independent boundary conditions.The unknown boundary value cU(t,) is found by using linear interpolation and a matrixinversion. Improvements dx t) and c (t) are then obtained by one more integration.

If the linear two-point boundary value problem is solved in this way, the methodsuffers from the instability disadvantages of boundary integration methods.

In our view, the methods advocated in this paper could be used to solve the lineartwo-point boundary value problem arising within Newtons Method.

[ll] R.Fm and M. J. D. POWELL: rapidly convergent descent method for minimimtion.J 6 (1963).

C o m p u t e r

[12] M. R. HEYIZNB: Numerical methods of obtaining solutions of fixed end point problems in the calculusof variations. RM-102, The Rand Corp, August (1949).

1131 M. L. SIEIN: ref. cit.1141 L. V. RANKBRO~I~: On Newtons method. Tr udy M at. I nst. Stekfov 28, 104444 (1949).1151 R. KALAEN On nonlinear differential equations, the

J. Muth. Me . g, W-574 (1959).maximum operation and monotone convergence

1161 S. K. Mrnaa: Pro gramming in function space. To be published.1171 L. CCUAIZ: F unkti onalanalysis nd Numerische M athematok, Springer, Berlin (1964).1181 R. E. KOPP and R. MCGILL: Several trajectory optimization techniques. in: Computing Metko in

Optimi zation Problems. d. by A. V. B L KRINSRN N and L W. Nausrwr. Academic, New York (1964).


14/15

148 S .K . M rrn~

8. S O M E G E N E R A L I S AT I O N S

I n t h e i n t r o d u c t i o n w e h a v e s t a t e d t h a t t h e d e ri v a t io n s i n t h is p a p e r a r e f o rm a l . I nt h is s e c t io n w e b r ie f ly i n d i c a te h o w a m a t h e m a t i c a l ly r i g o r o u s t r e a t m e n t o f t h e s u c c es s iv ea p p r o x i m a t i o n m e t h o d s c a n b e g iv e n . To t h is e n d , it is c o n v e n i e n t t o c o n s i d e r t h e t 'o l lo w i n gg e n e r a l p r o b l e m :

W e a r e g iv e n a d y n a m i c a l s y s te m w h o s e b e h a v i o u r i s g o v e r n e d b y t h e f o ll o w i n go p e r a t o r e q u a t i o n

g ( u, x ) = . . . ( 71 ), w h e r e g : . ~ x U ~ Y

i s a n o n - l i n e a r m a p p i n g f r o m t h e H i l b e r t S p a c e ~ x U t o t h e H i l b e r t S p a c e ~ , a n d 0 xi s t h e n u l l e l e m e n t o f ~ ' . U i s t o b e t h o u g h t a s th e c o n t r o l s p a c e a n d ~ t h e s ta t e s p a c e .

L e t ~ b e a n o p e n s u b s e t o f ~ x U . T h e p r o b l e m o f o p t i m a l c o n t r o l i s t o f in d a p o i n tz - - ( u , x ) e f~ w h i c h s a ti s fi e s e q u a t i o n ( 7 1 ), s u c h t h a t t h e f u n c t i o n a l

f u, x , f : ~ U ~

i s a m i n i m u m .W e f o l l o w t h e n o t a t i o n a n d t e r m i n o l o g y o f D Im ,;D O l,m~ [19] .

Assumptions. i .f 8 C 2 ~ ; R) ; a 8 C 2 ~ ; )

i .e . t h e m a p p i n g s f a n d g p o s s e s s c o n t i n u o u s s e c o n d F r e c h e t D e r i v a ti v e s.

ii. T h e m a p p i n gDg (u ,x ) 8 . Z ( ~ x U ; ~ ) i s o n t o .

i ii. D ~ g ( u , x ) i s a l in e a r h o m e o m o r p h i s m o f ~ o n t o ~ .We a r e t h e n a b l e t o p r o v e [ 1 6 ] .

Theorem 1 . ( N e c e s s a r y c o n d it i o n ). U n d e r t h e a b o v e a s s u m p t i o n s , n e c e s s a r y c o n -d i t i o n s f o r f t o h a v e a m i n i m u m a t ( u , x ) s u b j e c t t oO(u, x)=O x are

< D j ( u , x ) , ( q ,0 ) > . + < 2 , D .O (U ,x ) ( t l , 0 ) > . = 0 72)

< D x f ( u ,x ), ( 0, t 2 ) > x + < 2 , D x g( u 0, x ) ' (0 , t 2 ) > x - 0 73)

a n d g ( u , x ) = 0 x ( 74 ), w h e r e < > , a n d < > x d e n o t e i n n e r p r o d u c t s i n H i l b e r t S p a c e sU a n d ~ r a n d 2 is a u n i q u e e l e m e n t o f ~ .

Theorem2 . ( N e c e s s a r y c o n d i ti o n ) . I f t h e h y p o t h e s e s o f T h e o r e m 1 h o l d a t t h e p o i n t(u , x) , t h e n a n e c e s s a r y c o n d i t i o n f o r f ( x ) t o h a v e a m i n i m u m a t ( u ,x o) s u b j e c t t og u , x )= o~ s

l n f D~h(u, x) ( t~, t2)~:O

l l t l t 1

D g(u O,x ) ' ( t l , t2)-~O

w h e r eh f f if + < 2 , g > , t = f t l,t2) an d l i t[I-- m ax (l l t tH, ]lt2[I).

[1 9 ] J D ~ : Foundationso f Modern Analyaia.Academic, New Yo rk (1960).


15/15

Successive approximation metho ds for the solution of optimal control problems 149

Theorem3. (Sufficient Co ndi t ion) . Let a l l the hypothe ses o f The orem 1 hold and le tthe necessary cond i t ions of Th eore m 1 be satisf ied. Fur ther le t

I n f D2h(u, x ) ( t l , t 2 )>0 , whe re h f f + < ) , g > . T h en t he reI l t l l l

Dg(u, x) t l , t2)=0~i s an open connec t ed ne ig hbo u rhood N o f (u, x), such thatf (u , x )>f (u , x ) fo r everyu x ~ N .

Theorem s 1-3 a re genera li sa tions o f fami l iar theorem s in the min imisa t ion o f a func t ionof n var iab les sub jec t to p cons t rain t s. In o rde r to so lve the m in imisa t ion prob lem, w ethere fore have to so lve the se t o f equa t ions

D ~ u , x , ~ ) ~ 0

D ,,~ u , x , ~ ) = 0

D ~ u , x , ~ ) = 0t (74)

One way o f so lv ing t hi s s e t o f equa t ion s i s t o u se New ton ' s M e tho d i n Func t i on Space .At each i t e ra t ion s tep we have to so lve the se t o f linear equa t ions g iven by

Dxh(~, ~,~)+D~h(~ , ~, ~).(0, 8x, O )+ D ~h (~, ~, ~).(Su, O, O)+D2~h(~,~ ~ . (0 , O , ~ )=0

D~ h ~, ~, ; t) + D ~J~ ~,~, ~)- 0,~x, 0)+D~.h a , ~ , ~) . ~u, o , o) ff ioD ,h(~, ~, ~)+ D ~h (fl , ~ , ~).(0, 6x , O )+ D2uh(~, ~,~)-(Su, 0, 0)

2 -+ D . ~ h u , ~ ~).(0, O 6~)----0

Base d on the w ork of ST~n~ and COLLATZ[17] suff ic ient con di t ions for th e N ew ton Proce ssto co nverge can be g iven .

The var ia t iona l p rob lem we have cons idered in th is paper ma y be recas t in to th is fo rmby wr i t ing the Eule r-Lagrange equa t ions in in tegra l fo rm.

9 . C O N C L U S I O N S

In th i s paper w e have cons idered som e successive approxim at ion me thods for theso lu t ion of a genera l c lass o f op t imal con t ro l p rob lems . The m ethods we have presen teda re fo rma l ly equ iva len t t o N ew ton ' s M e thod i n func t ion s pace . The m a in a dv a n t age o ft he me thods a r e r ap id it y o f conve rgence and s t ab l e compu ta t ion . Ho w ev e r i n manyproblems , i t m ay b e necessary to resor t to Grad ien t o r o ther m ethod s to ob ta in a suffic ientlygo od es t imate o f the nomina l co n t ro l func t ion . The m ethod d i rec t ly p rov ides ne ighbour ingopt imal feedba ck ga ins .

For the var ia t iona l p rob lems t rea ted here , i t has been assumed tha t inequa l i ty con-s t ra in t s on cont ro l and s ta te var iab les a re e i ther absen t o r adequa te ly approximated bymean s o f pena l ty func t ions . The resu lt s p resen ted here ex tend , in par t to cases where bo und son the con t ro l var iab le and s ta te var iab le a re p resent . W e shal l cover th i s in a subse quentpaper.

Acknowledgem~nt--Tl~ research wassupported b y the Cen tral Electricity Research Laboratories, Leather-head, England.

Date post:	02-Jun-2018
Category:	Documents
Upload:	fazdrul-akiff
View:	228 times
Download:	0 times

1 Successive Approx AUTO

Documents