12
Systems & Control: Foundations & Applications
Series Editor
Christopher I. Byrnes, Washington University
Associate Editors
S.- I. Amari, University of Tokyo B.D.O. Anderson, Australian National University, Canberra Karl Johan Astrom, Lund Institute of Technology, Sweden Jean-Pierre Aubin, EDOMADE, Paris H.T. Banks, North Carolina State University, Raleigh John S. Baras, University of Maryland, College Park A. Bensoussan, INRIA, Paris John Burns, Virginia Polytechnic Institute, Blacksburg Han-Fu Chen, Academia Sinica, Beijing M.H.A. Davis, Imperial College of Science and Technology, London Wendell Fleming, Brown University, Providence, Rhode Island Michel F1iess, CNRS-ESE, Gif-sur-Yvette, France Keith Glover, University of Cambridge, England Diederich Hinrichsen, University of Bremen, Germany Alberto Isidori, University of Rome B. Jakubczyk, Polish Academy of Sciences, Warsaw Hidenori Kimura, University of Osaka Arthur J. Krener, University of California, Davis H. Kunita, Kyushu University, Japan Alexander Kurzhanski, Russian Academy of Sciences, Moscow Harold J. Kushner, Brown University, Providence, Rhode Island Anders Lindquist, Royal Institute of Technology, Stockholm Andrzej Manitius, George Mason University, Fairfax, Virginia Clyde F. Martin, Texas Tech University, Lubbock, Texas Sanjoy K. Mitter, Massachusetts Institute of Technology, Cambridge Giorgio Picci, University of Padova, Italy Boris Pshenichnyj, G1ushkov Institute of Cybernetics, Kiev H.J. Sussman, Rutgers University, New Brunswick, New Jersey T.J. Tam, Washington University, St. Louis, Missouri V.M. Tikhomirov, Institute for Problems in Mechanics, Moscow Pravin P. Varaiya, University of California, Berkeley Jan C. Willems, University of Groningen, The Netherlands W.M. Wonham, University of Toronto
Xunjing Li Jiongmin Yong
Optimal Control Theory for Infinite Dimensional Systems
Birkhauser Boston • Basel • Berlin
Xunjing Li Department of Mathematics Fudan University Shanghai 200433 China
Jiongmin Y ong Department of Mathematics Fudan University Shanghai 200433 China
Library of Congress Cataloging-in-Publication Data
Li, HSiin-ching Optimal control theory for infinite dimensional systems / Xunjing
Li, Jiongmin Y ong. p. cm. - (Systems & control)
Includes bibliographical references and index. TSBN-I3: 978-1-4612-8712-4 e-TSBN-13: 978-1-4612-8712-4 DOT: 10.1007/978-1-4612-4260-4
I. Control theory. 2. Mathematical optimization. 3. Linear systems. I. Yong, 1. (Jiongmin), 1958- . II. Title. III. Series. QA402.3.IA878 1994 515'.64--dc20
Printed on acid-free paper
© Birkhauser Boston 1995 Softcover reprint of the hardcover 1 st edition 1995
94-37168 CIP
~® Birkhauser ~
Copyright is not claimed for works of U.S. Government employees. All rights reserved. No partofthis publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior permission of the copyright owner.
Permission to photocopy for internal or personal use of specific clients is granted by Birkhauser Boston for libraries and other users registered with the Copyright Clearance Center (CCC), provided thatthe base fee of$6.00 per copy, plus $0.20 per page is paid directly to CCC, 222 Rosewood Drive, Danvers, MA 01923, U.S.A. Special requests should be addressed directly to Birkhauser Boston, 675 Massachusetts Avenue, Cambridge, MA 02139, U.S.A.
Reformatted by Texniques, Inc. from authors' disks. Printed and bound by Quinn-Woodbine, Woodbine, NJ.
9 8 7 6 5 432 1
Contents
Preface .............................................................. ix
Chapter 1. Control Problems in Infinite Dimensions ............ 1
§1. Diffusion Problems ............................................. 1 §2. Vibration Problems ............................................ 5 §3. Population Dynamics ........................................... 8 §4. Fluid Dynamics ............................................... 12 §5. Free Boundary Problems ...................................... 15 Remarks .......................................................... 22
Chapter 2. Mathematical Preliminaries ......................... 24
§1. Elements in Functional Analysis ............................... 24 §1.1. Spaces ................................................... 24 §1.2. Linear operators .......................................... 27 §1.3. Linear functionals and dual spaces ......................... 28 §1.4. Adjoint operators ......................................... 31 §1.5. Spectral theory ........................................... 32 §1.6. Compact operators ........................................ 33
§2. Some Geometric Aspects of Banach Spaces .................... 36 §2.1. Convex sets ............................................... 36 §2.2. Convexity of Banach spaces ............................... 41
§3. Banach Space Valued Functions ............................... 45 §3.1. Measurability and integrability ............................ 45 §3.2. Continuity and differentiability ............................ 47
§4. Theory of Co Semigroups ...................................... 49 §4.1. Unbounded operators ..................................... 49 §4.2. Co semigroups ............................................ 52 §4.3. Special types of Co semigroups ............................ 55 §4.4. Examples ................................................. 57
§5. Evolution Equations .......................................... 63 §5.1. Solutions .................................................. 63 §5.2. Semilinear equations ...................................... 66 §5.3. Variation of constants formula ............................. 68
§6. Elliptic Partial Differential Equations .......................... 71 §6.1. Sobolev spaces ............................................ 71 §6.2. Linear elliptic equations ................................... 75 §6.3. Semilinear elliptic equations ............................... 78
Remarks .......................................................... 80
Chapter 3. Existence Theory of Optimal Controls ............. 81
§1. Souslin Space ................................................. 81
vi Contents
§ 1.1. Polish space ............................................... 81 § 1.2. Souslin space .............................................. 84 §1.3. Capacity and capacitability ............................... 86
§2. Multifunctions and Selection Theorems ........................ 89 §2.1. Continuity ................................................ 89 §2.2. Measurability ............................................. 94 §2.3. Measurable selection theorems ......... " ................. 100
§3. Evolution Systems with Compact Semigroups ................ 109 §4. Existence of Feasible Pairs and Optimal Pairs ................ 106
§4.1. Cesari property .......................................... 106 §4.2. Existence theorems ...................................... 110
§5. Second Order Evolution Systems ............................. 113 §5.1. Formulation of the problem ......... " ................... 113 §5.2. Existence of optimal controls ............................. 118
§6. Elliptic Partial Differential Equations and Variational Inequalities ................................. 121
Remarks ......................................................... 129
Chapter 4. Necessary Conditions for Optimal Controls - Abstract Evolution Equations ................. 130
§1. Formulation of the Problem .. " ...................... , ....... 130 §2. Ekeland Variational Principle ................................ 135 §3. Other Preliminary Results ................................... 137
§3.1. Finite co dimensionality .................................. 137 §3.2. Preliminaries for spike perturbation ...................... 143 §3.3. The distance function .................................... 146
§4. Proof of the Maximum Principle ............................. 150 §5. Applications ................................................. 159 Remarks ......................................................... 165
Chapter 5. Necessary Conditions for Optimal Controls - Elliptic Partial Differential Equations ......... 168
§1. Semilinear Elliptic Equations ....................... " ........ 168 §1.1. Optimal control problem and
the maximum principle .................................. 168 §1.2. The state constraints ..................................... 171
§2. Variation along Feasible Pairs ................................ 175 §3. Proof of the Maximum Principle ............................. 179 §4. Variational Inequalities ....................................... 183
§4.1. Stability of the optimal cost .............................. 184 §4.2. Approximate control problems ........................... 185 §4.3. Maximum principle and its proof ......................... 188
§5. Quasilinear Equations ........................................ 191 §5.1. The state equation and the optimal control problem ...... 191
Contents vii
§5.2. The maximum principle .................................. 196 §6. Minimax Control Problem .................................... 197
§6.1. Statement of the problem ............. " ................. 197 §6.2. Regularization of the cost functional. ..................... 199 §6.3. Necessary conditions for optimal controls ................. 200
§7. Boundary Control Problems .................................. 207 §7.1. Formulation of the problem .............................. 207 §7.2. Strong stability and the qualified maximum principle ..... 209 §7.3. Neumann problem with measure data .................... 212 §7.4. Exact penalization and a proof of
the maximum principle .................................. 214 Remarks ......................................................... 220
Chapter 6. Dynamic Programming Method for Evolution Systems ................................. 223
§l. Optimality Principle and Hamilton-Jacobi-Bellman Equations .......................................... 223
§2. Properties of the Value Functions ............................ 227 §2.1. Continuity ............................................... 228 §2.2. B-continuity ............................................. 231 §2.3. Semi-concavity ........................................... 234
§3. Viscosity Solutions ........................................... 239 §4. Uniqueness of Viscosity Solutions ............................. 244
§4.1. A perturbed optimization lemma ...... " ................. 244 §4.2. The Hilbert space Xu .................................... 248 §4.3. A uniqueness theorem .................................... 250
§5. Relation to Maximum Principle and Optimal Synthesis ....... 256 §6. Infinite Horizon Problems .................................... 264 Remarks ......................................................... 272
Chapter 7. Controllability and Time Optimal Control . ....... 274
§l. Definitions of Controllability ................................. 274 §2. Controllability for linear systems ............................. 278
§2.1. Approximate controllability .............................. 279 §2.2. Exact controllability ..................................... 282
§3. Approximate controllability for semilinear systems ............ 286 §4. Time Optimal Control - Semilinear Systems ................. 294
§4.1. Necessary conditions for time optimal pairs ............... 294 §4.2. The minimum time function .............................. 299
§5. Time Optimal Control- Linear Systems ..................... 302 §5.1. Convexity of the reachable set ............................ 303 §5.2. Encounter of moving sets ................................. 308 §5.3. Time optimal control ..................................... 315
Remarks ......................................................... 317
viii Contents
Chapter 8. Optimal Switching and Impulse Controls ......... 319
§1. Switching and Impulse Controls .............................. 319 §2. Preliminary Results .......................................... 322 §3. Properties of the Value Function ............................. 328 §4. Optimality Principle and the HJB Equation .................. 331 §5. Construction of an Optimal Control .......................... 334 §6. Approximation of the Control Problem ....................... 338 §7. Viscosity Solutions ........................................... 344 §8. Problem in Finite Horizon .................................... 352 Remarks ......................................................... 359
Chapter 9. Linear Quadratic Optimal Control Problems ..... 361
§1. Formulation of the Problem .................................. 361 §1.1. Examples of unbounded control problems ................. 361 §1.2. The LQ problem ......................................... 366
§2. Well-posedness and Solvability ............................... 371 §3. State Feedback Control ...................................... 379
§3.1. Two-point boundary value problem ....................... 379 §3.2. The Problem (LQ)t ...................................... 382 §3.3. A Fredholm integral equation ............................ 386 §3.4. State feedback representation of optimal controls ......... 391
§4. Riccati Integral Equation .................................... 395 §5. Problem in Infinite Horizon .................................. 401
§5.1. Reduction of the problem ................................ 401 §5.2. Well-posedness and solvability ............................ 405 §5.3. Algebraic Riccati equation ............................... 407 §5.4. The positive real lemma ........... , .................. " . .408 §5.5. Feedback stabilization .................................... 412 §5.6. Fredholm integral equation and
Riccati integral equation ................................. 414 Remarks ......................................................... 415
References ......................................................... 419
Index .............................................................. 443
Preface
Infinite dimensional systems can be used to describe many phenomena in the real world. As is well known, heat conduction, properties of elasticplastic material, fluid dynamics, diffusion-reaction processes, etc., all lie within this area. The object that we are studying (temperature, displacement, concentration, velocity, etc.) is usually referred to as the state. We are interested in the case where the state satisfies proper differential equations that are derived from certain physical laws, such as Newton's law, Fourier's law etc. The space in which the state exists is called the state space, and the equation that the state satisfies is called the state equation. By an infinite dimensional system we mean one whose corresponding state space is infinite dimensional. In particular, we are interested in the case where the state equation is one of the following types: partial differential equation, functional differential equation, integro-differential equation, or abstract evolution equation. The case in which the state equation is being a stochastic differential equation is also an infinite dimensional problem, but we will not discuss such a case in this book.
It is known by now that the Pontryagin maximum principle, the Bellman dynamic programming method, and the Kalman optimal linear regulator theory are three milestones of modern (finite dimensional) optimal control theory (see Fleming [1]). The study of optimal control theory for infinite dimensional systems can be traced back to the beginning of the 1960's. A main goal of such a theory is to establish the infinite dimensional version of the above-mentioned three fundamental theories. In the past 30 years, many mathematicians and control theorists have made great contributions in this research area.
In recent years we and some of our colleagues have been involved in the study of optimal control theory for infinite dimensional systems. Compared with the works of other mathematicians, we find that ours have their own flavor, and some of the methods might interest other people working in this area or in some related fields. Also, in the past few years, we have taught several courses entitled "Optimal Control for Distributed Parameter Systems" to graduate students at the Institute of Mathematics, Fudan University. The materials were taken from our recent works; many new results developed by other mathematicians in recent years were also adopted. We decided that it would be better to write a book to give a unified presentation of these theories.
The main feature of this book is the following. First of all, we have given a unified presentation of optimal control theory for infinite dimensional systems. This includes the existence theory, the necessary conditions (Pontryagin type maximum principle), the dynamic programming method (involving the viscosity solution of infinite dimensional Hamilton-Jacobi-
x Pr'eJace
Bellman equations), and the linear-quadratic optimal control problems. Second, we have made efforts to provide self-contained proofs for many preliminary results that were not contained in previous control theory books. This will help many graduate students or scholars from other fields understand the theory. Among these proofs, let us mention two such efforts: (i) We have spent a very reasonable number of pages introducing the theory of Souslin space and general selection theorems for measurable multifunctions, which are essential in the presentation of existence theory. (ii) We present a perturbed optimization lemma resulting from the work of Ekeland and Lebourg, which is closely related to the well-known Ekeland's variational principle and plays an important role in the proof of the uniqueness of viscosity solutions. Third, instead of just making some negative statements, we have presented or cited a number of counterexamples, such as nonclosable operators, continuous functions not attaining an infimum on the closed unit ball in infinite dimensional spaces, nonconvexity of the reachable set in infinite dimensional spaces, etc. This will help the readers understand some basic features of infinite dimensional spaces. Finally, to keep the book at a reasonable length, we had to leave out much important material. To compensate for this, we have made some brief bibliographic remarks at the end of each chapter to survey some of the related works.
From the above, we see that this book is something between a monograph and a textbook. It is our desire that the book be useful for graduate students and researchers in the areas of control theory and applied mathematics. People from other fields such as engineering and economics might also find the book valuable.
There is a vast amount of literature devoted to the study of optimal control theory for infinite dimensional systems and related problems. We have not attempted to give a complete list of references. What we have cited at the end of the book are those that we find closely related to our presentation. We admit that very many important works might be overlooked. Fortunately, while preparing the book, we were informed that Professor H.O. Fattorini (of UCLA) was writing a book relevant to ours. The readers are suggested to consult that book for some related works on the topic and a possibly better list of references.
The book is organized as follows. We begin with many examples of control problems in infinite dimensions in Chapter 1. Chapter 2 discusses some very basic elements of preliminaries such as functional analysis, semigroups, evolution equations, and elliptic partial differential equations. The rest of the book is divided into five " parts." The first part is Chapter 3, in which the existence theory for optimal controls is presented. The systems discussed contain elliptic, parabolic, and hyperbolic partial differential equations. Results for elliptic variational inequalities are also presented. The second part consists of Chapters 4 and 5. In these two chapters, we present necessary conditions of the Pontryagin maximum principle type. The results cover abstract evolution equations, elliptic partial differential equations, and vari-
Preface xi
ational inequalities. Distributed control, boundary control, and minimax problems, etc. are discussed. The main tools are the Ekeland variational principle and the spike variation technique for different systems. The third part consists of Chapters 6 and 8. In this part, we present the Bellman dynamic programming method for optimal control of abstract evolution equations. A viscous solution in infinite dimensions is the main topic in these two chapters. The relation between the Pontryagin maximum principle and the viscosity solution to the Hamilton-Jacobi-Bellman equation is studied. The optimal switching and the impulse control problem are also discussed. The fourth part is Chapter 7, in which we discuss the controllability and the time optimal control problem. Linear as well as semilinear systems evolutions are treated. In the last part, Chapter 9, we discuss the linear-quadratic (LQ for short) optimal control problem. We concentrate on the parabolic problem with unbounded control. The general problem (not necessarily parabolic) with bounded control is also covered. The operators appearing in the cost functional are allowed to be indefinite. For the finite horizon case, it is shown that, in some sense, the solvabilities of the LQ problem, an operator Riccati equation and a Fredholm integral equation are equivalent. Then the infinite horizon case is briefly discussed. The stabilization of the system is naturally contained in this part.
Some words about the numbering convention. All the heads (by this we mean definitions, examples, lemmas, propositions, remarks, theorems, etc.) are numbered consecutively within each section of each chapter with the first number being the section number. For example, Theorem 3.2 is the second head in Section 3. When a head of another chapter is cited, the number of that chapter will be indicated each time. The equations are also numbered consecutively within each chapter with the first number being the section number. Thus, (4.5) refers to the fifth equation in Section 4.
The authors would like to acknowledge support from the following: the NSF of China, the Chinese State Education Commission NSF, and the Fok Ying Tung Education Foundation. For many years the authors benefited from their colleagues at Fudan University, to whom we would like to extend our thanks. Also, the following colleagues deserve special acknowledgment: Shuping Chen and Kangsheng Liu (Zhejiang University); Zuhao Chen and Shige Peng (Shandong University); Fulin Jin, Liping Pan, Shanjian Tang, and Yunlong Yao (Fudan University); and Jingyuan Yu (Beijing Institute of Information and Control). During the preparation of the book, the second author spent a year participating in the Control Year held at the Institute for Mathematics and Its Applications, University of Minnesota, U.S.A. He would like to thank Professor Avner Friedman for his kind invitation and partial support. During that period the second author benefited from many stimulating conversations, discussions and collaborations with other visitors. Among them, the following deserve special acknowledgment: Gang Bao, Eduardo Casas, Hector O. Fattorini, Scott W. Hansen, Suzanne M. Lenhart, Walter Littman, Wensheng Liu, Zhuangyi Liu, Jin Ma, Srdjan
xii Preface
Stojanovic, and Bing Yu Zhang. In his mathematical career, the second author has been deeply influenced mathematically by Professor Leonard D. Berkovitz, to whom he owes sincere gratitude and would like to dedicate the book. Finally, the authors would like to thank their family members for their understanding and patience during the long and tedious preparation of the book.
Xunjing Li and Jiongmin Yong Fudan University, China
August 1994
