TR,ENDS IN AERODYNAMICS DESIGN AND OPTIMIZATION: A MATHEMATICAL VIEWPOINT

S hlomo Ta'a,sa.n* Carnegie Mellon University

Pittsburgh

Abstract

In this paper we discuss key issues and directions of development in aerodynamics design and optimiza- tion. We focus on a mathematical viewpoint that address the main difficulties in these problems. Prob- ably the most important issue is the construction of efficient solution algorithms. This however, turns out to be tigl~t~ly related to other issues such as well posed- ness and the condition number of the Hessian. An import4ant analytical tool for analyzing deferent ele- ments of the problem is described. It is found use- ful in several ways including assessment of t'he well- posedness of the problem; reformula&ion of the de- sign ~pt~imization problem that yields better mathe- matical problem setup; and construction of efficient preconditioners for gradient based methods. Differ- ent solution strategies are discussed including black box met hods, sensitivity gradients, adjoint methods, multigrid acceleration of the analysis as well as the full optimization problem, and non-gradient meth- ods.

1 Introduction

In the last few years there has been a growing interest in the numerical solution of optimization problems governed by large scale problems. This new interest was a direct result of the improvement in computer technology.

Probably the most challenging problems are those which involve complex governing equations. This include, for example, Euler eq~at~ions, Navier- Stokes equations, Acoustic wave equations, and t,he Maxwell's equations.

Some global quantities governed by t'he solutions of such problems are required to be minimized (max- imized) in terms of some prescribed design variables; for example, find a shape of a wing with minimal drag

* Associat,e Professor, Mathematics O Copyright 01995 by the American Institute of Aeronau-

tics and Astronautics, Inc. All rights reserved.

for a given lift. The resulting mathematical problem is formulated as a constrained optimization problem which can be viewed in some cases as control prob- lems.

Currently used methods can be divided into gra- dient methods and non-gradient methods. Gradi- ent methods have been used extensively in the re- cent years while non-gradient methods are in the emergence stage. Existing methods fo'r calculat- ing gradients include black box methods which use finite differences [GNH] , automatic differen- t,iation methods [CGBN] , adjoint methods and their different accelerations using multigrid ideas [AT11 ,[AT21 ,[BD],[JI,[TKSl] ,[TKS2]

Efficient gradient calculation can be done using the adjoint equations, and in area of aerodynamics design was first suggested in [J]. In this algorithm each op- timization step requires the solution of the state and costate equations, and an efficient implementation was achieved by using multigrid methods for both equations. No acceleration of the optimization pro- cess was involved in this work.

The one shot method proposed in [TI] for control problems, uses the adjoint method as well, together with multigrid acceleration for state and costate; but includes an acceleration of the minimization process as well. Its development so far was done for ellip- tic partial differential equations as the constraints. Its main idea is that smooth perturbation in the data of the problem introduces smooth changes in the solution, and highly oscillatory changes in the data produces highly oscillatory changes in the so- lution. Moreover, highly oscillatory changes are lo- calized. These observations enable the construction of very efficient optimization procedure, in addit!ion to very efficient solvers for the state and costate vari- a.bles. Actually, the optimization process was embed- ded into the solution process of the governing equa- tion. Design variables that correspond to smooth changes in the solution were solved on coarse levels and those corresponding to highly oscillatory changes were solved for on appropriate finer grids. The re-

sulting method can be viewed as a preconditioning of the gradient descent method where the new condition number is independent of the grid size, and is of or- der 1. Thus, within a few optimization iterations one reaches the minimum. The method was developed first for small dimensional parameter space, where the optimization was done on the coarsest grid, yet converging to the fine grid solution [TI]. Later in [TKSl],[TKS2] t8he met'hod was applied to cases of moderate number of design variables and where the optimization was done on few of the coarsest grids. The extension of these ideas to the infinite dimen- sional design space was done in [ATl],[AT2] where both boundary control as well as shape design prob- lems were treated. The performance of one shot methods is independent of the number of design vari- ables, and typically the total cost of the algorithms is just a few times that of the state equation.

the set of necessary conditions, together with an en- ergy estimate that guarantee convergence. The evo- lution process can be accelerated using multigrid or other acceleration techniques making the approach is very flexible. Current experience with these methods shows that the full optimization problem is indeed solved within the cost of just a few analyses indepen- dent of the number of design variables. Moreover, no multigrid is essential for achieving that efficiency.

In this paper we review some of the existing meth- ods for aerodynamics design optimization and present an important practical analysis tool for the opti- mization problem under study. It was originated in [ATl],[AT2] in the development of multigrid one shot methods for infinite dimensional design space. How- ever, that toolcan be used for a variety of other pur- poses as well. Its gives the structure of the functional near the minimum and reveals important properties

In [AT11 ,[AT21 specid relaxation techniques for the solution of constrained optimization problems were designed. These relaxation methods were constructed so tha.t high frequencies in the design variables will be fast to converge. Using essentially the same anal- ysis tool presented there, one can construct efficient preconditioners for the gradient method. This, when combined with adjoint formulation may lead to meth- ods which are alternative to the one shot methods, and have simpler implementation.

A search for a special type of non-gradient methods for optimization problems was initiated in [T2]. The

of the solution, e.g., stability to changes in shapes as a result of changes in flow conditions. It enable refor- mulation of the design problem such that the same engineering task is kept, yet a better mathematical formulation is obtained. In addition its gives a use- ful infinite dimensional approximation to the Hessian which can be used to accelerate gradient based meth- ods.

We also discuss non-gradient methods which use pseudo-time techniques for reaching the minimum. A general description and some theoretical foundation is given.

goal was to construct methods that do not depends on multigrid acceleration, and that the cost of solving 2 Issues in Aerodynamics De- the full optimization problem using them will be close to that of solving the analysis problem. The first ob- sign servation ma.de was that the solution when using the adjoint method is an intersection of three hypersur- faces describing the state equation, costate equation and design equation (together forming the necessary conditions of the minimization problem). The adjoint met,hod can be viewed as marching on the intersec- tion of the hypersurfaces corresponding to the state and the costate equa.tions, in the direction of the in- tersection with the design hypersurface. Each st,ep in t8hat process require the solution of a large scale

The main issues in formulating a design problem as an optimization problem are s tabi l i ty and comp lex i t y .

In aerodynamics design problems one is interested to design configurations such as wing or body shapes that result in better performance. Such performance measure is given in terms of global quantities gov- erned by the solutions of the fluid dynamics equations such as lift, drag etc. The mathematical problem is formulated as a constrained optimization problem.

problem (da te and costate equa,tions). In most appli- An essential property of any formulation of a design cation the number of design variables is significantly problem as an optimization problem is the s tab i l i t y of smaller t h m the number of state or costate variables the solutions (in addition to their existence). From and marching on the design hypersurface is therefore physical consideration one would like the design to a much less expensive process. In [T2] pseudo-time depend continuously on dat'a in the problem. For ex- techniques that march along the design hypersurface ample, a shape of an optimal wing should not change are presented and analyzed. These methods are not too much if the flow conditions change slightly. Oth- based on gradient information which may be expen- erwise, such an optimal wing would be of littjle prac- sive to calculate. The construction of these methods tical use. involve an approximation of an evolution process for Unlike the fluid dynamics equations which are

given by the physics, the cost functional and the de- sign space are not unique even for a given design task. For example, instead of designing for the shape one can design for the slopes, or curvatures of the shape. The actual shape can be reconstructed from those easily and from t'he engineering point of view t'hose formulations are equivalent. The mathematical na- ture of different formulations for the design space are very different as will be shown later. Changes in the design space can change dramatically the structure of the minimum; changing an ill-conditioned problem into a well-conditioned one.

The above is relat,ed to another important concept which we refer to as complexity of the optimization problem. It is given in terms of the asymptotic be- havior of the condition number of the Hessian as a function of the number of the design variables. In general the behavior is of the form O ( M Y ) where M is the number of design variables. The parameter y is a measure for the level of complexity of the prob- lem. Higher values of y correspond to more difficult op tirnization problems.

2.1 Main Algorithmic Difficulties

When considering gradient based methods for the so- lutjion of optimization problem it is useful tlo consider the level lines of the cost functional. If the level lines are close to circles, then gradient based algorithms will be fast to converge since the gradient (with a mi- nus sign) point's toward the minimum, If on the ot8her hand those level curves are thin ellipses the gradient, does not point toward the minimum in general, and therefore gradient based methods will be slow to con- verge. The thin ellipses correspond to bad condition- ing (large ratio of largest to smallest eigenvalue) of the Hessian.

It is important to notice the following two cases. The first is when by removing just a few eigenvalues a well conditioned system is obtain. In such cases meth- ods which uses approximate Hessian which is con- structed during the iterative process, such as BFGS, lead to very effectlive solvers. The second case is when the condition number remains high even after remov- ing a large number of eigenvalues. This is typical to problems arising from discreti~at~ion of partial differ- ential equations where the number of design variables is large. Iterative algorithms of the BFGS type can- not serve as a remedy in this case and different ap- proaches are needed. One such approach is described in section 5.

3 Gradient Algorithms

Consider the problem

min, F ( a , /7(a)) L(U(a), a ) = 0 (1)

Gradient based minimization methods can be writ- ten as

where 5 is a step size, whose magnitude ca,n be deter- mined using a line search.

Several approaches for calculating gradients of F subject to the constraints exist, and we discuss some of them in this section.

We introduce the not'ation A for the Hessian, which gives the following representation of the functional in the vicinity of the minimum cro

where ii = a - -0. It can be shown that convergence rate for gradient type methods is determined by

The structure of the minimum is essentially deter- mined by A and it)s analysis in the context of fluid dy- namics equation will be demonstrated later. It plays a major role in the optimization problem and its so- lution processes.

3.1 Black Box Methods

The simplest approach to solve optimization prob- lems governed by t,he fluid dynamics equations con- sists of calculating the gradient in the following way. Let 6F be the change in the cost functional as a result of a change 6 in the design variables. The following relation holds

All quant'it'ies in this expression are straightforward to calculate except for z. The dimensionality of this quantity (for the discretization) is the dimension of U times the dimension of a .

The calculation of is done using finite differ- ences. That is, for each of t,he design parameters aj

in the representation of a as a = xy=l aj ej , where ej are a set of vectors spanning the design space, one perform the following process

Solve L(U(a + cej ), a + cej ) = 0 given a , c 3.3 Multigrid Acceleration

Questions regarding the size of c t80 be taken in this approach that will not be discussed here.

Once the above process was complet'ed one combine t,he result into

which is used in calculation of the gradient

Since in practica.1 problems the dimension of U may be thousands to millions, the feasibility of calculating gradients using this approach is limited to cases where the number of design variables is very small.

3.2 Adjoint Methods Adjoint methods are efficient ways for calculating gra- dients for constrained optimization problems, even for very large dimensional design space. The term in the representation of the gradient can be replaced us- ing the equat,ion

leading to

Introducing the variable X = -(L;)-' $$ we obtain the gradient in the form

where X (the codate variable) satisfies the equation

A minimization algorit'hm which uses an adjoint formulation can then be written as repeated applica- tion of the following three steps

Given a solve L (a , U) = 0 for U

Given a , U solve L: X + = 0 for X

The simplest form of multigrid acceleration of the gra- dient descent met,hod was suggested first in [ J ] . The idea was to accelerate the solution process for the state and costate equations using a multigrid solver. The implementation of this idea requires program- ming a multigrid code for the adjoint equation, which is essentially very similar to the original state equa- tions.

A m~l t~ igr id development that accelerate also the gradient descent method in addition to acceleration of the state and costate equations was initiated in [TI] and referred to as one-shot method. Further devel- opment for aerodynamics application using the small disturba.nce arrd full potential equations was carried out in [TKSl] ,[TKS2]. In these works the parametric representation of the shape to be optimize was done using a fixed number of shape functions. Extension of the idea to the case where the shape is represented by a function was done in [ATI],[AT2], In all one shot methods one construct a sequence 6f projection operators Pj on the design space such that

and such that t'he condition number of the Hessian restricted to the space corresponding to each of the Pj is not large. The gradient descent method is then replaced by

The actual implementat<ion of the method takes into account an additional property of the partition of the design space, namely, different projection op- erators Pj correspond to different level of smoothness in the solution. Thus, the actual update of the design variables Pja is done on the appropriate grid. The efficiency of the method depends on the largest condi- tion number of the Hessian restricted to the subspaces defined by the operators Pj . In actual numerical ex- periments these condition numbers were of the order of 1.

3.4 Single Grid Preconditioners

In the absence of a multigrid code for solving the state and costate equat'ion one has to consider single grid iteration. Preconditioning techniques have been used widely for accelerating the convergence of solutions for a variety of discretizations of PDE. Gradient de- scent methods viewed as solution processes for a set of

viewed as an approximation to the following evolu- tion process

d -U + L(U,a) = 0 dt (I5) with the boundary condition

(17) At the minimum the following equation has to be satisfied

Gradient methods on the other hand can be viewed as approximation to

(18) The left hand side of this equation is the gradient of the functiond subject to the PDE, and its behavior

(19) in the vicinity of the minimum needs to be analyzed. In order to do that we examine the perturbation in

(20) the solution as a result of a perturbation in the design function a . The linearity of the interior equation - im- plies that the perturbation variables ii, E , $, fi satisfy

of Gradients Re- the same equations as u , v, A, p in the interior of the

duction to the Boundary domain, and the boundary conditions for them are

g = s In order to get a quantitative description of the level ax curves of the cost function and to be able to deter- - f i + AllG + A12E = 0 (27)

mine the structure of the functional near the mini- where mum, a Fourier analysis of the gradient of the func- tional is carried out. In most problems of engineer- All = 2b; + (P - P*)PUU]

ing interest the design variable are associated wit,h A12 = ~ [ ~ U P U + (P - P*)PUV] (28)

boundary quantlities and the gradient of the cost func- ~l~~ change in the gradient is given by tion is a quantity defined on pa.rt of the boundary as well. Thus the analysis of the gradient is done on the -[-A d + Azlii + Az2Z;] boundary of the domain. dx

For the purpose of demonstrating the analysis we where consider the following problem

P

min / (p - ~ * ) ~ d x (21) The analysis continues by assuming & to be a J

subject to Fourier component, that, is,

& = exp(iwx) (31)

p2 & a ( g - ) ( ) = ( ) 11 (22) Since the problem involves coefTicients which are not constants, e.g,. All etc., we perform the analysis in a

with the boundary c~ndi t~ion small vicinity of the boundary such that is smaller than the scale on which the coefficient changes signif-

aa a11

icant,ly. In that case we can assume that all coeffi- v = -

dx (23) cients in the problem are constants and their values are given by freezing them at t,he point a t which the

where P2 = - M2) and P = f (u2 + v2)7 'Ome analysis is performed, say lo E aQ, prescribed funct~ion f (q) . For ~implicit~y the domain Thus, zi = iw exp(iwr) and using the interior equa- is S1 = {(x, y))y 2 0,-00 < x < 00). t8ions for ( 2 1 , G ) we conclude t'hat,

In order t*o analyze the gradient of the functional subject to the PDE, we introduce Lagrange multipli- ers ( A , p ) which can be shown to satisfy (32)

Using the boundary conditions for b we get that its boundary values are fi = [Al 1 lw l/P +A12iw] exp(iwx) and from the interior equations for ( A , 9 ) it is easy to see that

Combining these results we obtain that the change in the gradient, corresponding to a change in the de- sign variable by ti = exp(iwx) is

Thus, for this problem the symbol of the Hessian d(w ) satisfies

5.1 Problems Classification

Using the above analysis we c,an classify problems ac- cording to the asymptotic behavior of d (w) for large

Iw I. Ill-Posedness that result from the behavior of high

frequencies can be characterized' as

To see that such a behavior cause ill-posedness, con- sider a. solution a and a perturbation of it in the form a + cexp(iwx). The gradients evaluated for these two choices for the design variable are A(a) and A ( a ) + cd(w) exp(iwx). The later can be ap- proximated by

A(a) + c / lw 1' E A(a ) for large lw 1 (36)

Since this is true for an arbitrary c we gei that if a is a solution then a + cexp(iwx) is an approximate solution for an arbitrary c, for large w. This implies t,hat small changes in the data of tmhe problem will cause large changes in the solution.

Well posed problems are characterized by

Here small changes in the design variable in the high frequency range cause large changes in the gra- dient,. Hence, small changes in the data results in small changes in the solution, and the solution has good s tabi l i t ,~ pr~pert~ies.

On a finite grid with meshsize h one consider w in the range lwl < ~ / h . Thus, for well-pose problems, the eigenvaloes of the Hessian corresponding t,o the highest frequencies behave as

The condition number of the Hessian is O(&). This quantity is important in evaluating the performance of gradient based algorithm. As was mentioned in section 3, the convergence of gradient methods is de- termined by I - SA, and the high frequency compo- nents in the representation of the solution converge at a rate

The expected rate of convergence for the full design problem is therefore

From this analysis we conclude that t h ~ complexity of a given problem can be determined by the exponent 7.

Easy optimization problems: 0 5 y << 1.

Dificult optimization problems: y 2 1

5.2 Problem Reformulation Unlike the constraint PDE which governs the opti- mization problem under study, the parameter space as well as the cost functional are not uniquely deter- mined for a given engineering task. It is very likely that in general a proper choice for one or both can lead to well posed problems that are easier to solve.

We demonstrate this fact by an example. The prob- lem given in section 5 is shown to have good stability properties for the high frequencies. However, y = 2 for that problem, and the rate of convergence for gra- dient based methods is expected to be 1 - o (M-~) , for M design variables. Different choices for design space and cost functional will be shown to have very different behavior, although the engineering task re- mains roughly the same.

Consider the problem

subject t,o

with the boundary condition

The doma,in R , as well as P, p, f are as defined previ- ously.

Following the same procedure as before, we intro- duce Lagrange multipliers (A, p) which can be shown to satisfy

with t3he boundary condition

At the minimum the following equation has to be satisfied

Following a similar derivation as before we find that the perturbations G , G , X, ji corresponding to a pertur- ba.t,ion 6 in a sa.trisfy

and

Combining t,hese results we obtain that the change in the gradient,, corresponding t'o a change in the de- sign variable by 6 = exp(iwx) is

Thus, for t8his problem we find that

~ d ( w ) l = O(1) for large w (49)

As a result of this calculation we can derive t,he following conclusion. If the design variables are the slopes instead of the shape itself, a well-posed prob- lem is stmill ~ b t ~ a i n ~ d . Moreover, t,his problem is much easier to solve that the one for the shape directly. Note t,llat from the engineering point of view both problems can be used tmo perform the required design. In the second one , a reconstruction of the shape frorn the slopes has to be done and this itself is a stable problem.

5.3 Efficient Preconditioners

The observation that the convergence rate for gradi- ent descent methods is governed by I - SA suggests that effective preconditioners can be constructed us- ing t8he behavior of d (w) for large w. The idea is simple. Assume that

and let R be an operator whose symbol ~at~isfies

1 lR(w) = 0(-) for large lw 1

lw l Y (51)

The behavior of the preconditioned method is deter- mined by -

I - SRA (52)

whose symbol

approaches a constant for large w. A prgper choice of 5, e.g., using a line search leads to a convergence rate which is independent of the dimensionality of the design space. This is not the case if the symbol of the iteration operator has some dependence on w.

As an application of the idea consider the problem (21)-(23). It was shown there that y = 2 for that problem. From the above, an effective preconditioner R must satisfy lR(w) I = o(&) and this is obtained for

since the symbol of -$ is given by - (w 1 2 . Thus, the implementation of a preconditioned iteration for problem (21)-(23) consist of repeated application of the two steps

where 5 is found using a line search. Note that the construction of the preconditioner was done on t'he differential level but the numerical implementmation is using some approximation of i t , e.g., finite difference

6 Analysis of Pseudo-Time Met hods

The behavior of pseudo-time methods can be done using energy estimates for the relat'ed evolution prob- lem. We demonstrate t8his analysis on a slightly dif- ferent problem to avoid some technicalities. Consider the problem

and the same boundary conditions for the perturba-

i t ions.

min (u - ~ * ) ~ d s (56) We now show that perturbation from the minimum converge to zero. Consider the X part of it. From the

subject to equations (22),(23) with /? = 1. integration by parts formula and the condition ;\ = 0 The necessary conditions consist of equation (22) we have

and (24) in the interior and the following set of equa- tion on the boundary.

v = a a ax

-11, + 2(u - u*) = 0 (57) Then using Poincare's inequa,lit'y we have, for arbi- -A = 0 trary functions that vanish on part of Q, e.g., the

boundary, Define R, , R, , Rx, R, by

- Jlw n 'dv 5 - c L m 2 d v (65)

( ) - -- a y a~ for some c > 0 independent of 4. Thus,

and d / ,

- Jn ( ) ) ( (59) and therefore

and consider the evolution problem

and a similar equation for (A, p) . It can be easily seen that we get for each of the four quantities u, v , A , p the following parabolic equation

with different a boundary condition. Using integra- tion by parts for this equation one obtain

and this relation holds for u , 21, A , p. as well as for their perturbations. In the first order system we had only one boundary condition for each of the two by two systtems. The syst,em of four parabolic equa.tions re- quires t,wo additiona.1 boundary conditions. These a,re obtained from the first order st,eady state problem for the st,a.te and costtatte unknowns. Summa,rizing, we have the following boundary conditions

V = a " ax

- p + 2(u - u*) = 0 -A = 0

where Xo is the initial approximation for ;\. That is, A converges exponentially to zero. Using the Neu- rnann boundary condition for fi and the fact that X converges to zero we obtain the same result for ji. Using the boundary condition -i, + 2ii = 0 we get that ii tends to zero as well, and from the Neumann condition for v" and the integration by parts we ob- tain the convergence of v" to zero. The error in the design variable 6 converges to zero as well. Thus, the pseudo-time algorithm converges to the minimum starting from an arbitrary point.

Discussion

In this paper we have discussed major directions of developments in the area of aerodynamics design op- timization. Gradient as well as non-gradient meth- ods have been discussed. An analytical tool for un- derstanding different aspects of the design problem have been described. Its useful for several aspects of the problem; proper for~nulation of the engineering tasks; reformulation to get better mathematical prob- lems which are easier to solve; construction of sin- gle grid preconditioners; mul'tigrid acceleration. Non- gradient methods based on pseudo-time embedding of the necesmry conditions have been discussed, show- ing pro~nising potential in aerodynamics applications.

References

[ATl] Arian, S. Ta'asan, Sha.pe Optimization in One Shot. Proceedings of a Workshop on Optimal De- sign and Control held at Blacksburg, VA, April 8-9, 1994.

[AT21 Arian, S. Ta'asan, Mult$igrid One Shot Met,h- ods for Optimal Control Problems: Infinite Di- mensional Control. ICASE report No. 94-52.

[BD] F. Beux and A. Dervieux. A Hierarchical Ap- proach for Sha.pe Optimization, Inria Rapports de Recherche, N. 1868 (1993)

[CGBN] A. Ca.rle, L.L. Green, C.H. Bischof and P. Newman: " Applica.t,ion of Automatic Differenti- ation in CFD" AIAA 94-2197, June 1994.

[GNH] L. Green, P. Newman and K. Haigler: "Sensi- tivit'y Derivat'ives for advanced CFD Algorithms and Viscous Modelling Parameters via Auto- matic Differentiation", AIAA 9303321, 1993.

[HMY] W.P. Huffmann, R.G. Melvin, D.P. Young, F.T. Johnson , J .E. Bussolett,i, M.B. Bieterman a.nd C.L. Hilmes, "Pra,cttical Design and Op- t$imiza,t,ion in Computationa,l Filud Dynamics", AIAA 93-3111 July 1993.

[.J] A. J ameson: " Aerodynamics Design via Control Theory" J . Sci. Comp. Nov. 21, 1988.

[MPS] B. Mandel, J . Periaux, B. Stoufflet. Optimum Design Methods in Aerodynamics, AGARD- FDP-VKI Special Course, April 25-29, (1994)

[TI] S. Ta'asan: One-Shot Method for Optimal Con- t'rol of Dishibuted Parameter Systems I: Finite Dimensiona.1 Cont,rol. IClASE Report No. 91-2, 1991

[T2] S. Ta'asan: Pseudo-Time Methods for Con- strained Optimization Problems Governed by PDE. ICASE Report No. 95-32, 1995.

[TICS11 S. Ta'asan, M.D. Salas and G. Kuruvila: Aerodynamics Design and Optimizat ibn in " One Shot7'. Proceedings of the AIAA 30th Aerospace Sciences Meeting and Exhib;t , Jan 6-9, 1992, Reno NV.

[TIC321 S. Ta'asan, G . ICuruvila, and M.D. Salas: A New Approach to Aerodynamics Optimization. Proceeding of the First European Conference on Numerical Methods in Engineering, Sept 1992, Brussels, Belgium.