[American Institute of Aeronautics and Astronautics 20th AIAA Applied Aerodynamics Conference - St....

1American Institute of Aeronautics and Astronautics

TOWARDS A HYBRID AERODYNAMIC DESIGN PROCEDUREBASED ON NEURAL NETWORKS AND EVOLUTIONARY METHODS

Man Mohan Rai*NASA Ames Research Center, Moffett Field, CA-94035

ABSTRACT

Neural networks and evolutionary methods arefinding increasing use in aerodynamic design. Neuralnetworks have been used in the context of responsesurface methodology to construct accurate responsesurfaces that model the behavior of the objectivefunction in design space. The network-based responsesurface is then searched for the optimum. Sequences ofsuch response surfaces have also been used to traversethe design space in cases where the initial design is farfrom optimal. Neural networks have an advantage overconventional polynomial-based methods because they donot require the regression terms to be specified inconstructing response surfaces. Evolutionary algorithmshave also been used successfully in aerodynamic design.Examples include three-dimensional wing design as wellas compressor and turbine airfoil design. Evolutionarymethods are far more capable than local searchprocedures in finding global optima of multimodalfunctions and in handling cases with multiple feasibleregions. They are also finding use in multiple-objectiveand multidisciplinary optimization. Here we present anevolutionary method and apply it to some test problemsas well as nozzle and turbine airfoil design. The resultsof applying a neural network-based design procedure tothe same aerodynamic design problems are alsopresented. The possibility of integrating designprocedures based on neural networks and evolutionaryalgorithms is explored here, with the objective ofcreating a hybrid that has the strengths of both methods.

___________________________*Senior Scientist, Information Sciences and TechnologyDirectorate. Associate Fellow, AIAA.NASA has filed a patent application that covers some ofthe original ideas in this article.

INTRODUCTION

In recent years soft computing methods such asneural networks and evolutionary algorithms have foundincreasing use in many disciplines of aeronauticalengineering, including aerodynamic design. Neuralnetworks have been used in the context of responsesurface methodology to construct accurate responsesurfaces that model the behavior of the objectivefunction in design space. Networks have also beensuccessful in extrapolating data thus yielding significantreductions in design costs. Evolutionary methods areessentially used as optimization procedures inaerodynamic design. They have an advantage overconventional local search methods, such as gradient-based methods, in finding global optima of multimodalfunctions and handling cases with multiple feasibleregions that are disjoint. They are also findingincreasing use in multiple-objective andmultidisciplinary optimization.

Artificial neural networks have been widelyused in aeronautical engineering. Recent aerodynamicapplications include, for example, flow control,estimation of aerodynamic coefficients, compactfunctional representations of aerodynamic data for rapidinterpolation, grid generation, aerodynamic design andthe interpolation of wind tunnel data.1-9 Neural networkapplications in aeronautics are not limited toaerodynamics. Hajela and Berke10 provide a review of avariety of neural network applications in structuralanalysis and design.

A method for aerodynamic design optimizationthat is based on neural networks is presented in Rai andMadavan11-12,15-16 and Rai17,18. The method offersadvantages over traditional optimization methods. Itprovides a greater level of flexibility than other methodsin dealing with design in the context of both steady andunsteady flows, partial and complete data sets, combinedexperimental and numerical data, inclusion of variousconstraints and rules of thumb, and other issues thatcharacterize the aerodynamic design process. Neuralnetworks provide a natural framework within which a

20th AIAA Applied Aerodynamics Conference24-26 June 2002, St. Louis, Missouri

AIAA 2002-3143

Copyright © 2002 by the American Institute of Aeronautics and Astronautics, Inc. No copyright is asserted in the United States under Title 17, U.S. Code.The U.S. Government has a royalty-free license to exercise all rights under the copyright claimed herein for Governmental purposes.All other rights are reserved by the copyright owner.


succession of numerical solutions of increasing fidelity,incorporating more realistic flow physics, can berepresented and utilized for optimization. Neuralnetworks also offer an excellent framework for multiple-objective and multi-disciplinary design optimization.Simulation tools from various disciplines can beintegrated within this framework and rapid trade-offstudies involving one or many disciplines can beperformed.

Another attractive feature of this neuralnetwork-based design system is that it can efficientlyuse distributed and parallel computing resources. Themethod lends itself to parallelism at many levels. At thecoarsest level, multiple aerodynamic simulations can beperformed simultaneously and independently onmultiple processors. In situations where individualsimulations are computationally intensive, eachsimulation can be partitioned across multipleprocessors. In addition, neural network training can bedistributed over multiple processors to further acceleratethe design process.

In earlier work Rai and Madavan11-12 developeda design optimization method based on artificial neuralnetworks. The advantages of both traditional responsesurface methodology13,14 (RSM) and neural networks areincorporated in this design method by employing astrategy called parameter-based partitioning of the designspace. Starting from the reference design, a sequence ofresponse surfaces based on both neural networks andpolynomial fits are constructed to traverse the designspace in search of an optimal solution. This methodwas used to reconstruct the shape of a turbine airfoilgiven the pressure distribution and some relevant flowand geometry parameters. The shape of the airfoil wasnot known a priori. Instead, it evolved from a simplecurved section of nearly uniform thickness. The optimalairfoil closely matched the shape of the original airfoilthat was used to obtain the pressure distribution. Thisconstituted a "blind" test of the design methodology.

In another design study, the method12 was usedto redesign a generic gas generator turbine (G3 turbine),to improve its unsteady aerodynamic performance.15 TheG3 turbine was originally designed to operate in thehigh-subsonic regime. However, an unsteady analysisshowed very strong interaction effects including anunsteady shock in the axial gap region between thestator and rotor rows. The design optimization methodyielded a modified design that was geometrically veryclose to the reference design but eliminated the unsteadyshock and thus reduced the dynamic loads on theairfoils. The work output of the turbine was unchanged.

This method was also used to enhance the unsteady,aerodynamic characteristics of a transonic turbinestage.16 Design optimization resulted in a weakenedstator trailing edge shock. This in turn resulted insignificant reductions in the dynamic loads on the statorand rotor airfoils and also eliminated unsteady boundarylayer separation on the rotor suction surface. Theimprovements in aerodynamic characteristics wereobtained without a reduction in turbine work output.These results15-16 extend the application domain of theneural net-based design method from steady flowenvironments to unsteady flow environments and furtherdemonstrate the versatility of the method.

One problem with using a neural network torepresent the functional behavior of the objectivefunction in high-dimensional design spaces lies incontrolling its capacity (defined as the maximumnumber of training sets that can be learned withouterror). The generalization capability of neural networkscan be poor in the absence of adequate data. In suchsituations optimizing the capacity of the network iscritical in order to obtain reasonable generalizationcapability. The earlier method12 alleviated this problemby using the neural network to represent the functionalbehavior of the design objective with respect to onlythose variables that resulted in complex variations ofthis function. This required a partitioning of the designvariables into two sets. A simple partitioning was usedto demonstrate the utility of the method.12,15-16

Parameter based partitioning can be used to advantagegiven a good partitioning of the design variables.

The results presented by Rai and Madavan12,15-16

demonstrated both the ability of the earlier designmethodology to solve some complex aerodynamicdesign optimization problems as well as its versatilityin handling different objective functions and constraints.However, the application of this approach to even morecomplex optimization problems such as three-dimensional aerodynamic design, multiple objectivedesign and multidisciplinary optimization requires aconsiderable reduction in both the number ofoptimization steps required to obtain the optimal shapeand the CPU requirements for generating the trainingdata. Reductions in costs can be achieved in severalways. One of them is using response surfaces withimproved generalization capabilities (response surfacesthat are reliable both inside and outside the region ofdesign space containing the training data). A second wayis exploiting synergies between the neural networkmodel and the simulation codes such that thesimulations are obtained rapidly. A third approach


consists of using an appropriate formulation of thedesign objective.

With these objectives in mind, Rai17 proposeda new neural network-based design method that does notuse parameter-based partitioning of the design space.Instead, the neural network is used to model thefunctional dependence of the design objective on all ofthe design variables. Excellent generalization is achievedthrough a powerful training algorithm. Design costs arereduced because of this improved generalizationcapability. Significant reductions in design costs arealso achieved by exploiting synergies between theneural network model and the simulation codes that areused to generate the training data. Additionally, themethod is formulated to facilitate multiple-objectivedesign optimization. This facet of the method can beutilized in some single-objective design optimizationcases to further reduce design costs. The new designmethodology was applied to two-dimensional turbineairfoil design. Design costs for this case were reduced bya factor of 100 compared to the costs reported by Raiand Madavan12.

An enhanced version of this method ispresented in Rai18. The neural networks that are used aremore powerful in their generalization ability than thoseused by Rai17. Results demonstrating this aspect arepresented. Examples include both data with and withoutnoise. Polynomials in one and multiple-dimensions andpolynomials combined with trigonometric functions areused to test the generalization ability of the networks.The design method employing these new techniques isused to design a three-dimensional turbine vane. Thevane evolves from a generic shape to the optimal shapesuch that a given target pressure distribution isobtained.

Genetic and evolutionary algorithms19 have anadvantage over conventional gradient-based searchprocedures in that they are capable of finding globaloptima of multimodal functions (not guaranteed) andhandling cases with multiple feasible regions that aredisjoint. In aerodynamic design genetic algorithms havebeen used in airfoil and wing design20-21 and compressorand turbine airfoil design. They are also findingincreasing use in multiple-objective andmultidisciplinary optimization22. One disadvantage ofgenetic algorithms is that they generally require morefunction evaluations than other conventionaloptimization schemes to obtain the minimum. Ingeneral they are not the preferred method when a purelylocal search of a smooth landscape is adequate.

Here we present an evolutionary search methodand investigate its strengths in the context of some testproblems as well as nozzle and turbine airfoil design.The results of applying a neural network-based designprocedure to the same design problems are alsopresented. The possibility of integrating designprocedures based on neural networks and evolutionaryalgorithms is explored; the objective being a hybridaerodynamic design procedure that has the strengths ofboth methods.

AERODYNAMIC DESIGN USINGEVOLUTIONARY METHODS

In this investigation we use an evolutionarymethod that draws upon ideas from several geneticalgorithms and evolutionary methods. One of them is arelatively new member to the general class ofevolutionary methods called differential evolution23.This method is easy to use and program and it requiresrelatively few user-specified constants. These constantsare easily determined for a wide class of problems. Fine-tuning the constants will off course yield the solutionto the optimization problem at hand more rapidly. Themethod can be efficiently implemented on parallelcomputers and can be used for continuous, discrete andmixed discrete/continuous optimization problems. Itdoes not require the objective function to be continuousand is noise tolerant. Also, the method does not requirethe transformation of continuous variables into binaryintegers.

As with other evolutionary methods andgenetic algorithms, differential evolution is apopulation based method for finding global optima. Thethree main ingredients are mutation, recombination andselection. Much of the power of this method is derivedfrom a very effective mutation operator that is simpleand elegant. Mutations are obtained by computing thedifference between two randomly chosen parametervectors in the population and adding a portion of thisdifference to a third randomly chosen parameter vector toobtain a candidate vector. This has the preferred effect ofautomatically reducing the magnitude of mutation as theoptimization process converges. In addition themagnitude of mutation vectors computed in thismanner, are directionally dependent. For example, in thecase of an objective function in two dimensions whosecontours are approximately elliptical in shape, themutation vectors would generally be longer in thedirection of the major axis and shorter in the direction ofthe minor axis. In other words the mutation operatoradapts to the particular objective function and thisresults in more rapid convergence to the optimal value.The recombination operator adds to the diversity of thepossible candidate vectors for selection and in generalresults in faster convergence to the optimum.


To describe one version of the method23, weconsider the set of parameter vectors at the nthgeneration, X j,n . The subscript j refers to the jth

parameter vector in a population of N parameter vectorsand,

X = x ,x ,....xj,n 1,j,n 2,j,n D,j,n (1)

where xi,j,n corresponds to the parameter value in the

ith dimension in a D-dimensional problem. The initialpopulation is assumed to be randomly distributed withinthe lower and upper bounds specified for eachdimension. The mutation, recombination and selectionoperators are then applied to the population of parametervectors as many times as required. To evolve theparameter vector X j,n we first randomly pick three other

parameter vectors X , Xa,n b,n and Xc,n such that

a b c j≠ ≠ ≠ . A trial vector Y = y , y ,...y 1 2 D is

then defined as

Y = X + F(X - X )a,n b,n c,n (2)

where F is a user specified constant, 0 < F < 1( ) . The

candidate vector Z z ,z ,....z1 2 D= is then defined as

z

y if r C

x if r > Ci

i i

i,j,n i

=≤

(3)

where ri is a uniformly distributed random variable

0 r < 1i≤( ) and C is a user specified constant,

0 < C < 1( ).The final step in the evolution of X j,n

involves the selection process and, for the minimizationof the objective function f(X), is given by

X

Z if f(Z) f(X )

X if f(Z) > f(X )j,n 1

j,n

j,n j,n

+=

≤

(4)

Thus, selection involves a simple replacement of theoriginal parameter vector with the candidate vector if theobjective function decreases by such an action.

Several modifications to the mutation,recombination and selection procedures were made inthis study in order to enforce constraints, facilitatesearches involving multiple feasible regions embeddedin infeasible regions and, enrich the pool of potentialcandidate vectors. Although these modifications weredevised for the method of differential evolution, the

underlying principles can also be used with otherevolutionary strategies and genetic algorithms. Theprocedural modifications will be discussed elsewehere24.

The application of this evolutionary method toaerodynamic design is relatively straightforward. Theaerodynamic shape of interest is first parameterizedusing an appropriate method. The prudent selection ofgeometry parameters is one of the most critical aspectsof any shape optimization procedure. Variations of theaerodynamic shape can be obtained by varying theseparameters. Geometrical constraints imposed for variousreasons, such as structural, aerodynamic (e.g., toeliminate flow separation) should be included in thisparametric representation as much as possible.Additionally, the smallest number of parameters shouldbe used to represent the aerodynamic shape. The secondstep involves the specification of the upper and lowerbounds for the geometry parameters to be used in thesearch process. This step typically involves someknowledge of the aerodynamics involved and constraintssuch as the maximum and minimum thickness of anairfoil (from structural considerations or to preventchoking of the flow in a turbine). The third stepinvolves defining an appropriate objective function (afunction of the geometric parameters) to be minimized.A given engineering objective can be achieved usingdifferent objective functions; some more difficult thanothers to optimize. In some cases the search for anoptimum can be made significantly easier by using theappropriate objective function. The final step involvesusing the evolutionary method to determine the optimalset of geometric parameters.

Aerodynamic shape optimization varies fromsimple unimodal function optimization to multimodalfunction optimization, constrained optimization,optimization in cases where the search space containsdisjoint regions of feasibility, and multiple-objectiveoptimization. The performance of the evolutionarymethod used in this study was first investigated usingtest cases with some of these attributes. These cases arediscussed below.

Unconstrained Optimization of a MultimodalFunction

One of the most desirable attributes ofevolutionary search algorithms is their ability to locatethe global optimum of a multimodal function.Although this is not guaranteed, in many practicalsituations they tend to produce better solutions thanpurely local searches of the parameter space. The firsttest problem was chosen to highlight this particularaspect of the evolutionary method used in this study.


The function to be minimized is two-dimensional and isgiven by

f(x,y) 0.002 n (x a ) + (y - b )n6

n6

n 1

25 1

= + + −[ ]

−

=

−

∑ 1 (5)

The constants an in Eq. 5 are given by

a 32, a 16, a 0, a 16, a 32

a a n 6,7,...251 2 3 4 5

n n mod 5

= − = − = = == =

(6)

and the constants bn are given by

b 32b 16b 0b 16b 32

1,2,..5

6,7,..10

11,12,..15

16,17,..20

21,22,..25

= −= −

===

(7)

The lower and upper bounds for the search are asfollows:

− ≤ ≤65.536 x,y 65.536 (8)

The function given in Eq. 5 is also referred to as DeJong’s fifth function or Shekel’s foxholes23. It has 25minima in the region − ≤ ≤32.0 x,y 32.0. The functionis nearly constant everywhere except near the minima.Figure 1 shows contours of this function in theregion − ≤ ≤40.0 x,y 40.0. The minimum nearest thelower left hand corner of the square is the globalminimum (f 0.998004)≈ .

Twenty parameter vectors were used in thesearch process. Figure 2 shows the reduction in theobjective function with increasing number of functionevaluations (representative of the cost of optimization).This data was obtained as an average over 10optimization runs with different initial parameter vectorpopulations. All the test runs converged to the globaloptimum. Convergence to the optimum was defined bythe criterion f(x,y) 0.998004≤ . The evolutionarymethod used here required 680 steps to converge to theoptimum. In comparison Price and Storn23 required 672steps for convergence.

Constrained Optimization

Many engineering optimization problems areconstrained by equality and inequality constraints thatcan be linear or nonlinear. The second test case waschosen to evaluate the ability of the currentevolutionary method in solving such constrainedoptimization problems. This optimization problemrepresents the design of a gearbox (the Golinski speed

reducer25). It consists of one objective and 11 inequalityconstraints, of which 7 are nonlinear. There are sevenvariables and seven associated upper and lower bounds.The objective is to minimize the function

f(x ,x ,..x ) 0.7854x x (3.3333x 14.9334x

43.0934) 1.508x (x x )

7.477(x x ) 0.7854(x x x x )

1 2 7 1 22

32

3

1 62

72

63

73

4 62

5 72

= +

− − + +

+ + +

(9)

subject to the inequality constraints

27.0 x x x 01 22

3− ≤ (10)

397.5 x x x 01 22

32− ≤ (11)

1.93x x x x 043

2 3 64− ≤ (12)

1.93x x x x 053

2 3 74− ≤ (13)

745.0x x x 16.9 10 110.0x 04 2 3 63/

/

( )( ) + ×

− ≤2 61 2

(14)

745.0x x x 157.5 10 85.0x 02 3 73

52 6

1 2/

/

( )( ) + ×

− ≤ (15)

x x 40.0 02 3 − ≤ (16)

5.0x x 02 1− ≤ (17)

x 12.0x 01 2− ≤ (18)

1.5x x 1.9 06 4− + ≤ (19)

1.1x x 1.9 07 5− + ≤ (20)

The lower and upper bounds for the search are asfollows:

2.6 x 3.61≤ ≤ (21)

0.7 x 0.82≤ ≤ (22)

17 x 283≤ ≤ (23)

7.3 x x 8.34 5≤ ≤, (24)

2.9 x 3.96≤ ≤ (25)

5.0 x 5.57≤ ≤ (26)

Note that x3 is an integer variable but is treated as acontinuous variable. This simple approach worksbecause the optimal value for this variable correspondsto its lower bound and, this lower bound is an integer.


Ten parameter vectors were used in the searchprocess. The method yielded the following values forthe seven variables:

x 3.5000008 1 =

x 0.70000002 =

x 173 =

x 7.30000024 =

x 7.71532515 =

x 3.35021486 =

x 5.28665457 =

The corresponding function value is f 2994.36= and,this compares well with the result of Azarm and Li25 of2994.0. All the constraints are satisfied at the optimallocation. Four of them are active constraints. Figure 3shows the mean convergence rate, which was generatedby averaging the rates obtained in ten different runs ofthe method; each with a different initial population ofparameter vectors. The minimum is obtained to within0.1% in about 600 function evaluations.

Search spaces with multiple feasible regions

In aerodynamic design and other optimizationscenarios the search space may contain multiple regionsof feasibility embedded within infeasible regions. Inaerodynamic shape optimization certain values of thegeometric parameters may result in surface undulationsthat induce undesirable flow separation. A global searchmethod must be able to find the global optimum andperhaps several of the local optima in the feasibleregions and prioritize them. The third test case waschosen to highlight this particular capability of thecurrent evolutionary search method. The existence oflarge infeasible regions may actually hasten the searchprocess because of the corresponding reduction in thevolume of the design space to be searched.

The problem is defined as maximizing the two-dimensional function

f(x,y) x y2 2= + (27)

The search region is defined by the constraints

− ≤ ≤5.0 x,y 5.0 (28)

and the feasible regions are further constrained by

0.1 exp K (x x ) (y y ) 0n n2

n2

n 1

n nmax

− − − + −[ ]{ } ≤=

=

∑ (29)

These constraints (Eqs. 28 and 29) together yieldmultiple regions of feasibility that are disjoint but arecontained within a square. The number of feasibleregions is determined by the parameter nmax, but is notnecessarily equal to nmax because some of themcoalesce to form larger regions of feasibility. Figure 4shows the regions of feasibility obtained for nmax = 42and random choices for xn and yn. Each of theexponential terms (in isolation), in the summationgiven in Eq. 29, yields a circular region. Together theyyield nearly circular regions of feasibility that are eitherseparate or merge with others. The “radii” of thesenearly circular regions are determined by the constantsKn. A value of 1.5 was used to generate the large regionat the center of the square and a value of 20.0 was usedto generate the other regions.

The value of the objective function increasesmonotonically from the center of the square outward.The contours of this function are circles centered at theorigin (center of the square). Every feasible region inFig. 4 has at least two contour circles that are tangentto it; one at the smallest possible radius and one at thelargest possible radius. The function minimum andmaximum, in the given region of feasibility, lie atthese smallest and largest radius values, respectively, atwhich tangency occurs.

Ten parameter vectors were used in the searchprocess. They yielded three different maximum valuesincluding the global maximum. The locations of thesemaxima are indicated by the square symbols in Fig. 4.The number 1 designates the global maximum, and thenumbers 2 and 3 designate the other two local maximain descending order. Interestingly, just ten parametervectors yielded three different maxima. This facet of themethod may be critical in making a design trade-off. Forexample, the global maximum may not be achievablewith current manufacturing processes, and the designermay have to choose one of the local maxima.

AERODYNAMIC DESIGN USING NEURALNETWORKS

The neural network-based aerodynamic designprocedure uses a sequence of response surfaces totraverse the design space in search of the optimalsolution. Neural networks are used to construct theseresponse surfaces. The variation of the objectivefunction with respect to the design variables is modeledusing neural networks. Data for these networks arespecified at the vertices, edges and select interior pointsof multidimensional simplexes. An n-dimensional


simplex is a spatial configuration of n dimensionsdetermined by n+1 equally spaced vertices that lie on ahyper-sphere of unit radius. By this definition a two-dimensional simplex is an equilateral triangle that iscircumscribed by a unit circle. The design optimizationstrategy used here is a variant of RSM13-14. The methodsused to construct network models of the data aredescribed later in this section. The network-basedaerodynamic design procedure used in this study issummarized below:

1. Populate the design space in the vicinity of theinitial design. The initial design serves as the centroidof the first simplex in the optimization process. Asimplex in design space is constructed around thiscentroid and aerodynamic simulations are performedwith airfoil shapes corresponding to the geometryparameters associated with each of the vertices.

2. Train the neural networks to define the responsesurfaces. The geometric parameters that determine theshape of the aerodynamic surface are provided at theinput nodes of the neural networks. The objectivefunction is obtained at the output node. The trainednetworks then constitute the response surface.

In cases with multiple design objectives, the variationof each design objective with respect to the geometryparameters is represented with a different neuralnetwork. Hence the method is formulated for themultiple-objective case. The single-objective situation,for example maximizing the efficiency of a turbinestage, constitutes a small subset of the problems thatcan be solved with the current approach. Using differentneural networks for different design objectives alsopermits the optimization of the network architecture fora given objective. This is of particular importance whenthere are large differences in complexity between designobjectives. Using a single neural network in such casesmay result in large bias error for the complex objectivesand large variance for the simpler objectives.

In cases where the physical location on the aerodynamicsurface is relevant, for example when an airfoil sectionwith a prescribed pressure distribution is sought, theproblem is formulated as a multiple-objective designoptimization case. An objective function is defined ateach surface point at which a target pressure is specified.

3. Search the region of the design space represented bythe response surface. Geometrical and other constraintscan be easily incorporated within this search procedure.In addition, constraints that limit the search procedure to

the volume of the simplex or a user specified “trustregion” can also be incorporated.

4. Relocate the simplex. If the local optimum obtainedin the previous step lies on or outside the boundaries ofthe simplex then this point is chosen as the newcentroid and steps 1-4 are repeated until the searchculminates inside the simplex. However, the processcan be stopped at any time when the design is deemedadequate.

5. Validate the design. As a final step in the process anaerodynamic analysis is performed for the optimaldesign (airfoil shape) to determine the adequacy andquality of the design.

Constructing Response Surfaces With NeuralNetworks

Feed-forward artificial neural networks areessentially nonlinear estimation techniques. One of thedifficulties associated with using them in modeling dataarises from having to employ nonlinear optimizationmethods to obtain the connection weights. The weightvectors are not uniquely defined; many different weightvectors may yield acceptably low training error for agiven set of training data and network architecture.However, this multiplicity of acceptable weight vectorscan be used to advantage.

One could select the neural network (orequivalently the corresponding weight vector) with thesmallest validation error if validation data is available.In aerodynamic design this approach requires additionalaerodynamic simulations. It is reasonable to expect thegeneralization ability of the weight vector selected inthis manner to be superior compared to the rest of theweight vectors. In turn, this superior generalizationability can be expected to result in a reduction in thenumber of optimization steps required to minimize theobjective function. Hence, the use of this method canresult in an overall decrease in design costs in caseswhere the additional computing time used to generatevalidation data is offset by a reduction in the requirednumber of optimization steps.

In the absence of validation data, multipletrained neural networks can be effectively utilized bycreating a hybrid network26-27. The output of the hybridnetwork can be defined as a simple average of theoutputs of all the trained neural networks. It can beshown26 that the sum-of-squares error (SSE) thusobtained (or integrated squared error) in modeling thefunction underlying the data, is a factor of N less than


the average SSE, where N is the number of trainednetworks. The essential assumption that is made toobtain this result is that the errors produced by thedifferent networks are not correlated and that the meanerror is zero. When these conditions are not met theSSE of the hybrid network continues to be less than orequal to the average SSE. However, it is not necessarilyreduced by a factor of N. Note that the networks used inthis ensemble average do not have to possess the samearchitecture or even be trained on the same training set.

A second and more general way of combiningmultiple trained networks is to weight the output ofeach network such that error of the combined output isminimized27. Given the weights α i (i 1,..N),= their

optimal values are obtained by minimizing the function

α αi j ijj 1

N

i 1

N

C==∑∑ (30)

subject to the constraint

α ii 1

N

=∑ =1 (31)

The matrix C in Eq. 30 is the error correlation matrix.Details of this method of creating a hybrid network arediscussed by Perrone and Cooper.27 Unlike the simpleaveraging technique, this method does not explicitlyrequire that the mean error be zero, or, that the networkmodels be mutually independent. However, practicalconsiderations such as maintaining the full rank of C,and imposing the constraint α i ≥ 0 to prevent theextrapolation of data, once again require network modelsthat are not correlated and have zero mean. In many ofthe cases presented in this study, there was littledifference in modeling error whether the simpleaveraging technique or the more general approach givenin Eqs. 30-31 was used. In some cases the more generalensemble approach did yield a better model.

The creation of a hybrid network serves toreduce the variance. Hybridization requires the neuralnetwork training method to yield numerous trainednetwork models that are not correlated and have zeromean. The nonlinear optimization process used to traineach network can be started with different randomweight vectors to accomplish this task. Methods thatimprove the generalization ability of the individualnetworks, such as regularization and networkarchitecture optimization can be embedded at this level.

The generalization ability of hybrid networkswas tested using low-order polynomials and a Gaussianfunction in Rai17. Excellent generalization (both in theregion where data was available and outside of thisregion) was obtained. A feed-forward network with twohidden layers and the more general ensemble method ofcreating the hybrid network (Eqs. 30-31) were used inthese cases. These encouraging results naturally led to astudy of the generalization accuracy that can be obtainedfor higher-order polynomials, polynomials combinedwith other functions, polynomials in multipledimensions, and cases where the training data iscontaminated by noise in Rai18. Some of the resultsobtained are presented below. In the interest of brevity,details such as the number of neurons in each layer etc.are not included here.

Figures 5 and 6 show the generalizationobtained with a hybrid network consisting of ten neuralnetworks for a fifth-order polynomial with six trainingpoints. The full line was obtained with the neuralnetwork and the dashed line was obtained using theexact function. Neural network generalization isexcellent throughout the region –3 ≤ x ≤ 3 althoughtraining data is available only in the region 0 ≤ x ≤ 1.The ability of the hybrid network to extrapolate isevident in this case. Clearly this does not constituteproof that hybridization will always work as well(especially in the extrapolation mode). Note that asimple polynomial fit (fifth-order) would yield perfectgeneralization. However, it is equally important to notethat the network was not supplied with the informationthat the function underlying the training data was afifth-order polynomial.

The neural network generalization shown inFig. 5 is surprisingly accurate even in the extrapolationmode. A natural question to ask at this point is why thenetwork generalization closely approximates theoriginal function used to generate the training data,given that there are an infinite number of smooth curvesthat would fit this data. The answer lies in the fact thatthe given data (6 distinct pairs) when interpolated usinga set of polynomial basis functions (for example theLegendre polynomials) uniquely define the originalcurve, assuming a convention such as interpolating thedata with the lowest-order polynomial. Hence, theneural network will reproduce the original function tothe extent that it can mimic the polynomial basisfunctions. Neural networks can be made to mimic someclasses of functions in a straightforward manner eitherthrough the choice of network connectivity or


preprocessors that process the input data before they arefed to the input nodes.

Figure 7 shows the generalization obtained forthe function y xsin(2 x)= π with eleven training points.

As in the previous case, the generalization obtainedwith the hybrid neural network is excellent in the region− ≤ ≤2 x 3 although training data was supplied only inthe region 0 x 1≤ ≤ . The Gaussian function used byRai17 and the combination of the linear function and asine wave depicted in Fig. 7, demonstrate that theenhanced generalization that is possible with the neuralnetworks (and the corresponding training algorithm)used in this study is not limited to polynomials.

One question that arises in the use of RSM indesign optimization is whether the search process islocal (as in the case of conventional gradient basedsearch methods) or global as in the case of evolutionaryand genetic algorithms. Figure 8 shows thegeneralization obtained with a hybrid network for afourth-order polynomial. This polynomial exhibits aminimum at two different locations. Approaching fromthe left (x 0)< a gradient search algorithm wouldterminate at the local optimum 1 x 2opt< < . On the

other hand, the hybrid network model obtained for thiscase does exhibit two minima (just as the originalfourth-order polynomial) with the global minimumat 4 x 5opt< < . The network model can be searched for

the global minimum using an evolutionary or geneticalgorithm. In situations where function evaluations arecomputationally intensive (as in aerodynamic design)there may be an advantage in first modeling the datausing a hybrid neural network and subsequentlyconducting a global search of the model, instead ofdirectly searching for the global minimum using anevolutionary or genetic algorithm. In any case, Fig. 8indicates the global nature of RSM-based designoptimization. In practice, because of the expenseinvolved in constructing models for large regions inmultidimensional design space, searches for the globalminimum based on RSM may be restricted to locatingand prioritizing minima that are not widely separated.This restriction can be alleviated to some extent byusing multiple response surfaces in the search process.

The presence of noise in data that is used foroptimization can have a profound effect on theperformance of an optimization scheme. Gradient searchmethods are particularly sensitive to noise because ofthe resulting contamination of the gradient and search-direction vectors. Genetic algorithms are robust even inthe presence of noise in the sense that they tend to settle

down to a function minimum. However, this minimummay turn out to be a false minimum generated by thenoise. This can result in convergence to a location farfrom the actual location of the optimum. Optimizationmethods based on RSM have the advantage ofovercoming noise-related problems by fitting a smoothcurve to the noisy data.

Hybrid networks can be used to significantadvantage in cases where the data is corrupted withnoise. Figures 9 and 10 show the generalizationobtained for a cubic polynomial. The training data wasfirst generated using the polynomial and then corruptedwith uniformly distributed noise (20% of the maximumvariation of the polynomial in the region where the datais specified). Good generalization can be obtained forthis case even with simple polynomial fits when manydata points are used. However, the hybrid network yieldsexcellent generalization both in the interpolation andextrapolation modes with just eight training points(only four are required for the noise-free case). Thequality of generalization is dependent on the diversity inthe individual networks that constitute the hybridnetwork, or in other words the degree to which they arenot correlated. The advantages of network hybridizationare clearly evident in this case. However, figures 9-10represent preliminary efforts at obtaining superiorgeneralization for noisy data with hybrid networks.

The examples provided above demonstrate howthe nonlinear nature of the estimation method and theconsequent lack of uniqueness of the weight vector canbe used to advantage in modeling data. Thehybridization procedure that is used relies on being ableto generate several appropriate weight vectors. Inaerodynamic optimization, the ability to extrapolateaccurately in regions far from where the data reside,results in substantial reductions in data requirements andthus significantly lower simulation costs.

AERODYNAMIC SIMULATION METHOD

The method used to perform aerodynamiccomputations is a third-order-accurate, iterative-implicit,upwind-biased scheme that solves the time-dependentEuler and Reynolds-averaged, thin-layer, Navier-Stokesequations. The flow around the turbine airfoil used inthe design studies presented later is computed usingmultiple grids; an inner “O” grid that contains theairfoil and an outer “H” grid that conforms to theexternal boundaries. Details regarding the computationalmethod are presented in Rai.28 This method wasdeveloped to compute flows starting from arbitrary(albeit reasonable) initial conditions. In the current


design scenario, rapid solution convergence can beobtained by exploiting synergies between thesimulation algorithm and the neural network model.Several modifications to the basic method and boundaryconditions were required to obtain aerodynamicsolutions in a cost-effective manner.

AIRFOIL PARAMETERIZATION

Geometry parameterization and prudentselection of design variables are among the most criticalaspects of any shape optimization procedure. Since thisstudy focuses on airfoil design, the ability to representvarious airfoil shapes with a common set of geometricalparameters is essential. Variations of the airfoilgeometry can be obtained by varying these parameters.Geometrical constraints imposed for various reasons,such as structural, aerodynamic (e.g., to eliminate flowseparation), etc., should be included in this parametricrepresentation as much as possible. Additionally, thesmallest number of parameters should be used torepresent the family of airfoils.

The method used here for parameterization ofthe airfoil shape is described in detail by Rai andMadavan12 and is reviewed here for completeness. Figure11 illustrates the method for a generic airfoil. Somesalient features of the method are noted below:

1. The leading edge is constructed using two differentellipses, one for the upper surface and one for the lowersurface. The eccentricity of the upper ellipse and thesemi-minor axes of both ellipses are specified asgeometric parameters (eu, tu, and tl), respectively. All

other related parameters can be determined analytically.The major axes of both ellipses are aligned with thetangent to the camber line at the leading edge. Thistangent is initially aligned with the inlet flow but isallowed to rotate as the design proceeds. The angles αuand α l determine the extent of the region in which the

leading edge is determined by these ellipses. The twoellipses meet in a slope continuous manner.

2. The trailing edge can also be constructed in a similarmanner with the major axes of the ellipses aligned withthe tangent to the camber line at the trailing edge.However, in this study the trailing edge was definedusing a single circle. The angles βu and βl determine the

extent of the region in which the trailing edge isrepresented by this circle.

3. The region of the upper surface between the upperleading edge ellipse and the trailing edge circle is defined

using a tension spline. This tension spline meets theleading edge ellipse and the trailing edge circle in aslope continuous manner. Additional control points forthe tension spline, for example the point (x ,y )u u in

Fig. 11, are introduced as necessary. These pointsprovide better control over the shape of the uppersurface. The lower surface of the airfoil between thelower leading edge ellipse and the trailing edge circle isobtained in a similar manner.

Thirteen geometric parameters were used to define theturbine airfoil used in this study. These parameters arelisted below:

1. Leading and trailing edge airfoil metal angles (2parameters).

2. Eccentricity of the upper leading edge ellipse (1parameter).

3. Angles defining the extent of the leading edgeellipses (2 parameters).

4. Semi-minor axes values at the leading edge (2parameters).

5. Angles defining the extent of the trailing edge circle(2 parameters).

6. Airfoil stagger angle (1 parameter).

7. Airfoil y-coordinate values at about 50% chord on theupper and lower surfaces (2 parameters).

8. Airfoil y-coordinate value at about 75% chord on theupper surface (1 parameter).

This method of generating the airfoil surfaceprovided a smooth shape transition from a curvedconstant thickness section to the optimal airfoil. Theintermediate airfoil shapes required by the optimizationprocedure were generated by smoothly varying some orall of these 13 parameters.

AERODYNAMIC DESIGN OPTIMIZATIONRESULTS

The aerodynamic design procedures based onboth neural networks and the evolutionary method areused here to evolve a two-dimensional turbine airfoilfrom a simple curved section of nearly uniformthickness. This is the same design study that wasperformed by Rai17. The target pressure distributionwas provided by Pratt & Whitney (P&W, Private


Communication, F. Huber, 1997). This pressuredistribution was computed using the midspan airfoilsection of a turbine vane from a modern jet engine. Theinlet total pressure and temperature, inlet and exit gasangles, the exit Mach number, the axial chord, thetrailing edge circle radius, the midspan radius and thenumber of vanes in the row were provided by P&W.Nozzle shape optimization results using the two designoptimization methods are also presented here. This caseillustrates the need for combining a powerful globaloptimization method (albeit a slower method) with arapid local optimizer.

Airfoil Design Optimization Using NeuralNetworks

The network-based design method uses asequence of response surfaces to enable a search of thedesign space thus permitting the use of initial designsthat are far from the optimal design. To illustrate thiscapability the shape of the airfoil was initially chosento be a nearly constant thickness, curved section. Theinlet and exit metal angles for this curved section wereset equal to the corresponding gas angles but allowed tochange during the optimization process. This airfoilsection is shown in Fig. 12 marked as A.

It is possible to solve this problem in twodifferent ways, either as a single- or as a multiple-objective case. The multiple-objective approach workswell even with only the minimum number of datapoints per simplex (n + 1 data points for an n-dimensional simplex). This is because lower-ordermodels of pressure, as a function of the geometryvariables, are rendered into corresponding higher-ordermodels of the objective function. The multiple-objective approach is used here to evolve the airfoil. Inthis approach the pressure at each point on the airfoil (atwhich the target pressure is specified) is modeled usinga different hybrid neural network. In all, the pressurewas specified at 45 different airfoil surface locationsthus resulting in as many objective functions and hybridneural networks. The different objective functions werethen weighted equally and combined to obtain a singlesum-of-squares objective function given by

SSE = (P - p )i i2

i=1

imax

∑ (32)

where, Pi is the target pressure, pi is the pressure at the

same axial location for the airfoils generated during theoptimization process, and imax is the total number of

airfoil surface points at which the target pressure isspecified. This approach permits the use of differentweights for the individual objective functions. Thus, forexample, one could prescribe larger weights for theleading edge region than in the trailing edge region.

The network-based design method was thenused to obtain the optimal geometry. Only theminimum number of data points, n + 1 data points foran n-dimensional simplex, were used in this designstudy. Figure 12 shows the evolution of the airfoilshape and also compares the corresponding pressuredistributions with the target pressure distribution. Thesurface pressure and the axial distance along the airfoilare normalized using the inlet total pressure and theairfoil axial chord, respectively. Aerodynamicsimulations of different fidelities were used at differentstages of the design process. The airfoil shape wasoptimized using solutions to the Euler equations untilthe airfoil denoted C in Fig. 12 was obtained. Solutionsto the Reynolds-averaged Navier-Stokes equations werethen used to achieve the final design shown as airfoil Din Fig. 12. Significant reductions in simulation costscan be achieved by using solutions of lower fidelity inthe initial stages of the design process.

Design optimization was performed with only6 variables (t , t , , , and )u l u l u lα α β β until the airfoil

marked C was obtained. Thereafter all 13 of thevariables mentioned before were used in the designprocess. Significant reductions in design costs can beachieved by using a subset of the geometry parametersin the early stages of design. Airfoils B, C and D wereobtained after 2, 4 and 6 optimization steps,respectively. A smooth transition of the curved sectionto the optimal airfoil geometry is seen in Fig. 12. Theagreement between the target pressure distribution andthe computed pressure distribution, obtained at the endof the optimization process, is good (airfoil D).

Airfoil Design Optimization Using theEvolutionary Method

In the evolutionary approach to designoptimization, an initial airfoil shape does not have to bespecified. Instead the lower and upper bounds, for eachof the geometry variables, are specified for the searchprocess. These upper and lower bounds together define ahypercube in design space. The parameter valuesemployed to generate the nearly constant thicknesscurved section, marked as airfoil A in Fig. 12, wereused to define the lower bounds for the search. In otherwords, these parameter values represent one vertex of


the hypercube in design space. The upper bounds werechosen to be sufficiently large so that the optimalgeometry is an interior point of the search space. Theoptimal geometry is sufficiently far away from theupper bounds to allow the intermediate airfoils,generated during the evolutionary process, to approachthe optimum from all directions. The upper and lowerbounds were chosen in this manner to create a searchspace as large as that encountered in the network baseddesign process described earlier.

The sum-of-squares error defined in Eq. 32 wasused as the objective function to be minimized by theevolutionary method. Unlike the network basedapproach where the surface pressures at the targetlocations were first modeled as a function of thegeometry parameters, and the SSE was then constructedusing Eq. 32, here the SSE for each airfoil generated inthe evolutionary process is directly computed as a singlevalue. Additionally, design optimization was performedwith only 6 variables (t , t , , , and )u l u l u lα α β β and, the

Euler equations were used to obtain the pressure data forall the airfoils. Thus, the optimal airfoil obtained inthis process should be compared with airfoil C of theprevious process and not airfoil D.

The evolutionary design method was applied toobtain the optimal airfoil shape. Twenty parametervectors were used in this process. The search spannedtwenty generations of airfoils. The best airfoil andcorresponding flow field at each generation were stored.This ‘best’ flow field was used to initialize the flowfields for all the airfoils used in the next generation. Theshape of the optimal airfoil obtained at the end oftwenty generations was very close to that of airfoil C.The corresponding pressure distributions were also inclose agreement.

Figure 13 shows the variation of the meansquare error as a function of the cumulative computingtime (on a single processor CRAY-C90). Thiscumulative value includes the CPU time used togenerate the grids, perform the aerodynamicsimulations, train the neural networks (for the network-based design), and search for the optimal airfoil. Figure13 shows design costs for both network-based design aswell as evolutionary design. In the case of network-based design, only the cost incurred for the ‘Euler’optimization is included. The network-based approach isfaster than the evolutionary approach by about a factor8.5. In a similar study involving a low-speed turbineairfoil (the midspan section of the airfoil used by Rai18)the evolutionary method’s computational requirements

were larger by a factor of 20. The results of Fig. 13 alsodemonstrate that the network-based approach is notconfined to optimization studies in which the changesin shape are small; the initial value of SSE is almost20 times larger than the largest value encountered in theevolutionary search.

The experience of researchers in manydisciplines is that the evolutionary method is thepreferred choice in cases where the objective function ismultimodal, or the feasible region is disjoint. Theairfoil optimization case used here does not seem toexhibit these characteristics. Hence, the strengths of theevolutionary method have not been brought forth in thisapplication. It should be noted that it is possible to usethe network-based approach in some cases that exhibitmultimodality or multiple feasible regions. The lowercosts involved with this approach indicate that a numberof network-based design processes, each starting with adifferent airfoil shape can be initiated to hopefullycapture the global minimum. Ultimately, a goodaerodynamic design system will have to include a globalsearch strategy (evolutionary methods) as well as othersearch strategies that are perhaps not as global in naturebut much more rapid (network-based methods). Adesigner would resort to the evolutionary strategy forthe more revolutionary applications (searching largedesign spaces for the global optimum) and to thenetwork-based approach in cases where such a globalsearch is unnecessary but a rapid design procedure isrequired.

Nozzle Design Optimization Using NeuralNetworks And Evolutionary Methods

The second aerodynamic design optimizationstudy consisted of shape optimization of a supersonicnozzle. The nozzle pressure for the area distribution

A(x) 0.35(2.0 x ), 0 x 1.02= + ≤ ≤ (33)

was obtained from a simple one-dimensional analysis(inlet Mach number of 2.0). The computed pressurevalues at 21 equally spaced points in theregion 0 x 1.0≤ ≤ were provided to the design procedureas target values. The objective was to recover the nozzlecross-sectional area at these axial locations,A (i = 1, 2,......21).i This constitutes a 21 dimensionaloptimization problem for the evolutionary method.Unlike the airfoil optimization case, the shape of thenozzle was not parameterized to reduce the number ofgeometry parameters.

The optimization problem as stated above canbe solved rapidly in just a few generations with the


evolutionary method. Here the problem is modified tomake it considerably more difficult to solve in order totest the method’s ability to locate the global optimumin the presence of disjoint regions of feasibility. Insteadof searching for optimal values of Ai within ahypercube, we define

A r sin(4 r )i i i= π (34)

and search for the optimal values of ri in the hypercubedefined by

0.25 r 1.25, 1 i 21i≤ ≤ ≤ ≤ (35)

The optimal values of Ai lie between 0.70 and 1.05 (Eq.33) and, consequently the optimal values of ri liebetween 1.0 and 1.25 (Eqs. 34 and 35). The regionsdefined by the inequalities, 0.25 r 0.50i≤ ≤ and0.75 r 1.00i≤ ≤ are infeasible because they yield non-positive values of area. Equation 34 yields positivevalues of area in the region 0.50 r 0.75i≤ ≤ , howeverthis region does not contain the global optimum.Figure 14 shows the feasible regions in the r r1 2− planefor a case with only two variables A1 and A2. There arefour feasible regions and the one closest to the top righthand corner contains the global optimum. The 21

dimensional hypercube considered here has 221 feasibleregions with the global optimum contained in one ofthem. The feasible region containing the globaloptimum occupies only a miniscule fraction of the total

search volume (1/4 )21 .

The evolutionary procedure was applied to thisoptimization problem. The objective function wasdefined as defined as in Eq. 32, with imax 21= . Sixtyparameter vectors were used in the search process.Figure 15 shows the optimal (as obtained by theevolutionary method) and the exact area distributions inthe axial direction. The two are in close agreement witheach other demonstrating the success in finding theglobal optimum in this case with 221 feasible regions.Figure 16 shows the corresponding optimal and exactpressure distributions in the nozzle.

As indicated earlier there are considerableadvantages to developing a hybrid aerodynamic designprocess that possesses the best attributes of both theevolutionary method and the neural-network basedmethod. In the current nozzle-design case, it would bedifficult for the user to provide the neural network basedmethod with an initial geometry that lies in the feasibleregion containing the global optimum. Here we use theevolutionary method to obtain such an initial geometryfor the neural network based method and thensubsequently use the rapid convergence properties of thelatter method to search for the minimum. In general the

transfer of control from one method to another will bebased on heuristics (such as an order of magnitudereduction in the objective function value) and may needto be repeated a couple of times at different stages of theevolutionary process. The overall computational costmay still be a fraction of the cost associated with apurely evolutionary approach.

In this study, for the purpose of illustration, theparameter vector corresponding to the lowest value ofthe objective function at each generation is identified.This “best” parameter vector is used to generate theinitial nozzle design when it first arrives in the feasibleregion containing the global optimum. Obviously thisapproach is not feasible in the general case. It is usedhere only to depict the best-case scenario where thetransfer of control is optimal. A heuristic method suchas picking the best parameter vector after a certainnumber of generations, or after every order-of-magnitudereduction in the objective function, would transfercontrol later in the evolutionary process than the currentoptimal transfer of control. However, such a “heuristics-based’” transfer of control would not require informationregarding the position of the best parameter vectorrelative to the feasible region containing the globaloptimum.

Figures 15 and 16 show the initial geometryand corresponding pressure distribution supplied to theneural network based system. This initial geometry wasthen transformed into the optimal geometry using theneural network based method. Figure 17 is a plot of theconvergence rate and shows the variation of theobjective function with the number of nozzle flowsolutions used in the optimization process. The purelyevolutionary method required 60000 functionevaluations to reduce the objective function by about 7orders of magnitude. The hybrid method with optimaltransfer of control requires about 20% as many functionevaluations. The rapid convergence obtained with theneural network based scheme is particularly noteworthy.Only 185 nozzle flow solutions were required by thisscheme to reduce the value of the objective functionfrom approximately 1.0 to 4×10-7. This indicates thatthe evolutionary process can be tapped several times fora solution that lies within the feasible region containingthe global optimum without incurring a large penalty.

Figure 18 shows the results of tapping theevolutionary process for the best parameter vector after100 and 500 generations (heuristic approach). Theneural network based approach converges to a localoptimum when it is initialized with the best parametervector obtained at the end of 100 generations. This isbecause the initial design supplied to the neural-networkbased method is in a feasible region that does notcontain the global optimum and, consequently the sum-of-squares error quickly asymptotes to a rather large


value. The evolutionary process yields an initial designthat lies in the feasible region containing the globaloptimum when it is tapped after 500 generations. Inthis case the neural network based design system rapidlyyields the global optimum (92 nozzle flow solutions fora 10 ten-order of magnitude reduction in SSE). Thusthe hybrid method halves the number of flow solutionsrequired for design optimization. The hybrid approach asdescribed above is also applicable in a similar manner tocases where the function is multimodal.

SUMMARY

In the past few years genetic algorithms andneural networks have found increasing use in manydisciplines of aeronautical engineering. In particularthere have been several applications in aerodynamicdesign. Both these approaches have their relativeadvantages and disadvantages. Genetic algorithms andevolutionary strategies are population-based approachesto optimization. They are capable of providing a globalsearch of the region of interest and are hence very usefulin finding global minima of multimodal functions andin solving optimization problems where the constraintscreate multiple feasible regions that are disjoint.However they are almost always slower than methodsthat use local information such as RSM and gradientbased methods. On the other hand, these latter methodsare not the preferred methods for multimodal functionsand disjoint feasible regions. Although in some casesthese methods can be used for such applications byemploying multiple searches, each starting at a differentfeasible point in the search region.

In this paper a new evolutionary strategy thatis based on the method of differential evolution andother population-based search algorithms was used tosolve problems in aerodynamic design optimization.The method was first tested on model optimizationproblems that are constrained and exhibit multiplefeasible regions and, are multimodal. The methodperformed well in all these cases. It was then used inturbine airfoil shape optimization and found to be muchslower than the network-based design procedure. Thesecond aerodynamic design problem consisted of nozzleshape optimization in which the region of interestcontained 221 disjoint feasible regions; a situation inwhich the network-based approach is not feasible sinceit requires the initial design to be located in the feasibleregion containing the global optimum. Although theevolutionary method located the global optimumwithout requiring any fine-tuning, it was slower than acombined approach in which the problem was firsttackled with the evolutionary method and then

converged using the network based approach. These twoaerodynamic design problems indicate the need for ahybrid aerodynamic design procedure that has thestrengths of both methods. Current research is focusedon combining them more effectively.

ACKNOWLEDGEMENTS

The author would like to thank Dr. J.Townsend of NASA Langley Research Center forinformation regarding the Golinski speed reducer and,Dr. N. K. Madavan of NASA Ames Research Center forreferring him to the article by Price and Storn23.

REFERENCES

1. Fan, X., Herbert, T., and Haritonidis, J. H.,“Transition control with Neural Networks,” AIAAPaper No. 95-0674, AIAA 33rd Aerospace SciencesMeeting, Reno, Nevada, January 9-12, 1995.

2. Kawthar-Ali, M. H., and Acharya, M., “ArtificialNeural Networks for Suppression of the Dynamic StallVortex Over Pitching Airfoils,” AIAA Paper No. 96-0540, AIAA 34th Aerospace Sciences Meeting, Reno,Nevada, January 15-18, 1996.

3. Faller, W. E., and Schreck, S. J., “Unsteady FluidMechanics Applications of Neural Networks,” AIAAPaper No. 95-0529, AIAA 33rd Aerospace SciencesMeeting, Reno, Nevada, January 9-12, 1995.

4. Rodi, P. E., “Three-Dimensional Hyperbolic GridGeneration Using Neural Network ControlledGoverning Equations,” AIAA Paper No. 96-0028,AIAA 34th Aerospace Sciences Meeting, Reno, Nevada,January 15-18, 1996.

5. Norgaard, M., Jorgensen, C. C., and Ross, J. C.,“Neural Network Prediction of New Aircraft DesignCoefficients,” NASA TM 112197, May 1997.

6. Meade, A. J., “An Application of Artificial NeuralNetworks to Experimental Data Approximation,” AIAAPaper No. 93-0408, AIAA 31st Aerospace SciencesMeeting, Reno, Nevada, January 11-14, 1993.

7. Huang, S. Y., Miller, L. S., and Steck, J. E., “AnExploratory Application of Neural Networks to AirfoilDesign,” AIAA Paper No. 94-501, AIAA 32ndAerospace Sciences Meeting, Reno, Nevada, January10-13, 1994.


8. Sanz, J. M., “Development of a Neural NetworkDesign System for Advanced Turbo-Engines,” FourthU.S. National Congress on Computational Mechanics,San Francisco, California, August 7-8, 1997.

9. Greenman, R., and Roth, K., “High-LiftOptimization Design Using Neural Networks on aMulti-Element Airfoil,” ASME Paper No.DETC98/CIE-6006, 18th ASME Computers inEngineering Conference, Atlanta, Georgia, September13-16, 1998.

10. Hajela, P., and Berke, L., "Neural Networks inStructural Analysis and Design: An Overview," AIAAPaper No. 92-4805-CP, 4th AIAA/USAF/NASA/OAISymposium on Multidisciplinary Analysis andOptimization," Cleveland, Ohio, September 21-23,1992.

11. Rai, M. M., and Madavan, N. K., "Application ofArtificial Neural Networks to the Design ofTurbomachinery Airfoils," AIAA Journal of Propulsionand Power, Volume 17, No. 1, January 2001, pp. 176-183.

12. Rai, M. M., and Madavan, N. K., "AerodynamicDesign Using Neural Networks," AIAA Journal,Volume 38, No.1, January 2000, Pages 173-182.

13. Myers, R. H., and Montgomery, D. C., “ResponseSurface Methodology: Process and ProductOptimization Using Designed Experiments,” JohnWiley and Sons, New York, 1995.

14. Montgomery, D. C., “Design and Analysis ofExperiments,” John Wiley and Sons, New York, 1997.

15. Madavan, N. K., Rai, M. M., and Huber, F. W.,"Redesigning Gas Generator Turbines for ImprovedUnsteady Aerodynamic Performance Using NeuralNetworks," AIAA Journal of Propulsion and Power,Volume 17, No. 3, May-June 2001, pp. 669-677.

16. Rai, M. M., Madavan, N. K., and Huber, F. W.,“Improving the Unsteady Aerodynamic Performance ofTransonic Turbines Using Neural Networks,” AIAAPaper No. 2000-0169, AIAA 38th Aerospace SciencesMeeting, Reno, Nevada, Jan. 10-13, 2000.

17. Rai, M. M., “A Rapid Aerodynamic DesignProcedure Based on Artificial Neural Networks,” AIAAPaper No. 2001-0315, AIAA 39th Aerospace SciencesMeeting, Reno, Nevada, Jan. 8-11, 2001.

18. Rai, M. M., “Three-Dimensional AerodynamicDesign Using Artificial Neural Networks,” AIAA PaperNo. 2002-0987, AIAA 40th Aerospace SciencesMeeting, Reno, Nevada, January 14-17, 2002.

19. Goldberg, D. E., Genetic algorithms in Search,Optimization and Machine Learning,” Addison-Wesley,1989.

20. Obayashi, S., and Tsukahara, T., “Comparison ofOptimization Algorithms for Aerodynamic ShapeOptimization,” AIAA Journal, Vol. 35, No. 8, August1997, pp. 1413-1415.

21. Holst, T. L., and Pulliam, T. H., “AerodynamicShape Optimization Using a Real Number EncodedGenetic Algorithm,” AIAA 19th Applied AerodynamicsConference.

22. Obayashi, S., and Yamaguchi, Y., “MultiobjectiveGenetic Algorithm for Multidisplinary Design ofTransonic Wing Platform,” AIAA Journal of Aircraft ,Vol. 34, No. 5, 1997, pp. 690-693.

23. Price, K., and Storn, N., “Differential Evolution,”Dr. Dobb’s Journal, April 1997, pp. 18-24.

24. Rai, M. M., “Constraint Satisfaction inEvolutionary and Genetic Algorithms,” in preparation.

25. Azarm S., and Li, W. C., “Multi-Level DesignOptimization Using Global Monotonicity Analysis,”ASME Journal of Mechanisms, Transmissions, andAutomation in Design, Vol. 111, pp. 259-263, June1989.

26. Perrone, M. P., “General Averaging Results forConvex Optimization, Proceedings of the 1993Connectionist Models Summer School, M. C. Mozeret. Al. (Eds.), pp. 364-371.

27. Perrone, M. P., and Cooper, L. N., “WhenNetworks Disagree: Ensemble Methods for HybridNeural Networks,” Artificial Neural Networks forSpeech and Vision, R. J. Mammone (Ed.), 1993, pp.126-142.

28. Rai, M. M., "Navier-Stokes Simulations of Rotor-Stator Interaction Using Patched and Overlaid Grids,"AIAA Journal of Propulsion and Power, Vol. 3, No. 5,September-October 1987, pp. 387-396.


Location of global minimumFig. 1. Contours of De Jong’s fifth function (Shekel’s foxholes).

0 200 400 600 800 1000

-1

0

1

2

3

10

10

10

10

10

Number of Function Evaluations

Fun

ctio

n V

alue

Convergence history averaged over ten optimization runs

Fig. 2. Convergence history for De Jong’s fifth function.


0 500 1000 1500 20002950

3000

3050

3100

3150

3200

3250

3300

Number of Function Evaluations

Fun

ctio

n V

alue

Convergence history averaged over ten optimization runs

Fig.3. Convergence history for Golinsky’s speed reducer problem.

1

2

3

Fig. 4. Disjoint regions of feasibility with prioritized maxima.


Training data

Neural network generalization

y = x (1 - x

-2 -1 0 1 2 3x-4

-2

0

2

4

6

y2 )(4 - x 2)

Fig. 5. Neural network generalization obtained for a fifth-order polynomial (close range).

Training data


-3 -2 -1 0 1 2 3-120

-90

-60

-30

0

30

60

90

120

x

y

y = x (1 - x2 )(4 - x 2)

Fig. 6. Neural network generalization obtained for a fifth-order polynomial (extendedrange).


Training data


y

-2 -1 0 1 2 3-3

-2

-1

0

1

2

3

y = xsin(2πx)

xFig. 7. Neural network generalization obtained for the function y=xsin(2πx).

Training data


y = (x - 1)(x - 2)(x - 3)(x - 5)

-1 0 1 2 3 4 5-10

0

10

20

30

40

x

y

Fig. 8. Neural network generalization obtained for a fourth-order polynomial.


Training data


x

y

-1.2 -0.8 -0.4 0.0 0.4 0.8 1.2-0.5

-0.3

-0.1

0.1

0.3

0.5

y = x (1 - x2 )

Fig. 9. Neural network generalization obtained for a third-order polynomial with addednoise (close range).

Training data


y = x (1 - x2 )

-3 -2 -1 0 1 2 3-30

-20

-10

0

10

20

30

x

y

Fig. 10. Neural network generalization obtained for a third-order polynomial with addednoise (extended range).


αu

βu

αl

βl

xu uy,

xl ly,

Fig. 11. Typical airfoil geometry showing defining angles and control points.


0.0 0.2 0.4 0.6 0.8 1.0

1.0

0.0 0.2 0.4 0.6 0.8 1.0

1.0

0.0 0.2 0.4 0.6 0.8 1.0

1.0

0.0 0.2 0.4 0.6 0.8 1.0

1.0

x/c

x/c

x/c

x/c

p

p

p

p

A

A

B

B

C

C

D

D

Neural net-based design

P&W target design

Fig. 12. Transformation of the airfoil geometry from a curved plate to the optimal shape: A, initial design;B, midway through Euler optimization; C, start of Navier-Stokes design; and D, final design.


0 2500 5000 7500 10000

-4

-3

10

10

CPU Time (seconds)

Mea

n S

quar

e E

rror

Convergence history for neural network based design

Convergence history for design based on evolutionary method

Fig.13. Comparison of airfoil design costs obtained with optimization procedures based on neural networksand evolutionary methods.

Feasible Region 1

Feasible Region 2

Feasible Region 3

Feasible Region 4

r1

(0.25,0.25) (1.25,0.25)

(0.25,1.25) (1.25,1.25)Region containing global minimum

r 2

Fig. 14. Search region in two-dimensions showing feasible regions.


0.0 0.2 0.4 0.6 0.8 1.00.4

0.6

0.8

1.0

1.2

Exact nozzle geometry

Optimal nozzle geometry

Initial nozzle geometry (for NN-based design)

x

A(x

)

Fig. 15. Variation of the nozzle cross-sectional area in the axial direction.

0.0 0.2 0.4 0.6 0.8 1.00.25

0.50

0.75

1.00

1.25

1.50

Exact pressure distribution

Optimal pressure distribution

Initial pressure distribution (for NN-based design)

x

p / p

inle

t

Fig. 16. Nozzle pressure distribution in the axial direction.


0 20000 40000 60000

-7

-6

-5

-4

-3

-2

-1

0

1

10

10

10

10

10

10

10

10

10

Convergence history for neural network based design (initial nozzle geometry obtained from evolutionary method,optimal transfer of control)


Sum

of s

quar

ed e

rror

Number of nozzle flow solutions

Fig. 17. Convergence history for the nozzle design optimization study (evolutionary method andhybrid method with optimal transfer of control).

0 20000 40000 60000

-7

-6

-5

-4

-3

-2

-1

0

1

10

10

10

10

10

10

10

10

10

Sum

of s

quar

ed e

rror

Number of nozzle flow solutions


Convergence history for neural network based design (prematuretransfer of control)

Convergence history for neural network based design (suboptimal transfer of control)

Fig. 18. Convergence history for the nozzle design optimization study (evolutionary method andhybrid method with premature and sub-optimal transfer of control).

Date post:	10-Dec-2016
Category:	Documents
Upload:	man
View:	213 times
Download:	0 times

[American Institute of Aeronautics and Astronautics 20th AIAA Applied Aerodynamics Conference - St....

Documents