to complex multidisciplinary systems can be prohibitivewhen the complexity of the simulation codes is large. In-creasingly, response surface approximations(RSAs), andspecifically quadratic approximations, are being inte-grated with nonlinear optimizers in order to reduce theCPU time required for the optimization of complex mul-tidisciplinary systems. RSAs provide a computationallyinexpensive lower fidelity representation of the systemperformance. Thecurse of dimensionalityis a majordrawback in the implementation of these approximationsas the amount of required data grows quadratically withthe number of design variables.
In this paper a novel technique to reduce the magni-tude of the sampling toO(n) is presented. The techniqueuses prior information to approximate the eigenvectorsof the Hessian matrix and only requires the eigenvalues
∗Graduate Research Assistant, Student Member AIAA.†Associate Professor, Associate Fellow AIAA.‡Professor.
Copyright2001 by Victor M. Perez and John E. Renaud. Publishedby the American Institute of Aeronautics and Astronautics, Inc. withpermission.
to be approximated by response surface techniques. Thetechnique is implemented in a sequential approximateoptimization algorithm and applied to engineering prob-lems of variable size and characteristics. Results demon-strate that a full order reduction can be accomplishedwithout compromising the performance of the algorithm.
1 IntroductionTwo trends have emerged when integrating RSAs
within nonlinear optimization tools: 1.) the use of globalapproximations where a RSA of the entire design spaceis developed and 2.) the use of local approximations,where RSAs are built within a local region around thecurrent design. In general a single optimization is per-formed when employing global approximations. Thecost of developing a good global response surface is ob-viously higher than for local responses as a more com-plex model is required to mimic the system. When usinglocal response surfaces, a sequential approximate opti-mization (SAO) methodology can be used. In SAO thedesign space is sampled around each design iterate togenerate the data base required for constructing a loworder polynomial using regression analysis.
The authors have investigated two different ap-proaches for design sampling in SAO frameworks. Thefirst is an optimization based sampling, which hasroots in the original Concurrent SubSpace Optimization(CSSO) algorithm of Sobieszczanski-Sobieski (1988). Itwas later modified for response surface approximate op-timization in Renaud and Gabriele (1993, 1994) and Wu-jek et al. (1997) and expanded to a formal frameworkfor trust region model management in Rodr´ıguezet al.(1998b). In this approach, each of the disciplines per-form an optimization subject to move limits. The re-quired inputs from other systems are computed by lin-ear approximations. The design points visited throughthe subspace optimizations are stored and serve as thedatabase for the RSA construction. The other approach,is a statistically based sampling using design of exper-iments (DOE) arrays as reported in Rodr´ıguez et al.(1998a, 2001). At each SAO iteration, a set of de-sign points is selected for sampling using a DOE ar-ray. The design points are evaluated using the localdisciplinary design tools, where linear approximationsare used for the non-local input states. The resultingdatabase is used to build a RSA. Many other researchstudies have combined DOE techniques and RSA for op-
Rodrıguez et al. (2001) performed a comparisonbetween the optimization based data generation (RS-CSSO) and a statistical based DOE approach using or-thogonal arrays (OAs). Results of that study show thatwhile low strength orthogonal arrays seem to performwell compared to the RS-CSSO approach, RS-CSSO isstill more robust in driving the optimization. An attemptto overcome the natural advantages of the optimizationbased sampling was investigated in P´erezet al. (2000).In that study the DOE based sampling strategy was mod-ified by projecting the orthogonal arrays onto the lin-earized descent feasible region. Results indicate thatthere is evidence that the RSA constructed with such adatabase, provides a better approximation of the systemwhen constrained optimization is performed.
1.1 Quadratic approximations in optimization
Quadratic approximations are an important compo-nent in nonlinear programming. Almost all gradientbased optimization algorithms use some type of secondorder information. Due to the high cost of computingactual Hessian information at each design point, opti-mization algorithms use approximations to the Hessianmatrix. These are point approximations, i. e. expan-sions of the Hessian about the current design point. Themost common type of approximations are Hessian up-dates, in which the optimizer uses zero and first orderinformation to update the value of the Hessian matrixto match at least the previous design point. Examplesof this type of update formulas are the BFGS (namedafter it developers Broyden-Fletcher-Goldfarb-Shanno),the DFP (Davidon-Fletcher-Powell) formula and the SR1(symmetric-rank one) update. An alternate approach tocompute quadratic approximations has been proposed byCanfield (2001) in which zero and first order informationfrom the current and all previous design points is used tocompute the approximation.
The quadratic RSA used by Renaud and Gabriele(1994); Wujek and Renaud (1998a,b); Rodr´ıguezet al.(1998a,b, 2001); P´erezet al. (2000); Perez and Renaud(2000) and P´erezet al. (2001) uses a different approach.While the zero and first order terms are point informa-tion, the second order matrix is computed by samplinga local region around the current design point. TheRSA constructed in this way, provides zero and first or-der matching at the current design which is importantfor convergence purposes. The second order response
2American Institute of Aeronautics and Astronautics
is influenced by designs sampled accross the samplingregion, which allows the algorithm to take larger stepsduring optimization. When the algorithm gets closer tothe minimum, a reduction of the sampling region assuresthat the second order information will get closer to thetrue Hessian. The second order information constructedin this way is called a Hessian-RS or simply Hessian forthe purpose of this paper. The number of independentcoefficients in the Hessian matrix isn(n+1)/2.
The O(n2) number of independent coefficients re-quired is a drawback for application of this technique tolarge optimization problems. To partially alleviate thisconcern, designers can sample the data in parallel, by de-coupling the subsystems and avoiding an expensive fullycoupled analysis. This creates data of so calledvariablefidelity.
1.2 Variable fidelity sampling
In dealing with multidisciplinary problems, a singlesystem evaluation may consist of tens or hundreds ofcalls to the individual discipline simulations or contribut-ing analysis (CAs). The use of SAO is justified by a re-duction in the cost of the optimization, (i. e. the numberof CA calls). Therefore, sampling a full system analysisto construct RSAs should be avoided. A better approachis to sample individual single disciplines and approxi-mate the required input states by linear approximations.Each individual discipline can generate, therefore, an ap-proximate system analysis and compute a full set of ap-proximate states of the system, some of low fidelity (lin-ear approximations) and some of medium fidelity (i. e.those computed directly by the CA). If a design pointis sampled with only one approximate analysis, the datagenerated is referred to asvariable fidelitydata. Howeverif a design point is evaluated by all of the approximatesubsystem analysis,medium fidelitydat! a can be gen-erated by gathering the medium fidelity states computedby each approximate analysis. Further details on con-structing RSAs with medium and variable fidelity can befound in Rodr´ıguezet al. (2001).
2 Adaptive Experimental Design
An important difference, from the experimental pointof view, between traditional laboratory experiments andthe computational experiments embedded in SAO, is thatin the latter the experiment is repeated several times atdifferent locations, up to convergence or stopping of thealgorithm. At each new iteration a new sampling is per-formed of the same system but in a new sampling re-
gion. The cost of constructing a quadratic approxima-tion grows quadratically with the number of design vari-ables. Even with the use of a variable fidelity database,the cost of building such database, in a problem with alarge number of design variables, may override the nat-ural advantages of using a RSA. This problem is knownas thecurse of dimensionality.
A natural way to overcome this problem is to reducethe number of coefficients to be fitted in the Hessian. Thesimplest approach is to fit only the main diagonal termsand set to zero the value of the off-diagonal terms. Sincethis approach may capture some of the nonlinearity ofthe problem, due to the highly nonlinear nature of engi-neering and in particular MDO problems, important in-formation regarding interaction of the design variables isnot taken into account. Moreover, the curvature of theactual response of the system, may change from iterationto iteration, making difficult the decision of which coeffi-cients to compute. However we are not completely blindabout the behavior of the system, as previous informationcan give a hint about the nature of the response surface.
In Perezet al. (2001) the authors investigate the useof information already available from the previous ap-proximation to reduce the size of the experimental designwhile maintaining the quality of the approximation. As aresult, the total cost of the optimization is reduced.
At the very first iteration, a full matrix of second or-der terms (H) is approximated using anO(n2) samplingarray. This matrix can then be diagonalized using eigen-value decomposition:
H = UHUUT , (1)
whereU is the eigenvector matrix andHU is the di-agonal matrix of eigenvalues. At the next iterations, anexperimental array ofO(n) is used to sample the designspace. Before performing the least squares fitting, theeigenvector matrixU is used as a transformation matrixto rotate the design space.
xU = UTx. (2)
In the transformed space, only the main diagonalterms of the Hessian are computed. As a result, back inthe normal space, a full matrix is obtained. If the curva-ture of the function has not changed much, this approx-imate Hessian will be similar to that obtained if a full
3American Institute of Aeronautics and Astronautics
Hessian was computed, as the off-diagonal terms of theHessian in the transformed design space will vanish or atleast will be small compared to those in the main diag-onal. The transformation matrix can be kept as long asthe curvature of the function does not change too much.In that case, the full matrix of second order terms has tobe updated and a new transformation matrix can be com-puted. This approach is called Adaptive ExperimentalDesign (AED).
In Perezet al. (2001) the adaptive experimental de-sign methodology was implemented within the trustregion augmented Lagrangian algorithm of Rodr´ıguezet al. (1998b). The implementation was tested usinga suite of MDO test problems. Results show that themethodology can be applied to engineering problems,significantly reducing the amount of data required to fit afull quadratic function. The reduction is expected to in-crease as the number of design variables in the problemis increased, as the proposed model reduces the numberof parameters to be fitted fromO(n2) to O(n) for mostiterations.
3 Extended Adaptive Experimental Design (EAED)Though the results in P´erezet al. (2001) show the
methodology has considerable savings with respect to afull Hessian update, a full order reduction is not beingaccomplished. At the beginning of the optimization andafter several iterations, a full Hessian update must be per-formed. As a result, several costly full Hessian (O(n2))updates have to be performed. The present paper extendsthe AED methodology to achieve a full reduction in thesampling size fromO(n2) to O(n) by avoiding full Hes-sian updates.
Spectral (eigenvalue) decomposition of the Hessianmatrix gives two types of information. The eigenvectorsform an orthogonal basis which defines the curvature ori-entation. The eigenvalues provide the magnitude of thecurvature along the eigenvectors. If the proper orienta-tion is known (eigenvectors) an update of the main di-agonal terms (eigenvalues) in the transformed space issufficient to compute a full Hessian.
Computing the Hessian of a quadratic approximationthus, can then be viewed as two complimentary tasks.First, find the proper orientation of the curvature and sec-ond, compute the magnitude of the curvature along thatorientation. The magnitude of the curvature can be ap-proximated by sampling the design space, anO(n) task.In order to keep anO(n) sampling size, the orientationhas to be approximated by other means. Even knowingthe actual eigenvectors at any given point, the technique
would require an update scheme since the nonlineari-ties of the problem will keep the eigenvectors changing.Based on the ample experience in quasi-Newton meth-ods, a good choice is to use first order Hessian updates tocompute the eigenvectors.
Assume that at any given pointxk during the opti-mization, zero (f k)and first (∇ f K) order information isknown. Also, zero, first and second (Hk−1) order infor-mation is available for the last design pointxk−1. Anupdate of the eigenvectors can be accomplished in twosteps. Then the normal AED can be applied and a fullHessian approximation is obtained.
1. First an updated HessianHk∗ is computed based onthe available first and second order information.
2. thenHk∗ is decomposed to find its eigenvector matrixUk as in (1).
3. An experimental arrayO(n) A is used to sample inthe neighborhood of the current design point.
4. UsingUk the experimental arrayA is transformedaccording to (2). The main diagonal of the HessianHk
U is computed by a least squares methodology.5. A back transformation, returns the requested full
Hessian matrixHk.
It can be seen that the only difference to the adaptiveexperimental design of P´erezet al. (2001), is the use ofprevious information to compute the eigenvector matrix.This technique is called extended adaptive experimentaldesign (EAED).
4 First order Hessian updatesFirst order Hessian updates have been thoroughly
used in optimization and are very stable and can beproven to converge to the true Hessian. Their behaviorand properties are well known and documented.
In the present paper two update formulas are be-ing implemented: The effective BFGS update and thesymmetric-rank-1(SR1) update. Both are well knownand their use and properties for quasi-Newton methodsfor non-linear optimization are well understood (see forexample Nocedal and Wright (1999)). The main differ-ence between them is that BFGS generates a first order,rank-2 positive definite update, while SR1 is a first or-der rank-1 update not guaranteed to be positive definite.BFGS is well suited for quasi-Newton methods. Posi-tive definiteness is a requirement for line-search type op-timization algorithms, so an iterate is guaranteed to ex-ist. On the other hand SR1 is better for trust-region al-gorithms that do not require positive definiteness in theHessian approximation.
4American Institute of Aeronautics and Astronautics
Given the gradient at the current and last designpoints: ∇ f k+1 = ∇ f k+1(xk+1) and∇ f k = ∇ f k(xk). Wedefine:
sk = xk+1−xk. (3)
yk = ∇ f k+1−∇ f k. (4)
Hence, the BFGS Hessian update formula is:
Hk+1 = Hk− HkskskTHk
skTHksk+
ykykT
ykTsk. (5)
And the SR1 update formula is:
Hk+1 = Hk +(yk−Hksk)(yk−Hksk)T
(yk−Hksk)Tsk . (6)
Both BFGS and SR1 belong to a general class of up-date formulas known as the Broyden family. The EAEDpresented in this paper can make use of any Hessian up-date, since only eigenvector information is obtained bythis means. An example of an alternate Hessian update isthe Canfield approach (Canfield, 2001) which takes intoaccount all the previous zero and first order informationfor the update.
5 EAED algorithm details.In the general case, a response surface approximation
of each objective function and constraint would be per-formed. Each function has to be treated individually. Acomplete discussion of the implementation of the AEDwhen several functions have to be approximated can befound in Perezet al.(2001). In this section a detailed de-scription of the EAED will be provided. The generalitiesof SAO will only be outlined. For the sake of clarity it isassumed that an unconstrained problem is to be solved,therefore a single function has to be approximated:
min.w.r.t.x f (x)s.t. xmin ≤ x ≤ xmax (7)
An implementation of the EAED follows:
1. Givenx0, evaluatef 0 = f (x0) and∇ f 0 = ∇ f (x0).2. Set local move limits.3. Set k=0.4. Using anO(n) size experimental arrayAmxn, sample
the design space aroundxk and build the database.5. If k > 0 go to 6 else makeU1 = I and go to 8.6. Compute the updated Hessian matrixHk∗ according
to the desired update formula (5) or (6).7. Perform an eigenvalue decomposition ofHk∗ accord-
ing to (1). The eigenvector matrixUk will be usedas the transformation matrix.
8. Transform the experimental array:AkU = UkA.
9. Using least squares fitting, compute the componentsof the main diagonal matrixHk
U .10. Perform a back transformation to getHk =
UkTHk
UUk.11. Perform a minimization on the response surface ap-
proximation f (x) over the local move limits. Thenew local optimum isxk+1.
12. Computef k = f (xk+1) and∇ f k+1 = ∇ f (xk).13. Accept or rejectxk+1. If accepted, make k=k+1.14. Update local move limits.15. If ∆xk−xk−1 ≤ ε stop. Else go to 4.
At the very first iteration, only the main diagonalterms are approximated by a response surface. At thefollowing iterations, the previous Hessian approximationis updated using either BFGS or SR1. A spectral decom-position is performed over the updated Hessian to extractthe eigenvector matrix. The eigenvector matrix is used totransform the design space and compute the main diago-nal terms using andO(n) database. We called this tech-nique EAED-BFGS and EAED-SR1 respectively. It isimportant to understand the difference between the wellknown trust-region quasi-Newton method and the trustregion model management framework for approximateoptimization that is being implemented in this paper.Traditional quasi-Newton techniques perform an updateof the Hessian based on zero and first order informationat the current and previous design points. The updatedmatrix is an approximation of the Hessian at the cur-rent design point. In the methodology presented in thispaper, zero and first order information from the currentand previous points are used to update the eigenvectors.The eigenvalues are computed using a least squares re-gression approach for RSA based on response samplingabout the current design. Note that this RSA approachprovides a Hessian matrix which is influenced by datasampled over the whole of the sampling region.
5American Institute of Aeronautics and Astronautics
6 Test problemsTo demonstrate the capability of the proposed EAED
method to construct RSAs for optimization with a reduc-tion in the sampling size toO(n), three problems withdifferent characteristics are tested. Two of them are sin-gle discipline problems and one is a true MDO problem.
In this study, a comparison of the following tech-niques is performed.
1. Full Hessian (FH) Here all the coefficients of theHessian matrix are computed at each iteration.This is the traditional approach requiring aO(n2)database size to construct the RSA.
2. Extended adaptive experimental design with BFGSHessian update (EAED-BFGS). The EAED is usedto reduce the number of coefficients to fit ton asin Perezet al. (2001). The required transformationmatrix is obtained by a BFGS update of the previousHessian.
3. Extended adaptive experimental design with SR1Hessian updates (EAED-SR1). The same as inEAED-BFGS, but the transformation matrix is ob-tained by a SR1 update of the previous Hessian ap-proximation.
6.1 Barnes problemThis is a small mathematical problem known as the
Barnes problem (for a description of the problem seePerezet al., 2001). It has two design variables and threeconstraints which makes it a very easy problem to vi-sualize the results. Also the size of the problem allowsus to evaluate convergence for some of the Hessian co-efficients using the different approaches and analyze theresults.
The performance results, measured by the number ofiterations required to converge, are shown on Table 1.The number of iterations refers to the number of timesthe RSA was constructed. The most important resultis that a reduction of order in the construction of theRSA does not affect the performance of the optimiza-tion. EAED-BFGS and EAED-SR1 require almost thesame number of iterations to converge as does the fullHessian update approach. Although they only requireone order of magnitude smaller database. An extra runwas performed in which only the main diagonal terms ofthe Hessian were computed. The increase in the num-ber of iterations to converge for the main diagonal casecan be explained by the lack of the off diagonal termsin the Hessian matrix. The inclusion of the main diago-nal technique in this comparison is with the sole purposeof showing the disadvantages of using such a technique
with real problems. In nearly linear problems or thosewhere iterations between the variables are negligible, themain diagonal approach will compete with the proposedtechniques.
Approach Iterations
Full Hessian 15
EAED-BFGS 16
EAED-SR1 15
Main diagonal 21
Table 1. Performance results for the Barnes problem.
Though this is a small problem and no spectacularsavings in the number of samplings can be shown, it canprovide us with an overview of what happens to the Hes-sian coefficients as the optimization runs. To this mean,the values of the Hessian coefficients for the objectivefunction and the second constraint (which is active atthe optimum) for the four methods described above weretracked. The values of the coefficients are compared towhat a full Hessian update would have produced.
Figure 1 shows the values history for the coefficientsof the Hessian matrix using EAED-BFGS. The threeplots at the top correspond to the coefficients of the ob-jective function. Note that at the end of the optimiza-tion contrary to what we would have expected, the co-efficients do not converge to that of the full Hessian.Moreover, the coefficients oscillate and at the end Hf11and Hf12 even have different sign. Lets recall that theBFGS provides the Hessian with a positive definite up-date. This is very useful where a line search is performedas you can guarantee a minimum along the line, howeverin this case a positive definite update can bias the trans-formation matrix to an undesired direction. Interestinglyenough, at the end of the optimization the Hessian matrixcomputed by EAED-BFGS is positive definite while thefull Hessian one is not. Besides this, the term Hf22 is thedominant one and has an acceptable approximation eventhough there is some osci! llation.
The coefficients for the second constraint are shownin the bottom three plots of Figure 1. Note that the maindiagonal terms are zero while the off diagonal term hasa small value. In this case after three iterations EAED-BFGS is capable of fitting the right values.
Even though term by term the EAED-BFGS is notcapable of reproducing the values of the Hessian terms,the overall approximation is good enough for efficient
6American Institute of Aeronautics and Astronautics
0 10 20−0.1
−0.05
0
0.05
0.1 Hf(1,1)
0 10 20−0.1
0
0.1
0.2
0.3Hf(2,2)
0 10 20−0.04
−0.02
0
0.02
0.04
0.06Hf(1,2)
0 10 20−4
−2
0
2
4
6x 10
−4 Hg2(1,1)
0 10 20−2
0
2
4
6
8x 10
−4 Hg2(2,2)
0 10 200
0.5
1
1.5x 10
−3 Hg2(1,2)
Full HessianEAED−BFGS
Figure 1. Hessian coefficients using EAED-BFGS
0 5 10 15−0.2
−0.1
0
0.1
0.2 Hf(1,1)
0 5 10 15−0.2
−0.1
0
0.1
0.2
0.3Hf(2,2)
0 5 10 15−0.1
−0.05
0
0.05
0.1Hf(1,2)
0 5 10 15−4
−2
0
2
4
6x 10
−4 Hg2(1,1)
0 5 10 15−2
0
2
4
6
8x 10
−4 Hg2(2,2)
0 5 10 150
0.5
1
1.5x 10
−3 Hg2(1,2)
Full HessianEAED−SR1
Figure 2. Hessian coefficients using EAED-SR1
performance compared to that of a full Hessian update.
In Figure 2 the coefficients history is plotted for theEAED-SR1 technique. It is shown that the EAED-SR1follows the values of the coefficients in a much bettermanner than the EAED-BFGS. At the end of the op-timization the coefficients are easily reproduced by theEAED-SR1 though small variations still remain. For thecase of the second constraint, the values are accurate af-ter a couple of iterations.
6.2 High performance low-cost structure (HPLCS)
This is a structural optimization problem in which theobjective of the design is to minimize the weight of thestructure while the payloads sustained are at their maxi-
mum. The multi-objective optimization is transformedinto a single objective optimization via a cost perfor-mance index. The problem was introduced in Wujeket al.(1995) and consists of a total of 17 design variables(cross sections, trusses longitudes and payloads) and 13inequality constraints. Figure 3 shows the structure to beoptimized.
In order to compare the EAED to the conventionalRSA, two different experimental arrays have to be used.One that requiresO(n2) points and anotherO(n) for theEAED. For the HPLCS, the full Hessian approximationrequires 153 coefficients to be fitted, therefore an orthog-onal array with 162 points (162 points, 19 variables, 9levels strength 2) was used. For the EAED an orthogonalarray with 20 points (19 variables, 2 levels, strength 2)
7American Institute of Aeronautics and Astronautics
P1 P2
P3 P4
M1 M2
M3 M4
L1 L2
L3
A1
A2
A3
A4
A5
A6
A7
A8
A9
A10
Figure 3. High performance low-cost structure test problem.
was used.
Though the original problem as stated in Wujeket al.(1995) is composed of three disciplines, they are linkedby simple feed forward coupling. This allows the treat-ment of the analysis as a single code, therefore only highfidelity states are queried.
Both EAED-SR1 and EAED-BFGS are tested andcompared against the full Hessian (FH) update. Two dif-ferent starting points are tested. Table 2 presents the re-sults for the number of iterations required to convergefor one starting point. In comparison to the Full Hessianapproach, the number of iterations required convergenceusing EAED is increased. This means that there was atoll in the quality of the approximations generated viaEAED. However the real impact can be seen in the to-tal number of function calls as depicted in Figure 4. Thecost of the optimization is measured relative to the num-ber of function calls. A function call is originated byeither a system analysis (in this case a single functioncall), a gradient evaluation (Grad) or a database query(DB). The number of function calls per iteration is fixedfor each of this categories.
Approach Iterations
P1 P2
Full Hessian 39 32
EAED-SR1 51 44
EAED-BFGS 54 45
Table 2. Iterations to converge for the HPLCS test problem.
0
1000
2000
3000
4000
5000
6000
7000
8000
Fun
ctio
nca
lls
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sav
ings
(%)
DB 6318 1024 1080 5103 884 893
Grad 663 871 918 535.5 751 759
SA 39 51 54 31.5 44 45
Theoretical S. 78.89% 78.89% 78.89% 78.89%
Actual S. 72.27% 70.77% 70.38% 70.09%
FH EAED-SR1EAED-BFGS
FH EAED-SR1EAED-BFGS
Point 1 Point 2
Figure 4. Cost in number of iteration and savings for the
HPLCS problem.
In Figure 4 one observes a dramatic decrease in thenumber of function calls required for the optimization.This is mainly due to a reduction in the cost of thedatabase query. Although a slight increase in the num-ber function calls required for the system analysis andsensitivities evaluation, the database generation is signif-icantly smaller. A total reduction of around 70% in thecost of the optimization is achieved with both the EAED-SR1 and EAED-BFGS. The figure also shows a compar-ison between theoretical and actual reduction in the cost.The theoretical reduction is the reduction for a single it-eration. The increase in the number of iterations shownon Table 2 is reflected in the difference between the ac-tual and the theoretical savings.
6.3 Controls-augmented structure (CAS).
The controls-augmented structure (CAS) is a fullycoupled MDO problem consisting of two subsystems:structures and controls. A cantilever beam is subjectedto static and dynamic loads. At the tip there are two con-trollers, one for the lateral and once for the rotational dis-placement. The beam is split into 5 finite elements, andboth width and heigh of each element as well as a pro-portionality constant for the controls conform the set ofdesign variables. It involves a total of 11 design variablesand 43 states.
The information flow makes the problem fully cou-pled and a solution of the system for a given set of designvariables can only be obtained by iteration. There are 7inequality constraints bounding the two first natural fre-quencies, maximum stress, and static and dynamic loads,both lateral and rotational. Figure 5 shows the initial con-figuration of the CAS. The problem was initially intro-
8American Institute of Aeronautics and Astronautics
T T T
P=f(t)
A
B123
Figure 5. Controls-Augmented Structure.
0
2000
4000
6000
8000
10000
12000
CA
calls
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sav
ings
(%)
DB 7833.6 1746 1762 8396.8 1852 1794
Grad 1560.6 1649 1664 1672.8 1749 1694
SA 918 970 979 984 1029 997
Theoretical 59.94% 59.94% 59.94% 59.94%
Actual 57.67% 57.29% 58.11% 59.42%
FH EAED-SR1EAED-BFGS
FH EAED-SR1EAED-BFGS
Figure 6. Cost in number of CA calls and savings for the ACS
problem. Medium fidelity sampling.
0
1000
2000
3000
4000
5000
6000
7000
8000
CA
calls
-100%
-80%
-60%
-40%
-20%
0%
20%
40%
60%
80%
100%
Sav
ings
(%)
DB 3827.2 1023 1150 4390.4 935 1118
Grad 1524.9 1932 2173 1749.3 1766 2111
SA 897 1136 1278 1029 1039 1242
Theoretical 48.33% 48.33% 48.33% 48.33%
Actual 34.54% 26.38% 47.84% 37.63%
FH EAED-SR1EAED-BFGS
FH EAED-SR1EAED-BFGS
Figure 7. Cost in number of CA calls and savings for the ACS
problem. Variable fidelity sampling.
duced by Sobieszczanski-Sobieski and Boebaum (1990)and has been extensively used in several research papers.
The implementation of this problem introduces anew measure for the EAED. In both the Barnes and theHPLCS problems the data for the RSA was obtained byevaluating the system code. In the case of the CAS, asingle function evaluation has to perform several sequen-tial calls to the independent subdisciplines or contribut-ing analysis (CAs). The cost of a single function call is
therefore high and the use of SAO is justified. Each pointin the database is evaluated by the decoupled CAs.
As in the case of the HPLCS, the optimization wasperformed starting from two different points, one in thefeasible and one in the infeasible design space. In bothcases the optimization was carried out with data of vari-able and medium fidelity as described in Section 1.2.The number of iterations measures how many times thedatabase was gathered and the full system analysis per-formed. Table 3 shows the number of iterations for thetwo cases described above, two starting points (P1 andP2) for each. It can be seen that using EAED increasesthe number of iterations required to converge. This is ex-pected as the both, the data gathered is smaller, and themodel is more limited, specially at the beginning, whenfew is known about the orientation of the eigenvectors.However, this do not reflex the true cost of the optimiza-tion. Figure 6 and 7 show the cost in number of CA callsfor both medium and variable fidelity databases. In thecase of the medium fidelity database, the savings are justbelow 60%. Both starting points are consistent with theresults, though slight different number of calls. As in theHPLCS, no significant difference is shown between theEAED-SR1 and the EAED-BFGS. The theoretical sav-ings for this problem, are smaller due to the fact that theSA requires a large number of CA calls in contrast toone function evaluation in the HPLCS case. The vari-able fidelity results, show the same tendency, though inthis case the DB generation is even cheaper than in themedium fidelity case, therefore the savings are still lessimpressive but not less significant: between 26% and48%.
Approach Medium Variable
fidelity fidelity
P1 P2 P1 P2
Full Hessian 31 33 30 34
EAED-SR1 32 34 38 35
EAED-BFGS 33 33 43 41
Table 3. Iterations to converge for the HPLCS test problem.
(EAED) is a powerful technique for reducing the compu-tational cost of constructing quadratic response surface
9American Institute of Aeronautics and Astronautics
approximations. The technique reduces the size of thedatabase required for constructing a quadratic responsesurface from ordern2 to ordern. The methodology hasbeen shown to have comparable performance in sequen-tial approximate optimization to that of constructing fullquadratic response surface approximations where ordern2 sampling is used. The reduced order approximationcaptures the essential information of a full quadratic ap-proximation leading to similar results with one order ofmagnitude smaller sampling size required. Results fordifferent sizes of test problems have demonstrated scala-bility of the technique to medium size type of problems.The EAED can also be used to build improved Hessianupdates for non-linear programming methods. In thiscase, the sampling would be performed in a very smallneighborhood of the current design point.
Acknowledgements
This multidisciplinary research effort is supported inpart by the following grants and contracts: NSF grantsDMI98-12857 and DMI01-14975, NASA grant NAG1-2240 and CONACyT, Mexico.
References
Balabanov, V., Kaufman, M., Knill, D. L., Haim, D.,Golovidov, O., Giunta, A. A., Haftka, R. T., Gross-man, B., Mason, W. H., Watson, L. T. 1996: De-pendence of optimal structural weight on aerodynamicshape for a high speed civil transport. InProceedingsof the 6th AIAA/USAF/NASA/ISSMO Symposium onMultidisciplinary Analysis and Optimization, AIAA96-4046, pp. 599 – 613. Bellevue, WA. September 4-6.
Canfield, R. A. 2001: Sequential multipoint quadraticapproximation for numerical optimization. InPro-ceedings of the 42st AIAA/ASME/ASCE/ AHS/ASCStructures, Structural Dynamics, and Materials Con-ference, AIAA 2001-1498. Seattle, WA. April 16-19.
Chen, W., Allen, J. K., Schrage, D. P., Mistree,F. 1996: Statistical experimentation for affordableconcurrent design. InProceedings of the 6thAIAA/USAF/NASA/ISSMO Symposium on Multidisci-plinary Analysis and Optimization, AIAA 96-4085,pp. 921 – 930. Bellevue, WA. September 4-6.
Giunta, A., Watson, L. T. 1998: A comparison of ap-proximation modeling techniques: Polynomial ver-sus interpolating models. InProceedings of the 7thAIAA/USAF/NASA/ISSMO Symposium on Multidisci-
plinary Analysis and Optimization, volume AIAA 98-4758, pp. 392–404. Saint Louis, MO. September 6-8.
Giunta, A. A., Dudley, J. M., Narducci, R., Grossman,B., Haftka, R. T., Mason, W. H., Watson, L. T. 1994:Noisy aerodynamic response and smooth approxima-tions in hsct design. InProceedings of the FifthAIAA/USAF/NASA/ISSMO Symposium, volume AIAA94-4376, pp. 1117 –1128. Panama City, FL. Septem-ber 7-9.
Nocedal, J., Wright, S. J. 1999:Numerical Optimization.Springer.
Perez, V. M., Renaud, J. E. 2000: Decoupling the de-sign sampling region from the trust region in approxi-mate optimization. InProceedings of the InternationalMechanical Engineering Congress & Exposition, vol-ume 63 ofAD. Orlando, FL. November 5-10.
Perez, V. M., Renaud, J. E., Gano, S. E. 2000: Construct-ing variable fidelity response surface approximationsin the usable feasible region. InProceedings of the 8thAIAA/NASA/USAF Multidisciplinary Analysis & Opti-mization Symposium, AIAA 2000-4888. Long Beach,CA. September 6-8.
Perez, V. M., Renaud, J. E., Watson, L. T. 2001:Adaptive experimental design for construction of re-sponse surface approximations. InProceedings of the42nd AIAA/ASME/ASCE/AHS/ASC Structures, Struc-tural Dynamics, and Materials Conference, AIAA2001-1622. Seattle, WA. April 16-19.
Renaud, J. E., Gabriele, G. A. 1993: Improved coordina-tion in nonhierarchic system optimization.AIAA Jour-nal, volume 31, no. 12, pp. 2367–2373.
Renaud, J. E., Gabriele, G. A. 1994: Approximationin nonhierarchic system optimization.AIAA Journal,volume 32, no. 1, pp. 198–205.
Rodrıguez, J. F., P´erez, V. M., Padmanabhan, D., Re-naud, J. E. 2001: Sequential approximate optimiza-tion using variable fidelity response surface approxi-mations. Structural and Multidisciplinary Optimiza-tion, volume 22, pp. 24–34.
Rodrıguez, J. F., Renaud, J. E., Watson, L. T. 1998a:Convergence of trust region augmented lagrangianmethods using variable fidelity approximation data.Structural Optimization, volume 15, pp. 141–156.
Rodrıguez, J. F., Renaud, J. E., Watson, L. T. 1998b:Trust region augmented lagrangian methods for se-quential response surface approximation and opti-mization.Journal of Mechanical Design, volume 120,pp. 58–66.
Simpson, T. W., Peplinski, J., Koch, P. N., Allen, J. K.1997: On the use of statistics in design and the im-plications for deterministic computer experiments. InProceedings of the 1997 ASME Design Engineering
10American Institute of Aeronautics and Astronautics
Technical Conference, volume DETC97/DTM-3881.Sacramento, CA. September 14-17.
Sobieszczanski-Sobieski, J. 1988: Optimization by de-composition: A step from hierarchic to non-hierarchicsystems. InSecond NASA/Air Force Symposium onRecent Advances in Multidisciplinary Analysis andOptimization, NASA TM-101494, CP-3031, Part 1,1989, pp. 28–30. Hampton, VA.
Sobieszczanski-Sobieski, J., Boebaum, C. L. 1990: Sen-sitivity of controls-augmented structure obtained by asystem decomposition method.AIAA Journal.
Wujek, B. A., Renaud, J. E. 1998a: A new adaptivemove-limit management strategy for approximate op-timization, part i. AIAA Journal, volume 36, no. 10,pp. 1911–1921.
Wujek, B. A., Renaud, J. E. 1998b: A new adaptivemove-limit management strategy for approximate op-timization, part ii. AIAA Journal, volume 36, no. 10,pp. 1922–1937.
Wujek, B. A., Renaud, J. E., Batill, S. M. 1997: Aconcurrent engineering approach for multidisciplinarydesign in a distributed computing environment. InN. Alexandrov, M. Y. Hussaini, eds.,MultidisciplinaryDesign Optimization: State-of-the-Art, volume 80 ofProceedings in Applied Mathematics Series, pp. 189–208. SIAM.
Wujek, B. A., Renaud, J. E., Batill, S. M., Brockman,J. B. 1995: Concurrent subspace optimization usingdesign variable sharing in a distributed design envi-ronment. In S. Azarm, ed.,Proceedings of the DesignEngineering Technical Conference, Advances in De-sign Automation, ASME DE, volume 82, pp. 181–188.
11American Institute of Aeronautics and Astronautics
