IDENTIFICATION OF PARAMETER COUPLINGS IN TURBINE DESIGN USING NEURAL NETWORKS

Sanjay ~ o e l + General Electric Corporate Research and Development

Schenectady, New York 12301

Prabhat ~ a j e l a * Rensselaer Polytechnic Institute

Troy, New York 12180

design such location dependent heuristics have to be generated analytically using the design data.

A new technique which aids in the optimization of complex engineering problems by identifying parame- ter dominance and couplings in different regions of the search space has been developed. In this approach the search space is divided into multiple regions by clustering the data. The data from each cluster is used to train a feed-forward neural-network and the weight matrices of the trained neural-network are analyzed to identify parameter couplings in each cluster. This information is used for generating design rules which are associated with the location of design in the search space. The parameter couplings obtained using the described methodology have shown a remarkable sim- ilarity to the design heuristics for the turbine-design problem. Results for a three-stage power turbine are described in the paper along with a comparison to the existing methodology.

Introduction

There is a major push towards integrating multiple disciplines for solving complex engineering problems. A large number of these problems have non-linear, multi-modal search space, with large number of de- sign variables. The combined computation time for solution of multiple analysis codes is significant enough to restrict the number of iterations that can be performed during optimization. Since hill-climbing, and other gradient based numerical search techniques are prone to the problem of converging to a local opti- mum, and random search techniques like genetic al- gorithms and simulated annealing are computational- ly expensive, heuristics-based search techniques are most appropriate for solving large problems. Since the problem-heuristics can vary over the search-space, de- sign-rules for optimization which are based on the problem-heuristics should also change accordingly. While designers do associate heuristics with regions of the search-space at an abstract plane, in automated

-

t Engineer, Fluid Mechanics Program, member AIAA

$ Professor, Associate fellow A M

During design optimization a large amount of data is generated and from which heuristics can be derived for improvement of subsequent designs. Learning al- gorithms can be employed to extract a symbolic de- scription of design mechanics from the data and to provide design knowledge to improve the optimiza- tion efficiency. The process involves identifying pat- terns in the design space and associating them with de- sign changes. Generating a continuous distribution of desired design changes over the entire search space is computationally infeasible, and hence the search space must be divided into finite regions and sensitivi- ties computed for each region. The finite regions in the search space are computed by using a partitional clustering algorithm.

In recent years there has been a focus on knowledge- intensive learning methods which generalize from a single example of a concept by analyzing why the ex- ample is an instance of the concept. In 1986 Mitchell, Keller and Kedar-Cabelli[l] described explanation based generalization as a unified approach to ex- planation based learning. Minton[2] has created a sys- tem for improving the problem solving efficiency by generating search control knowledge using explana- tion-based-generalization. In such systems, the cost of evaluating the control knowledge at each point in the search may far outweigh the gains from the knowl- edge.

A significant amount of work has also been done in the area of analogical reasoning and case-based reason- ing, in which learning is achieved by storing examples of cases for future retrieval. For solving a new prob- lem, a stored case most similar to the current case is retrieved and generalization is done using local trans- formations[2]. The most serious drawback of case- based reasoning systems is that the expense of match-

Copyright 1994 by Sanjay Goel. Published by the American Institute of Aeronautics and Astronautics, Inc. with permission.

590

ing often exceeds the benefit that the stored examples produce.

The proposed neural network based approach does not need to store and retrieve all design cases since the heuristics extracted from these cases are translated into design rules. The expense of matching a cluster is insignificant compared to the expense of searching from all previous design cases because of a much high- er granularity of the clusters. In addition, the initial expense of clustering the data, training the neural- network, and determining parameter sensitivities, is amortized over multiple design optimizations. The use of rules ensures that any bad generalizations ob- tained in the weight analysis are rejected by a rule sus- pension mechanism which is based on the perfor- mance of the design rule. One drawback in the pro- posed approach is that heuristics can only be updated prior to starting a search unlike case-based reasoning where the database can be continuously updated dur- ing, the optimization. In order to utilize the designs generated during the current session a database of de- sign cases may be maintained and used in conjunction with the design rules for retrieving cases which have been run previously in the session, thereby eliminat- ing some design-analysis iterations.

lbrbine Desipn

Turbine design is a labor intensive process that re- quires the interaction of various engineering disci- plines. The design involves three major stages: cycle design, preliminary design, and detailed design. The objective of the design is to maximize the turbine per- formance which is usually the fuel efficiency of the tur- bine. As shown in figure 1 constraints from multiple engineering disciplines interact with the aerodynamic performance of the turbine. Bringing together consid- erations from multiple engineering disciplines into a single design system ensures that constraints from all disciplines are satisfied during optimization, and un- necessary iterations between different designers is avoided.

Turbine preliminary design does not have a closed form solution; that is, no explicit solution exists. The designer can only analyze the design using analysis programs in order to predict turbine performance. The designer normally starts with a crude estimate of the design and iteratively improves it to the extent that the resources permit, or no further improvement is obtained. Iterative design for optimizing the aerody- namic performance process has been automated by

using numerical optimization and heuristic search.[3,4] The automated process works efficiently by evaluating a large number of designs in a short peri- od without designer intervention. As the turbine mod- el becomes more complex by incorporating multiple engineering disciplines it becomes an increasingly complex optimization problem.

I

Figure 1: Multidisciplinary interactions for turbines.

The complexity of the problem can be somewhat alle- viated by reducing the number of design variables and problem constraints. Since the degree of influence of parameters varies in different regions of the search space, the overall complexity can be reduced by deter- mining dominant parameters in different regions. Hence, the process requires dynamically changing the set of design variables. This requires a description of parameter interactions in the search space which can be generated from previously generated optimization data. The proposed scheme has been implemented for the turbine design problem, and focuses on the gen- eration of efficient design heuristics which are re- quired to quickly converge to the optimal solution.

Context De~endence and Clustering

The turbine performance varies considerably as the design parameters are varied over the admissible de- sign range. Hence, it is not very meaningful to estab- lish the relations between input and output parame- ters over a wide range of design variation. In addition, training a neural network over a very large domain may be computationally infeasible. To get meaningful relationships, and to reduce the problem complexity, the search space needs to be organized into clusters where membership of each cluster is determined by using a formal analysis technique designed to recog- nize similarity. Since only single partitions are re- quired, a partitional clustering scheme is sufficient for the problem.

Fukunaga and Short[S] have used clustering for prob- lem localization, whereby a simple decision rule can be implemented in local regions or clusters of the pat- tern space. In multidimensional pattern space, clus- ters can be of arbitrary shapes and sizes, and the clus- tering criteria imposes a structure on the data.[6] The crucial problem in identifying clusters in data is to specify what proximity is, and how to measure it. The euclidean distance Minkowski metric has been used as the proximity index in several engineering applica- tions. It is easy to visualize clusters based on one, two or three attributes. However, visualizing a cluster with more than three attributes is not easy.

To adequately populate the design space, the design data used in generating heuristics must be uniformly distributed over the search space. To minimize bias, data generated in a genetic algorithm search during turbine optimization is used in the present work. At each design point the values of the design variables, the constraint parameters, and the objective function are stored. Both feasible and infeasible design points are included in the analysis data since the region around the feasible-infeasible boundary is of special interest for sensitivity analysis which is used for trade- off studies. However only design variables are used in the determination of the clusters. The data for each design variable is normalized between 0 and 1 to mea- sure the relative strengths of parameter interactions.

Neural Networks

A large number of complex engineering problems, with no closed form solution are solved iteratively us- ing analysis models which predict the performance of the design. An ideal design procedure would be to in- vert the governing equations so that design parame- ters can be obtained for the desired performance. So- bieski et al.[q, Novak and Haymann-Haber[8], and Giles, Drela, and Thompkins[9] provide simple il- lustrations of the inverse approach. However, the sheer complexity of obtaining the inverse solution makes this approach infeasible for all but the simplest of problems.

Figure 2 shows a schematic diagram of a feed-forward neural network with N layers of neurons each contain- ing ni nodes, where i is the layer index. The first layer contains the input nodes and the outputs are available at the neurons of the last layer. The neurons are inter- connected as shown in the figure, and the strength of these interconnections are referred to as network weights. wKij denotes the weight or strength of connec-

tion between the i-th and the j-th neuron of the K-th layer.

Nodes Weights

\ Output Layer

Figure 2: Schematic diagram of a neural network.

A backpropagation neural network is based on a mul- tilayer, feed-forward topology, with supervised learn- ing. At the start of network training, the weights are randomly initialized. The network is then exposed to a set of training data which includes a number of input vectors and the corresponding outputs. As the train- ing proceeds the weights are incrementally adjusted until the error between the network predicted output and and the actual known output for all patterns drops below a threshold. In each adjustment, weights lead- ing to the output nodes are adjusted in proportion to the difference between the output node's actual out- put and its desired or target output; the weights lead- ing into hidden nodes are in turn adjusted in propor- tion to their contributions to errant higher nodes until the training is complete. At the end of the training phase, the weights are frozen; then, unknown inputs can be fed into the stabilized network for character- ization.[lO]

Similar to an analysis model, a neural network trained to predict the performance of a design can map the de- sign parameters to the performance parameters. This input-to-output mapping is embedded in the weight matrices of the neural network. The mapping in the mathematical model of the problem is more accurate than a neural network; however, the complexity of the mathematical model coupled with the dimensionality of the problem makes it infeasible to extract explicit inverse relationship between output and input quanti- ties. On the other hand, the weight matrices of the neural net are well suited for mathematical manipula- tion which makes it relatively easy to extract parame- ter dependencies from the weight data. If the model is highly non-linear, the parameter dependencies ob- tained from this process may be significantly different in different regions of the search space, and may not be accurate enough to provide an explicit solution.

However, a localized mapping obtained in a clustered search space is accurate enough for the purpose of generating rules to guide heuristic-search during opti- mization.

Szewczyk[ll,l2] has proposed a scheme for analyzing the weights of a single hidden layer neural network for computing relative strengths of parameter interac- tions, and has used the scheme for decomposing struc- tural problems. In that analysis the signs of the weights were ignored leading to a loss of information during the weight analysis. The proposed weight analysis scheme is simpler, more accurate, and provides addi- tional information about parameter sensitivities. In addition this scheme has the flexibility of allowing use of weights from a multilayer feed-forward network.

Equation 1 describes this alternate process of calcu- lating the relationships between input and output quantities based on the weights of a backpropagation neural network. This represents an 'averaged' sensi- tivity matrix R (hereafter referred to as the sensitivity matrix), obtained by sequentially multiplying the weight matrices of the different layers of the trained backpropagation network.

where [W1nixni+l is matrix containing interconnection weights between layers i and i+l .

In the sensitivity matrix the output quantities are rep- resented by the columns and the input quantities by the rows. The sensitivity matrix is normalized with re- spect to the input variables to obtain the relative dom- inance. To obtain the relative sensitivity of an output element wrt all input parameters, element of a column in the matrix is divided by the magnitude of the largest component in the column.

T. . , = J where i = l...nl j = l...nN

(2)

The elements of the normalized sensitivity matrix contain information of both the relative influence of input parameters on an output, and the sign of the relationship. The sensitivity information of the objec- tive is used for making intelligent design changes dur- ing the design process. Sensitivities of the other out- put parameters are similarly used for constraint satis- faction of the initial design and to determine the feasi- bility of the design changes by determining the effects

of design changes on the constraints during the design process.

Rule Formulation

Turbine optimization is performed using an iterative design scheme (Figure 3) and a turbine model devel- oped by Goe1.[3,4] In this scheme the turbine parame- ters are modelled using parametric curves and other mathematical models to facilitate use of heuristic and gradient-based optimization methods. These search strategies make changes to the turbine model, which are then translated to the analysis model for purposes of evaluating the design. The process continues till all the specified design criteria are satisfied.

8 Generate Model

10 Execute Analysis Program

8 Translate Turbine Data

Figure 3: Iterative Design Scheme.

To assist this optimization process, design rules are generated using the weight analysis described above. The rules generated using this information identify the cluster to which the current design point belongs by measuring its distance from each cluster. A priori- tized list of parameters is determined on the basis of the normalized sensitivity matrix for the particular cluster. Influence of each input parameter on the constraints is evaluated using that information, and the parameter with the highest priority likely to stay within bounds is selected for change. After each de-

sign iteration, the process of selecting a design param- eter starts again and the iterative design continues as described in Figure 3.

Samples of rules generated for the design are shown below. The first rule is used for improvement of the turbine efficiency while the second rule is used for de- creasing the Stage3 swirl. The new rules generated us- ing the neural-net weight analysis scheme are given a higher weighting so that they are preferentially se- lected over the other rules in the rule base during the design process.

Goal: Weight: Conditions: Actions:

Goal: Weight: Conditions: Actions:

To increase efficiency 1 .o current-state E clusterl Increase Stg3-zwib Decrease Stg2-zwib Decrease Stg2-reaction Increase Stgl-zwib

To Increase Stg3 -swirl 1 .o current-state E clusterl Decrease Stg3-zwib Decrease Stg3-zwiv Increase Stg2-zwib

Results

the euclidean distance norm. Six largest clusters were used for the weight analysis and six small clusters were ignored since the data was not dense enough to get meaningful parameter sensitivities.

naining samples for each cluster contained 250-350 data points. Numerous configurations of a backpro- pagation neural network metas were evaluated and eventually a three-layer 9-24-7 configuration was se- lected. Different values of learning rate were consid- ered for the training. The best convergence was ob- tained by using a strategy of gradually decreasing the learning rate in steps. In the final analysis, the training error was reduced to less than 2%.

Weight analysis was performed on each cluster, and the results for cluster 1 analysis were as shown in Table 1. The rows represent the input variables and the col- umns represent the output parameters. An excellent correlation with known heuristics is obtained from the analysis. As an example Table 1 shows that the turning on stage1 is influenced only by the reaction on stage1 and not by reactions on stage 2 and stage 3 and it is re- duced by increasing the reaction which are known facts. Another observation from Table 1 is that the up- stream output parameters show a strong correlation with upstream design parameters and almost no cor- relation with downstream design parameters. Intu- itively, this makes sense as the flow moves from up-

stages. A similar weight analysis was performed using the ap-

Input (Design) Parameters: proach proposed by Szewczyk[lO,ll]. These results were not satisfactory and showed low correlation with

1.Stage-zwiv (zweifel #): Measure of vane known parameter dependencies. A clear separation 2.Stage-zwib (meifel#): Measure of blade solidity. of the dominant parameters from the rest of the pa-

3.Stage-reaction: Ratio of pressure drop across rameters was missing. In addition the sensitivities do

blade and vane on a stage. not have an associated sign, where the latter is useful in rule formulation.

Output Parameters:

1.Stage-turning: Flow turning at root.

2.Stage-swirl: Swirl on the flow at stage exit. Table 2 shows the parameter sensitivities of different parameters to the turbine efficiency across different

3.Efficiency: Turbine efficiency. clusters. There are significant differences in parame- ter sensitivities in different clusters. A particular pa-

Data generated during an optimization run using ge- rameter can have a positive gradient in one cluster and netic algorithm search was used for the analysis. 1500 a negative gradient in another. This information data points were randomly picked from the optimiza- makes the rules generated for optimization very use- tion data. The data was clustered into 12 clusters using ful, since the rules change from region to region in the

Table 1: Sensitivity matrix for cluster1 using proposed approach

I Efficiency I Stgl-turn

Table 2: Parameter sensitivities for efficiency across different clusters

search space. The analysis also shows that stage 3 pa- rameters have a strong correlation to the efficiency and are critical during optimization.

Cluster #

Stgl-reaction

S tg2-reaction

Stg3-reaction

Stgl-zwiv

Stg2-zwiv

Simple rules generated for efficiency improvement using the data in Table 2 are shown below. In these rules information of the constraint sensitivities was ig- nored for sake of simplicity. Six rules were generated based on the parameter dominance with one rule cor- responding to each cluster. In each rule the parame- ters are sequentially varied in order of their domi- nance in search. Another observation from Table 3 is that the stage2 reaction is a near optimum in clusters 1, 2, and 3 and stage3 reaction is a near optimum in clusters 4 and 5.

Rule 1

1

-0.217

0

-0.507

0.116

-0.031

Goal: To increase eficiency Weight: 1.0 Conditions: current-state E cluster2 Actions: decrease stg3 -mib

increase stg2-zwib increase stg3 -rxh decrease stgl -zwib increase stgl -rxh

Rule 2

2

0.112

0

0.92 1

-0.341

-0.609

Goal: To increase eficiency Weight: 1.0 Conditions: current-state E cluster2 Actions: increase stg3 -zwib

increase stg2-zwib decrease stg3-mh increase stg2-zwiv increase stgl -zwiv

3

0.468

0

0.640

-0.404

-0.352

4

0.212

0.226

0

-0.193

0.109

5

-0.435

0.724

0.051

0

1.000

6

0.533

-0.785

-1 .000

-0.395

0.389

Rule 3

Goal: To increase eficiency Weight: 1.0 Conditions: current-state E cluster3 Actions: increase stg2-zwib

decrease stg3 -rxh increase stg3 -zwib decrease stg3 -zwiv decrease stgl -zwib decrease stgl -rxh increase stgl -zwiv increase stg2-zwiv

Rule 4

Goal: To increase eficiency Weight: 1.0 Conditions: current-state E cluster4 Actions :increase stg3-zwib

decrease stg2-rxh decrease stgl -rxh increase stgl -zwiv decrease stgl -zwib

Rule 5

Goal: To increase efficiency Weight: 1.0 Conditions: current-state E cluster5 Actions: decrease stg2-zwiv

decrease stg3-zwib decrease stg2-rxh decrease stgl -zwib increase stgl -rxh decrease stg3-zwiv

Rule 6

Goal: To increase efficiency Weight: 1.0 Conditions: current-state E cluster6 Actions: increase stg3 -rxh

increase stg2-rxh increase stg3-zwib decrease stg3 -zwb decrease stgl -rxh increase stgl -zwiv decrease stg2-zwiv increase stgl -zwib

An optimization of the turbine was performed using these rules. A simple one-dimensional search was used where parameters were varied sequentially. Dur- ing the entire optimization the design process was in cluster 4, and hence only the rule for cluster 4 was used. The optimization performance using this rule

compared to that without the rule is shown in figure 4. The solid line shows the optimization using the clus- ter-based rules generated by the weight analysis. The rule produces a better result in 25 percent fewer itera- tions.

Efficiency vs Runs

0.9189 9.

Number of Runs

Figure 4. Graph showing optimization performance

Conclusions

A new approach has been developed to generate heu- ristics by learning from optimization data by using a backpropagation neural-network. In the new ap- proach the search space is clustered, a feed-forward neural network is trained for each cluster, parameter sensitivities for each cluster are determined by analyz- ing the respective weight matrices, and cluster-depen- dent rules are generated by using these parameter sensitivities.

Even though the approach worked well for the turbine from which design data was used in the analysis, it is unclear whether the same results can be used for a dif- ferent type of turbine or even a similar turbine in a dif- ferent load class.

Only very simple rules have been generated for the current investigation. i.e. parameters are sequentially varied based on the dominance in a region. More com- plex rules which incorporate the sensitivities of the constraints can improve the efficiency of the rules fur-

ther by preventing them from going into infeasible re- gions.

The authors would like to thank General Electric Cor- porate Research and Development for their support in this work. The authors would also like to acknowl- edge partial support for this work received under grant NAG- 1 - 1269 from National Aeronautics and Space Administration. The first author would like to thank his parents and wife for their encouragement and support during the work.

References 1. T. Mitchell, R. Keller, and S. Kedar-Cabelli,

"Explanation-Based Generalization: Unifying View". Machine Learning 1, 1 (1986).

2. S. Minton, "Learning Search Control Knowledge- An Explanation-Based Approach", Kluwer Academic Publishers, 1988.

3. S.S. Tong, D. Powell, and S. Goel, "Integration of Artificial Intelligence and Numerical Optimization Techniques for the Design of Complex Aerospace Systems," AIAA 1992 Aerospace Design Conference, February 3-6, 1992, Irvine.

4. S.Goe1, B GregoryandD.Cherry, "KnowledgeBased System for the Preliminary Aerodynamic Design of Aircraft Engine Turbines", SPIE 1993 Applcations of Artificial Intelligence 1993: Knowledge-Based Systems

in Aerospace and Industry, April 13-15, 1993, Orlando, Florida

5. K. Fukunaga, and R. D. Short (1978). "Generalized Clustering for Problem Localization." IEEE Transactions on Computers C 20, 176-183.

6. A.K. Jain, and R. C. Duber, Algorithms for Clustering Data, Prentice Hall, 1988.

7. H. Sobiesky, N.J. Yu, K-Y. Fung, and A.R. Seebass, "New Method for Designing Shock-Free Transonic Configuration." AIAA Journal, 17 (7). 722.

8. R.D. Novak andG. Haymann-Haber, "A Mixed-Flow Cascade Passage Design Procedure Based on a Power Series Expansion." Journal of Engineering for Power, April 1983,105,231-242.

9. M. Giles,M. Drela, and W. Thompkins,"NewtonSolu- tion of Direct andInverseTransonicEulerEquations,"Pro- ceedings of the AIAA 7th Computational Fluid Dynamics Conference,Cincinnati,Ohio, July 15-17,1985,390402.

10. Z. Szewczyk and P. Hajela, "Neurocomputing Strategies in Decomposition Based Structural Design", proceedings of the 34th AIAA/ASCE/AIIS/ASC SDM conference, La Jolla, California, pp 2458-2465, 1993.

11. Z. Szewczyk, "Neurocomputing Based Approximate Models in Structural Analysis and Design", PhD. thesis, May 1993, Rensselear Polytechnic Institute, Troy, New York.

12. M. Chester, "NeuralNetworks -A tutorial",Prentice Hall, Englewood Cliffs, New Jersey.