Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M1.1 Optimization of Mechanical Systems - Content of lecture - 1 Introduction 1.1 Motivational Examples 1.2 Formulation of the Optimization Problem 1.3 Classification of Optimization Problems 2 Scalar Optimization 2.1 Optimization Criteria 2.2 Standard Problem of Nonlinear Optimization 3 Sensitivity Analysis 3.1 Mathematical Tools 3.2 Numerical Differentiation 3.3 Semianalytical Methods 3.4 Automatic Differentiation 4 Unconstrained Parameter Optimization 4.1 Basics and Definitions 4.2 Necessary Conditions for Local Minima 4.3 Local Optimization Strategies (Deterministic Methods) 4.4 Global Optimization Strategies (Stochastic Methods) 5 Constrained Parameter Optimization 5.1 Basics and Definitions 5.2 Necessary Conditions for Local Minima 5.3 Optimization Strategies 6 Multicriteria Optimization 6.1 Theoretical Basics 6.2 Reduction Principles 6.3 Strategies for Pareto-Fronts 7 Application Examples and Numerical Tools
Transcript
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M1.1
Optimization of Mechanical Systems
- Content of lecture -
1 Introduction
1.1 Motivational Examples
1.2 Formulation of the Optimization Problem
1.3 Classification of Optimization Problems
2 Scalar Optimization
2.1 Optimization Criteria
2.2 Standard Problem of Nonlinear Optimization
3 Sensitivity Analysis
3.1 Mathematical Tools
3.2 Numerical Differentiation
3.3 Semianalytical Methods
3.4 Automatic Differentiation
4 Unconstrained Parameter Optimization
4.1 Basics and Definitions
4.2 Necessary Conditions for Local Minima
4.3 Local Optimization Strategies (Deterministic Methods)
4.4 Global Optimization Strategies (Stochastic Methods)
5 Constrained Parameter Optimization
5.1 Basics and Definitions
5.2 Necessary Conditions for Local Minima
5.3 Optimization Strategies
6 Multicriteria Optimization
6.1 Theoretical Basics
6.2 Reduction Principles
6.3 Strategies for Pareto-Fronts
7 Application Examples and Numerical Tools
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M1.2
Bibliography
Optimization:
S.K. Agrawal and B.C. Fabien: Optimization of Dynamic Systems. Dordrecht: Kluwer Ac-ademic Publishers, 1999.
M.P. Bendsoe and O. Sigmund: Topology Optimization: Theory, Methods and Applica-tions. Berlin: Springer, 2004.
D. Bestle: Analyse und Optimierung von Mehrkörpersystemen - Grundlagen und rech-nergestützte Methoden. Berlin: Springer, 1994.
D. Bestle and W. Schiehlen (Eds.): Optimization of Mechanical Systems. Proceedings of the IUTAM Symposium Stuttgart, Dordrecht: Kluwer Academic Publishers, 1996.
P. Eberhard: Zur Mehrkriterienoptimierung von Mehrkörpersystemen. Vol. 227, Reihe 11, Düsseldorf: VDI–Verlag, 1996.
R. Fletcher: Practical Methods of Optimization. Chichester: Wiley, 1987.
R. Haftka and Z. Gurdal: Elements of Structural Optimization. Dordrecht: Kluwer Aca-demic Publishers, 1992.
E.J. Haug and J.S. Arora: Applied Optimal Design: Mechanical and Structural Systems. New York: Wiley, 1979.
J. Nocedal and S.J. Wright: Numerical Optimization. New York: Springer, 2006.
A. Osyczka: Multicriterion Optimization in Engineering. Chichester: Ellis Horwood, 1984.
K. Schittkowski: Nonlinear Programming Codes: Information, Tests, Performance. Lect. Notes in Econ. and Math. Sys., Vol. 183. Berlin: Springer, 1980.
W. Stadler (Ed.): Multicriteria Optimization in Engineering and in the Sciences. New York: Plenum Press, 1988.
G.N. Vanderplaats: Numerical Optimization Techniques for Engineering Design. Colora-do Springs: Vanderplaats Research & Development, 2005.
P. Venkataraman: Applied Optimization with MATLAB Programming. New York: Wiley, 2002.
T.L. Vincent and W.J. Grantham: Nonlinear and Optimal Control Systems. New York: Wiley, 1997.
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M2.1
Optimization of mechanical systems
classical/engineering approach
analytical/numerical approach
intuition, experience of the design engi-neer and experiments and fiddling with hardware prototypes at the end of the de-sign process
intuition, experience of the design engineer and virtual prototypes based on computer simulations throughout the whole design process
sequential process
concurrent engineering
• only small changes of the design pos-sible to influence the system behavior
• more fundamental changes might re-quire the reconsideration of all previous design steps
• “optimal” solution without quantitative objectives
• hardware experiments are costly and time intensive (only few are possible)
• provides many design degrees of free-dom at beginning of the design process
• parameter studies can be executed fast and easy
• systematic way to find optimal solution with respect to defined criteria
• cost efficient • shortening of the development time • initial design for hardware experiments is
already close to optimum
idea
concept
CAD design
simulation
prototype
experiments product
optimization
production preparation
idea concept
design
calculation
prototype
experiments product
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M2.2
Optimization is a part of any engineering design process. Furthermore optimization is also extensively used in many other disciplines, such as e.g. industrial engineering, logistics or economics. The focus of this class is on optimization of mechanical systems using the analyti-cal/numerical approach. The concepts and methods are presented in a general manner, such that they can be applied to general optimization problems. The systematic formulation of an optimization problem requires the answers of three basic questions: 1. What should be achieved by the optimization? 2. Which changeable variables can influence the optimization goals? 3. Which restrictions apply to the system?
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M3
Iterative solution of the standard problem of parameter optimization
)( ),( ),( )()()( iii hgf ppp
initial design )0( ,0 p=i
evaluation of the performance
propose a better design 1 ,)1( +=+ iiip
performance satisfying?
yes design “optimal”
no
evaluation of functions • performance functions • constraints
simulation model
calculation of gradients • performance functions • constraints
)(ip
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M4
Integrated Modelling and Design Process
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M5
Classification of optimization problems
optimization
topology function parameters
scalar optimization
multi-criteria optimization
)(min pfp P∈
unconstrained
)(min pp
fh
�∈
constrained
)(min pp
fP∈
{ }ouhP ppp0)p(h0pgp ≤≤≤=∈= , ,)( �
scalarization
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M6.1
Geometric visualization in 2D
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M6.2
inequality constraints
equality constraints
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M7.1
Matrix Algebra and Matrix Analysis
vector nR∈x : [ ]n1 xx K=x , R∈ix
matrix nmR ×∈A :
=
mn
n1
1m
11
A
A
A
A
M
K
K
MA , R∈ijA ,
Basic Operations
operation notation components mapping
addition BAC += ijijij BAC += nmnmnm RRR ××× →×
multiplication with scalar
AC α= ijij AC α= nmnm RRR ×× →×
transpose TAC = ijij AC = mnnm RR ×× →
differentiation
ACdtd=
yx
C∂∂=
ijij Adtd
C =
j
iij y
xC
∂∂=
n×× → mnm RR n×→ mnm RR,R
matrix multiplication xAy ⋅=
BAC ⋅=
∑=k
kiki xAy
∑=k
kjikij BAC
mnnm RRR →×× p××× →× mpnnm RRR
scalar product (dot product, inner p.)
yx ⋅=α ∑=αk
kk yx RRR nn →×
vector product yxA = jiij yxA = nmnm RRR ×→×
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M7.2
Basic Rules
addition: CBACBA ++=++ )()(
ABBA +=+
multiplication with scalar: )()()( BABABA α⋅=⋅α=⋅α
BABA α+α=+α )(
transpose: AA =TT )(
TTT)( BABA +=+
TT)( AA α=α
TTT)( ABBA ⋅=⋅
differentiation: BABAdtd
dtd
)(dtd +=+
⋅+⋅
=⋅ BABABAdtd
dtd
)(dtd
dt
d)(
dtd y
yx
yx ⋅∂∂=
matrix multiplication: CABACBA ⋅+⋅=+⋅ )(
CBACBA ⋅⋅=⋅⋅ )()(
ABBA ⋅≠⋅ in general
scalar produkt: xyyx ⋅=⋅
xxx ∀≥⋅ 0 , 0xxx =⇔=⋅ 0
yxyx ,0 ⇔=⋅ orthogonal
Quadratic Matrices
identity matrix
=10
01
L
MOM
L
E
diagonal matrix { }
==
n
1
i
D0
0D
D
L
MOM
L
diagD
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M7.3
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M8
Deterministic optimization strategies
Optimization algorithms are iterative and efficient strategies work in two-steps:
initial design )0(,0i p =
evaluation of the performance of the current
design )(ip
propose a better design
1. search direction )i(s
2. line search )i(α
no
performance satisfying?
yes design “optimal”
)i()i()1i( spp α+=+
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M9
Deterministic Optimization Strategies
optimization strategy
search direction
model order
information order
search parallel to the axes hmod
)(ν
ν = es
gradient based method
)()( f νν ∇−=s
conjugate gradient method
)0()0( f∇−=s
)(2)(
2)1()1()1(
f
ff ν
ν
+ν+ν+ν
∇
∇+∇−= ss
Newton method ( ) )(1)(2)( ff ν−νν ∇⋅∇−=s
Example: quadratic criteria function
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M10
Line Search
possible requirements for line search
• exact minimization:
0f)(f!
)1v()v()v( =∇⋅=α′ +s
- many function evaluations � inefficient
• sufficient improvement:
in order to avoid infinitesimally small improvements, some conditions have been proposed, e.g. Wolfe-Powell conditions
1.0.g.e),1,(),0(f)(f
01.0.g.e),1,0(),0(f)0(f)(f!
!
=σρ∈σ′σ≥α′
=ρ∈ρ′αρ+≤α
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M11.1
Simulated Annealing
basic algorithm acceptance function cooling velocity
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M11.2
generation probability
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M12
Optimization by a Stochastic Evolution Strategy
from: P. Eberhard, F. Dignath, L. Kübler: Parallel Evolutionary Optimization of Multibody Systems with Application to Railway Dynamics, Vol. 9, No 2, 2003, pp. 143–164.
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M13
Particle Swarm Optimization
simulation of social behavior of bird flock (introduced by Kennedy & Eberhart in 1995) recursive update equation algorithm
k
ip position of particle i at time k
k
i∆p velocity of particle i at time k
]1,0[Ur,r 21 ∈ evenly distributed numbers
21 c,c,w control parameters
k
ip
1k
i
+∆p
k,best
ip
k,best
swarmp
1k
i
k
i
1k
i
++ ∆+= ppp
( ) ( )k
i
k,best
swarm
ki,22
k
i
k,best
i
ki,11
k
i
1k
ircrcw pppppp −+−+∆=∆ +
tradition/ inertia
learning social behaviour
terminate?
initialization
recursive update equation
find best particle and
best solution k,best
swarmp
k,best
swarmp
yes
no
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M14
Karush–Kuhn–Tucker Conditions
If *p is a regular point and a local minimizer of the optimization problem
)(fmin pp P∈
with { ,)(,)(h 0ph0pgp ≤=∈= RP
}mhh , RRRR →→ :h:g l ,
then Lagrange multipliers *λ and *µ exist, for which *p , *λ , *µ fulfill the following
conditions
0ppp
=∂∂
µ−∂∂λ−
∂∂
∑∑==
m
1j
jj
1i
ii
hgf l
0pg =)(
0ph ≤)(
0µ ≤
0)(h jj =µ p , m)1(1j =
If we introduce the Lagrange function
∑∑==
µ−λ−=m
1jjj
1iii )(h)(g)(f),,(L ppp:µλp
l
we can write the Karush–Kuhn–Tucker conditions as follows
0p
=∂∂ L
, 0λ
=∂∂ L
, 0µ
≥∂∂ L
,
0µ ≤ , 0)(h jj =µ p , m)1(1j =
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M15.1
Here simplifying assumption: only equality constraints
})(|{Pwith)(fmin h
P0pgpp
p=∈=
∈R
Karush-Kuhn-Tucker Condition (KKT) for minimizer ∗p
=
∇−∇=
∂∂∂∂
= ∗
∗∗∗ ∑
0
0
pg
pp
λ
pλpa
)(
)(g)(f
L
L
),( ii λ
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M15.2
)(T
)()1(
)()1(
)(T
ii22
fgf
ν
νν
νν
ν
λ
⋅
∂∂
−∇−=
−−⋅
∂∂
∂∂
−∇−∇
+
+∑
g
λp
g
λλ
pp
0p
gp
g
∇−=
−⋅
∂∂
∂∂
+
+
)(
)(
)1(
)1(
)(
T)()(
fν
ν
ν
ν
ν
νν
δgλ
p
0p
g
p
gW
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M15.3
In the case that the performance function and constraint equations are general nonlinear functions the parameter variation pδ is not necessarily the best possible parameter varia-
tion for the original optimization problem. In order to achieve a higher flexibility the method can be combined with a line search, sp α=δ .
∇−=
−⋅
∂∂
∂∂
)(
)(
)(
T)()(
fν
ν
ν
νν
gλ
s
0p
g
p
gW
equivalent to
=+⋅∂∂
∈=⋅∇+⋅⋅∈
0gsp
gsssWs
s|Swithf
2
1min h)()(
SRνν
Institute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. P. Eberhard Jun.-Prof. Dr.-Ing. R. Seifried WT 12/13 M16
Comparison of Various Deterministic Optimization
Algorithms for Nonlinear Constrained Optimization Problems
name algo-
rithm author availability mean number of function calls percent-
age of failure
f ig , jh f∇ ji h,g ∇∇ [%]
SUMT SUMT McCormick et al.
2335 24046 99 1053 69.9
NLP SUMT Rufer 1043 8635 111 957 15.6
VF02AD SQP Powell Harwell Subroutine Library
16 179 16 179 6.2
NLPQL (NCONF, NCONG)
SQP Schittkowski IMSL– Library
18 181 16 64 3.3
The results are based on 240 test runs with 80 test problems, using 3 different initial parameters for each problem.
