Post on 02-Jan-2016
transcript
WB1440 Engineering Optimization – Concepts and Applications
Engineering OptimizationConcepts and Applications
Fred van Keulen
Matthijs Langelaar
CLA H21.1
A.vanKeulen@tudelft.nl
WB1440 Engineering Optimization – Concepts and Applications
Contents
● Optimization problem checking and simplification
● Model simplification
WB1440 Engineering Optimization – Concepts and Applications
Model simplification
● Basic idea:
Expensivemodel
Optimizer
Cheapmodel
Optimizer
● Motivation:
– Replacement of expensive function, evaluated many times
– Interaction between different disciplines
– Estimation of derivatives
– Noise
WB1440 Engineering Optimization – Concepts and Applications
Model simplification (2)
● Drawback: loss of accuracy
● Different ranges: local, mid-range, global
● Synonyms:
– Approximation models
– Metamodels
– Surrogate models
– Compact models
– Reduced order models
Extractinformation
Constructapproximation
Procedure:
WB1440 Engineering Optimization – Concepts and Applications
Model simplification (3)
● Information extraction: linked to techniques from physical experiments: “plan of experiments” / DoE
● Many approaches! Covered here:
– Taylor series expansions
– Exact fitting
– Least squares fitting (response surface techniques)
– Kriging
– Reduced basis methods
– Briefly: neural nets, genetic programming, simplified physical models
● Crucial: purpose, range and level of detail
WB1440 Engineering Optimization – Concepts and Applications
Taylor series expansions
● Approximation based on local information:
21 )(''!2
1)('
!1
1)()( hxfhxfxfhxf
0
)( )(!
1
n
nn hxfn
N
n
nn hxfn0
)( )(!
1Truncation error!
● Use of derivative information!
● Valid in neighbourhood of x
)()(!
1
0
)( NN
n
nn hohxfn
WB1440 Engineering Optimization – Concepts and Applications
Taylor approximation example
1st order2nd order3rd order4th order5th order20th order
3
)5/cos(
51
52/
xx
ef
x
FunctionApproximation(x = 20)
x
WB1440 Engineering Optimization – Concepts and Applications
Exact fitting (interpolation)
● # datapoints = # fitting parameters
● Every datapoint reproduced exactly
● Example:
xaaf 10
2
1
1
0
2
1
1
1
f
f
a
a
x
x
x1 x2
f2
f1
WB1440 Engineering Optimization – Concepts and Applications
Exact fitting (2)
● Easy for intrinsically linear functions:
● No smoothing / filtering / noise reduction
● Danger of oscillations with high-order polynomials
n
iii fafafafaf
1221100
● Often used: polynomials, generalized polynomials:
1 2 1 2log( ) log log logm nf a bx x f a b m x n x
WB1440 Engineering Optimization – Concepts and Applications
9th orderpolynomial
Oscillations
● Referred to as “Runge phenomenon”
● In practice: use order 6 or less
2
1
1 25x5th order9th order
WB1440 Engineering Optimization – Concepts and Applications
Least squares fitting
● Less fitting parameters than datapoints
● Smoothing / filtering behaviour
● “Best fit”? Minimize sum of deviations:
N
iii xfxf
1
|)(~
)(|mina
● “Best fit”? Minimize sum of squared deviations:
N
iii xfxf
1
2)(
~)(min
a
x
f
f~
WB1440 Engineering Optimization – Concepts and Applications
Least squares fitting (2)
● Choose fitting function linear in parameters ai :
)()()()()(~
221100 xfaxfaxfaxfaxf mm
NmNmNNN
m
m
m
Na
a
a
a
xfxfxfxf
xfxfxfxf
xfxfxfxf
xfxfxfxf
xf
xf
xf
xf
2
1
0
2
1
0
210
2222120
1121110
0020100
2
1
0
)()()()(
)()()()(
)()()()(
)()()()(
)(~
)(~
)(~
)(~
εMaf ~
● Short notation:
WB1440 Engineering Optimization – Concepts and Applications
LS fitting (3)
● Minimize sum of squared errors:
(Optimization problem!)
MafMafεεa
~~
minTTL
MaMfMMafM0a
TTTL2
~2
~2
fMMMafMMaM~~ 1 TTTT
WB1440 Engineering Optimization – Concepts and Applications
Polynomial LS fitting
● Polynomial of degree m:
mmxaxaxaxaxf 2
21
10
0)(~
mmm
mmm
m
m
m
ma
a
a
a
xxx
xxx
xxx
xxx
xf
xf
xf
xf
2
1
0
2
1
0
2
22
22
12
11
02
00
2
1
0
1
1
1
1
)(~
)(~
)(~
)(~
mii
ii
ii
i
mm
imi
mi
mi
miiii
miiii
miii
xf
xf
xf
f
a
a
a
a
xxxx
xxxx
xxxx
xxxN
~
~
~
~
22
1
0
221
2432
132
2
WB1440 Engineering Optimization – Concepts and Applications
Polynomial LS example
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.2
0
0.2
0.4
0.6
0.8
1
1.2
samples
quadratic 6th degree
WB1440 Engineering Optimization – Concepts and Applications
Multidimensional LS fitting
● Polynomial in multiple dimensions:
ii fa
xyayxayaxa
xyayaxa
yaxaayxf
29
28
37
36
52
42
3
210),(~
● Number of coefficients ai for quadratic polynomial in Rn:
Curse of dimensionality!
)2)(1(2
1 nnm
WB1440 Engineering Optimization – Concepts and Applications
Fractional factorial design
Response surface
● Generate datapoints through sampling:
– Generate design points through Design of Experiments
– Evaluate responses
● Fit analytical model
● Check accuracy
2n full factorial designx1
x2
x3
WB1440 Engineering Optimization – Concepts and Applications
Latin Hypercube Sampling (LHS)
● Popular method: LHS
● Based on idea of Latin square:
● Properties:
– Space-filling
– Any number of design points
– Intended for box-like domains
– Matlab: lhsdesign0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
WB1440 Engineering Optimization – Concepts and Applications
(LS) Fit quality indicators● Accuracy? More / fewer terms?
● Examine the residuals
– Small
– Random! xi
Okay: >0.6
Okay: >>1
2
2
2
~1ff
Ri
i
1
~
2
2
mN
mffF
i
i
● Statistical quality indicators:
– R2 correlation measure:
– F-ratio (signal to noise):
WB1440 Engineering Optimization – Concepts and Applications
Nonlinear LS
● Linear LS: intrinsically linear functions (linear in ai):
paTxaxaxaxf 22
11
00)(
paTx xaxaeaxf 210 log)(
● Nonlinear LS: more complicated functions of ai:
● More difficult to fit! (Nonlinear optimization problem)
● Matlab: lsqnonlin
1)(
221
0
xaxa
xaxf
WB1440 Engineering Optimization – Concepts and Applications
LS pitfalls
● Scattered data:
● Wrong choice of
basis functions:
x
f
x
f
WB1440 Engineering Optimization – Concepts and Applications
Kriging● Named after D.C. Krige, mining engineer, 1951
● Statistical approach: correlation between neighbouring points
– Interpolation by weighted sum:
– Weights depend on distance
– Certain spatial correlationfunction is assumed(usually Gaussian)
N
iiii yxxxy
1
),()(
WB1440 Engineering Optimization – Concepts and Applications
Kriging properties
● Kriging interpolation is “most likely” in some sense (based on assumptions of the method)
● Interpolation: no smoothing / filtering
● Many variations exist!
● Advantage: no need to assume form of interpolation function
● Fitting process more elaborate than LS procedure
WB1440 Engineering Optimization – Concepts and Applications
Kriging example
● Results depend strongly on statistical assumptions and
method used:
Dataset z(x,y) Kriging interpolation
WB1440 Engineering Optimization – Concepts and Applications
Reduced order model
● Idea: describing system in reduced basis:
– Example: structural dynamics
fwKwMfKuuM~~~
● Select small number of “modes” to build basis
– Example: eigenmodes
WB1440 Engineering Optimization – Concepts and Applications
Reduced order model (2)
● Reduced basis:
Bwu kωωωB 21
Nk
iiiw
1
ωu
Bwu
fBf
KBBK
MBBM
T
T
T
~
~
~
fKBwwMB
fKuuM
● Reduced system equations:N1 Nk k1
fBKBwBwMBB TTT
kN NkNN
N1kN
WB1440 Engineering Optimization – Concepts and Applications
Reduced order models● Many approaches!
– Selection of type and number of basis vectors
– Dealing with nonlinearity / multiple disciplines
● Active research topic
● No interpolation / fitting, but approximate modeling
WB1440 Engineering Optimization – Concepts and Applications
Aerodynamic model
Example:Aircraft model
Structural model
Mass model
WB1440 Engineering Optimization – Concepts and Applications
(input)
output
Neural nets
To determine internal neuron parameters, neural nets must be trained on data.
x f(x)
WB1440 Engineering Optimization – Concepts and Applications
Neural net features● Versatile, can capture complex behavior
● Filtering, smoothing
● Many variations possible
– Network
– Number of neurons, layers
– Transfer functions
● Many training steps might be required (nonlinear optimization)
● Matlab: see e.g. nndtoc
WB1440 Engineering Optimization – Concepts and Applications
Genetic programming
● Building mathematical functions using
evolution-like approach
● Approach good fit by crossover and
mutation of expressions
^2
+
/
x2
x3
x1
2
32
1
x
x
x
WB1440 Engineering Optimization – Concepts and Applications
Genetic programming
● LS fitting with population of analytic expressions
● Selection / evolution rules
● Features:
– Can capture very complex
behavior
– Danger of artifacts /
overfitting
– Quite expensive procedure
WB1440 Engineering Optimization – Concepts and Applications
Simplified physical models
● Goal: capture trends from underlying physics through
simpler model:
– Lumped / Analytic / Coarse
● Parameters fitted to “high-fidelity” data
Simplified model
Correctionfunction
x f(x)
● Refinement: correction function, parameter functions ...
WB1440 Engineering Optimization – Concepts and Applications
Model simplification summary
Many different approaches:
– Local: Taylor series (needs derivatives)
– Interpolation (exact fit):
(Polynomial) fitting
Kriging
– Fitting: LS
– Approximate modeling: reduced order / simplified models
– Other: genetic programming, neural nets, etc
WB1440 Engineering Optimization – Concepts and Applications
Response surfaces in optimization● Popular approach for
computationally expen-sive problems:
1. DoE, generate samples (expensive) in part of domain
2. Build response surface (cheap)
3. Perform optimization on response surface (cheap)
4. Update domain of interest, and repeat
Expensivemodel
Optimizer
Cheapmodel
Optimizer
● Additional advantage: smoothens noisy responses
● Easy to combine with parallel computing
WB1440 Engineering Optimization – Concepts and Applications
Example: Multi-point
Approximation Method
Trust region
Design domain
Response surface
Sub-optimal point
Optimum
(Expensive) simulation