Evolutionary and Swarm-inspired
Heuristics in Optimization
Marcel Kronfeld
Andreas Zell
• Differential Evolution (DE)
• History
• Mechanism
• Particle Swarm Optimization (PSO)
• Idea: “Swarm Intelligence”
• Constriction variant
• Some comparisons
• Some advanced topics
Overview
2M. Kronfeld, A. Zell
• Given a population of vectors
• To mutate , select 3 individuals randomly
• Calculate trial vector
• Discrete recombination:
• Replace with only if it is an improvement
DE Basic Mechanism
x1... x
v xr1 c x
r2 xr3
xi
u recDiscr xi, v , pc
xi
u
xi
xr
3
xr
1
xr
2
c xr
2− xr
3
xi
v v
xr
1
3M. Kronfeld, A. Zell
1 init(P(0)= )
2 t:=0
3 while !terminated(P(t))
4 for i:=1 to do
5 v:=trialVector(P(t),scheme, )
6 u:=recombDiscrete( ,v, )
7 if evaluate(u) < evaluate( ) then
8 :=u
9 end if
10 end for
11 end while
• Several schemes have been proposed, e.g.:
• DE/rand/1
• DE/best/1
• DE/best/2
• DE/current-to-best/1
DE Pseudo Code
c
pc
v xr1 c x
r2 xr3
v xbest
c xr1 x
r2
v xbest
c xr1 x
r2 xr3 x
r4
v xri c1 x
bestx
ic2 x
r1 xr2
x1, ... , x
xi
xi
xi
4M. Kronfeld, A. Zell
• Strategy parameters min.:
• Usually
• Population size: e.g. 100
• Trial vectors
• Individual differences
• Vectorial subspace defined by current population
• Length correlated with extensions per dimension
• Simple calculations
• Won the 1st Int. EC challenge in 1996
DE Properties
c , pc
c 0.5,1 , pc 0.8
t0
t1
5M. Kronfeld, A. Zell
• Rastrigin’s (F6, left) and Ackley’s (F8, right)
• Both highly multi-modal
• Runs in 10D
Two Benchmarks
6M. Kronfeld, A. Zell
• Best/2, rand/1, current-to-best/1
DE Comparison
rand/1
current-to-best/1
best/2
rand/1
current-to-best/1
best/2
7M. Kronfeld, A. Zell
• Differential Evolution (DE)
• History
• Mechanism
• Particle Swarm Optimization (PSO)
• Idea: “Swarm Intelligence”
• Constriction variant
• Some comparisons
• Some advanced topics
Overview
8M. Kronfeld, A. Zell
PSO Idea and Mechanism
• Kennedy & Eberhart `95
• Swarm analogy
• Individuals move with velocity
• Individuals store and exchange information
• Historic (personal) best pos.
• Neighborhood best position
xn
v k t 1 vk t 1 r1 x k x k 2 r 2 xk x k
xk t 1 x k t v k t 1 1 k n , r 1 , r2 U 0,1
x t argmin f x t ' t ' t
x t argmin f x j t x j N x
• Strategy parameters: , 1 , 2 , e.g.0.8,1.5,1.5
vn
9M. Kronfeld, A. Zell
PSO Visualized
• Neighborhood topologies:
• Neighbours: all particles within distance k on a topology
• Grid , star , linear , random, etc.
• The of all neighbors is a particle'sx x
xi
xj
xi
xj
xk
vi
vj
xi
xj
xi
xi
vi
t
1 21
vi
t 1
Area of attraction:
a (hyper-)parallelepiped
Pseudo Code: PSO
1 init
2 t:=0
3 while !terminated(P(t))
4 for i:=1 to size(P(t)) do
5 for k:=1 to n do
6
7
8 end for
9 if evaluate( ) < evaluate( ) then
10
11 end if
12 end for
13 end while
v k
i: v k
i
1 r1 x k
ix k
i
2 r 2 x k
ix k
i
xk
i: x
k
iv
k
i
xi:= x
i
P 0 x1, v
1, ... , x , v , i 1 x
ix
i
xi
xi
11M. Kronfeld, A. Zell
• No selection - particles never “die”!
• Instead personal memory & social information
• Possible interpretation: crossover between selected
individuals
• Again very simple operations
• Critical: topology and parameter settings
• Neighborhood topology directly influences convergence
• Poor parameter settings may lead to “explosion”:
additional parameter
• Typical population size: (only) 30, still good exploration
PSO Properties
vmax
12M. Kronfeld, A. Zell
• How to choose to avoid explosion?
• Clerc & Kennedy `02: Simplified PSO as a linear
dynamic system: no randomness, constant attractor p
• Behavior of can be estimated by looking at the
eigenvalues of M:
Constricted PSO I
, 1, 2, vmax
v t 1 v t p x t
x t 1 x t v t 1
v t 1 v t y t
y t 1 v t 1 y tset y t p x t
set Pv
y, M
1
1 1 P t 1 MPt Mt 1
P0
e1 12
24
2, e2 1
2
24
2
Pt
13M. Kronfeld, A. Zell
• Analyzed points of periodicity and convergence
• Result: alternative constricted formulation:
• Select such that holds but convergence
is slow
Constricted PSO II
2
22
4 , 1 2 > 4
• Thus: only 2 parameters left (plus topology)
• Default setting: grid with
• Decrease / increase connectivity for faster convergence
1 2 2.05 0.73
1 2
v k t 1 v k t 1 r1 xk xk 2 r2 x k xk
max e1 , e2 1
14M. Kronfeld, A. Zell
• Again F6 and F8
• Topologies grid, linear, star
PSO Comparisons
linear
star
grid
linear
stargrid
15M. Kronfeld, A. Zell
• Comparing realGA, (5,20)-cmaES, DE, and PSO,
• unimodal functions
• All solve F1, ES is quickest
• CMA-effect visible on F2
Cross Comparisons I
F1-Parabola F2-Rosenbrock
GA
DE
ESPSO
GA
DEES
PSO
16M. Kronfeld, A. Zell
Cross Comparisons II
• Again GA, cmaES, DE, and PSO
• cmaES is usually quickest, PSO/DE have better exploratory behavior
F8-Ackley's F13-Schwefel's
GA
DE
ES
PSO
GA
DE
ES
PSO
17M. Kronfeld, A. Zell
• Project from Bioinformatics: Inferring a metabolic network (Dräger
`09)
• Winner: DE and PSO
Even More Important Comparisons I
18M. Kronfeld, A. Zell
• The CEC-2005 Benchmark
Suite
• All solvable 10D (left)
• Multi-modal 30D (bottom)
Even More Important Comparisons II
• Winner: cmaES with
incremental restarts (IPOP)
• Lately added to EvA2:
EvolutionStrategiesIPOP
19M. Kronfeld, A. Zell
CEC `05 Competitors
20M. Kronfeld, A. Zell
• We have learnt about
• Differential Evolution
• Particle Swarm Optimization
• Comparisons
• Let's look at some advanced topics
• Multi-modal optimization
• Multi-criterial optimization
• Dynamic target functions
From Algorithms to Advanced Topics
21M. Kronfeld, A. Zell
• Usually: return only one best solution
• Sometimes we're interested in more:
• Noise and many similar optima
• Additional criteria to be considered later
• E.g.: solution robustness/sensitivity – hard to test for
• Approaches:
• Niching PSO
• Clustering EA
• Multi-population cmaES
Multi-modal Optimization
Robust
solution
Sensitive
solution
22M. Kronfeld, A. Zell
• Simple example: buy a car
• Maximize speed and minimize cost
• Two conflicting objective functions and
Multi-criterial Optimization
fSpeed
fCost
fSpeed
fCost
x1 x
2 x3
x...
dominated area
pareto front
better
b e t t e r
xi
• Definition: Pareto Front (PF)
• Approaches:
• Weighted combination of
• Approximate whole PF
• Pareto-dominance in EA-
selection, PSO-attraction, ...
fi
23M. Kronfeld, A. Zell
• What if the target function changes over time?
• Online optimization in industrial environments
• Motion, traffic, weather, new jobs/measurements...
• Approaches:
• Locate several optima (→ multi-modality)
• Keep up diversity to follow changes
Optimization in Dynamic Environments
t=0 t=100
24M. Kronfeld, A. Zell
• Constrained target functions:
• Minimize
• Interesting for nonlinear functions/constraints
• Say
Constrained Optimization I
f x subject to g i x 0
f G8 x zsin
32 x1 sin 2 x2
x1
3x 1 x2
subject to
g 1 x x 1
2x2 1 0 ,
g 2 x 1 x 1 x2 42
0
f G8 : ( 0,10 ]2
,
25M. Kronfeld, A. Zell
• Initial situation: multi-modal target function
• Possible approaches:
• Treat every constraint as an additional criterion
• → Multi-criterial target function
• Penalize infeasible individuals
• Often the penalty is increased with optimization time
• → Dynamic multi-modal target function!
Constrained Optimization II
Constant penalty Proportional penalty
26M. Kronfeld, A. Zell
• DE and PSO are relatively new and popular heuristics
• Both simple and successful (esp. in real-world apps.)
• Constricted PSO with recommended stable params.
• Latest developments:
• IPOP-CMA-ES, PS-CMA-ES (CEC 2009)
Conclusion I
difference vector → discrete recomb., replace
individuals only in case of improvementparticle memory & neighborhood bests
build attractor for particle velocities
xi
xj
xi
xi
vi
t
vi
t 1xi
xr
3
xr
1
xr
2
c xr
2 xr
3
xi
v v
xr
1
27M. Kronfeld, A. Zell
• Where it gets even more interesting:
• Multi-modality in high dimensions
• Multi-objective target functions
• Dynamic target function
• Some open questions:
• How do multi-modal techniques relate to dynamic
adaptions?
• How can they be efficiently combined, e.g., to handle
dynamic constraints?
Conclusion II
28M. Kronfeld, A. Zell
• Storn&Price `95: Rainer Storn and Kenneth V. Price. Differential Evolution -
a Simple and Efficient Adaptive Scheme for Global Optimization over
Continuous Spaces", Technical Report TR-95-012, ICSI, 1995.
• Kennedy&Eberhart `95: James Kennedy and Russel Eberhart. Particle
swarm optimization. In Proceedings of the IEEE 1995 International
Conference on Neural Networks, pp. 1942-1948, 1995.
• Feoktistov `06: Vitaliy Feoktistov. Differential Evolution: In Search of
Solutions. Springer, Berlin, 2006.
• Clerc&Kennedy `02: Maurice Clerc and James Kennedy. The particle
swarm - explosion, stability, and convergence in a multidimensional
complex space. IEEE Transactions on Evolutionary Computation 6(1), pp.
58-73, 2002.
• Dräger `09: Andreas Dräger et al.: Modeling metabolic networks in C.
glutamicum: a comparison of rate laws in combination with various
parameter optimization strategies, BMC Systems Biology 2009, 3:5, 2009.
• CEC 2005: IEEE Congress on Evolutionary Computation. CEC 2005, 2-4
September 2005, Edinburgh, UK. IEEE 2005, ISBN 0-7803-9363-5
References
29M. Kronfeld, A. Zell