Evolutionary and Swarm-inspired

Heuristics in Optimization

Marcel Kronfeld

Andreas Zell

• Differential Evolution (DE)

• History

• Mechanism

• Particle Swarm Optimization (PSO)

• Idea: “Swarm Intelligence”

• Constriction variant

• Some comparisons

• Some advanced topics

Overview

2M. Kronfeld, A. Zell

• Given a population of vectors

• To mutate , select 3 individuals randomly

• Calculate trial vector

• Discrete recombination:

• Replace with only if it is an improvement

DE Basic Mechanism

x1... x

v xr1 c x

r2 xr3

xi

u recDiscr xi, v , pc

xi

u

xi

xr

3

xr

1

xr

2

c xr

2− xr

3

xi

v v

xr

1

3M. Kronfeld, A. Zell

1 init(P(0)= )

2 t:=0

3 while !terminated(P(t))

4 for i:=1 to do

5 v:=trialVector(P(t),scheme, )

6 u:=recombDiscrete( ,v, )

7 if evaluate(u) < evaluate( ) then

8 :=u

9 end if

10 end for

11 end while

• Several schemes have been proposed, e.g.:

• DE/rand/1

• DE/best/1

• DE/best/2

• DE/current-to-best/1

DE Pseudo Code

c

pc

v xr1 c x

r2 xr3

v xbest

c xr1 x

r2

v xbest

c xr1 x

r2 xr3 x

r4

v xri c1 x

bestx

ic2 x

r1 xr2

x1, ... , x

xi

xi

xi

4M. Kronfeld, A. Zell

• Strategy parameters min.:

• Usually

• Population size: e.g. 100

• Trial vectors

• Individual differences

• Vectorial subspace defined by current population

• Length correlated with extensions per dimension

• Simple calculations

• Won the 1st Int. EC challenge in 1996

DE Properties

c , pc

c 0.5,1 , pc 0.8

t0

t1

5M. Kronfeld, A. Zell

• Rastrigin’s (F6, left) and Ackley’s (F8, right)

• Both highly multi-modal

• Runs in 10D

Two Benchmarks

6M. Kronfeld, A. Zell

• Best/2, rand/1, current-to-best/1

DE Comparison

rand/1

current-to-best/1

best/2

rand/1

current-to-best/1

best/2

7M. Kronfeld, A. Zell

• Differential Evolution (DE)

• History

• Mechanism

• Particle Swarm Optimization (PSO)

• Idea: “Swarm Intelligence”

• Constriction variant

• Some comparisons

• Some advanced topics

Overview

8M. Kronfeld, A. Zell

PSO Idea and Mechanism

• Kennedy & Eberhart `95

• Swarm analogy

• Individuals move with velocity

• Individuals store and exchange information

• Historic (personal) best pos.

• Neighborhood best position

xn

v k t 1 vk t 1 r1 x k x k 2 r 2 xk x k

xk t 1 x k t v k t 1 1 k n , r 1 , r2 U 0,1

x t argmin f x t ' t ' t

x t argmin f x j t x j N x

• Strategy parameters: , 1 , 2 , e.g.0.8,1.5,1.5

vn

9M. Kronfeld, A. Zell

PSO Visualized

• Neighborhood topologies:

• Neighbours: all particles within distance k on a topology

• Grid , star , linear , random, etc.

• The of all neighbors is a particle'sx x

xi

xj

xi

xj

xk

vi

vj

xi

xj

xi

xi

vi

t

1 21

vi

t 1

Area of attraction:

a (hyper-)parallelepiped

Pseudo Code: PSO

1 init

2 t:=0

3 while !terminated(P(t))

4 for i:=1 to size(P(t)) do

5 for k:=1 to n do

6

7

8 end for

9 if evaluate( ) < evaluate( ) then

10

11 end if

12 end for

13 end while

v k

i: v k

i

1 r1 x k

ix k

i

2 r 2 x k

ix k

i

xk

i: x

k

iv

k

i

xi:= x

i

P 0 x1, v

1, ... , x , v , i 1 x

ix

i

xi

xi

11M. Kronfeld, A. Zell

• No selection - particles never “die”!

• Instead personal memory & social information

• Possible interpretation: crossover between selected

individuals

• Again very simple operations

• Critical: topology and parameter settings

• Neighborhood topology directly influences convergence

• Poor parameter settings may lead to “explosion”:

additional parameter

• Typical population size: (only) 30, still good exploration

PSO Properties

vmax

12M. Kronfeld, A. Zell

• How to choose to avoid explosion?

• Clerc & Kennedy `02: Simplified PSO as a linear

dynamic system: no randomness, constant attractor p

• Behavior of can be estimated by looking at the

eigenvalues of M:

Constricted PSO I

, 1, 2, vmax

v t 1 v t p x t

x t 1 x t v t 1

v t 1 v t y t

y t 1 v t 1 y tset y t p x t

set Pv

y, M

1

1 1 P t 1 MPt Mt 1

P0

e1 12

24

2, e2 1

2

24

2

Pt

13M. Kronfeld, A. Zell

• Analyzed points of periodicity and convergence

• Result: alternative constricted formulation:

• Select such that holds but convergence

is slow

Constricted PSO II

2

22

4 , 1 2 > 4

• Thus: only 2 parameters left (plus topology)

• Default setting: grid with

• Decrease / increase connectivity for faster convergence

1 2 2.05 0.73

1 2

v k t 1 v k t 1 r1 xk xk 2 r2 x k xk

max e1 , e2 1

14M. Kronfeld, A. Zell

• Again F6 and F8

• Topologies grid, linear, star

PSO Comparisons

linear

star

grid

linear

stargrid

15M. Kronfeld, A. Zell

• Comparing realGA, (5,20)-cmaES, DE, and PSO,

• unimodal functions

• All solve F1, ES is quickest

• CMA-effect visible on F2

Cross Comparisons I

F1-Parabola F2-Rosenbrock

GA

DE

ESPSO

GA

DEES

PSO

16M. Kronfeld, A. Zell

Cross Comparisons II

• Again GA, cmaES, DE, and PSO

• cmaES is usually quickest, PSO/DE have better exploratory behavior

F8-Ackley's F13-Schwefel's

GA

DE

ES

PSO

GA

DE

ES

PSO

17M. Kronfeld, A. Zell

• Project from Bioinformatics: Inferring a metabolic network (Dräger

`09)

• Winner: DE and PSO

Even More Important Comparisons I

18M. Kronfeld, A. Zell

• The CEC-2005 Benchmark

Suite

• All solvable 10D (left)

• Multi-modal 30D (bottom)

Even More Important Comparisons II

• Winner: cmaES with

incremental restarts (IPOP)

• Lately added to EvA2:

EvolutionStrategiesIPOP

19M. Kronfeld, A. Zell

CEC `05 Competitors

20M. Kronfeld, A. Zell

• We have learnt about

• Differential Evolution

• Particle Swarm Optimization

• Comparisons

• Let's look at some advanced topics

• Multi-modal optimization

• Multi-criterial optimization

• Dynamic target functions

From Algorithms to Advanced Topics

21M. Kronfeld, A. Zell

• Usually: return only one best solution

• Sometimes we're interested in more:

• Noise and many similar optima

• Additional criteria to be considered later

• E.g.: solution robustness/sensitivity – hard to test for

• Approaches:

• Niching PSO

• Clustering EA

• Multi-population cmaES

Multi-modal Optimization

Robust

solution

Sensitive

solution

22M. Kronfeld, A. Zell

• Simple example: buy a car

• Maximize speed and minimize cost

• Two conflicting objective functions and

Multi-criterial Optimization

fSpeed

fCost

fSpeed

fCost

x1 x

2 x3

x...

dominated area

pareto front

better

b e t t e r

xi

• Definition: Pareto Front (PF)

• Approaches:

• Weighted combination of

• Approximate whole PF

• Pareto-dominance in EA-

selection, PSO-attraction, ...

fi

23M. Kronfeld, A. Zell

• What if the target function changes over time?

• Online optimization in industrial environments

• Motion, traffic, weather, new jobs/measurements...

• Approaches:

• Locate several optima (→ multi-modality)

• Keep up diversity to follow changes

Optimization in Dynamic Environments

t=0 t=100

24M. Kronfeld, A. Zell

• Constrained target functions:

• Minimize

• Interesting for nonlinear functions/constraints

• Say

Constrained Optimization I

f x subject to g i x 0

f G8 x zsin

32 x1 sin 2 x2

x1

3x 1 x2

subject to

g 1 x x 1

2x2 1 0 ,

g 2 x 1 x 1 x2 42

0

f G8 : ( 0,10 ]2

,

25M. Kronfeld, A. Zell

• Initial situation: multi-modal target function

• Possible approaches:

• Treat every constraint as an additional criterion

• → Multi-criterial target function

• Penalize infeasible individuals

• Often the penalty is increased with optimization time

• → Dynamic multi-modal target function!

Constrained Optimization II

Constant penalty Proportional penalty

26M. Kronfeld, A. Zell

• DE and PSO are relatively new and popular heuristics

• Both simple and successful (esp. in real-world apps.)

• Constricted PSO with recommended stable params.

• Latest developments:

• IPOP-CMA-ES, PS-CMA-ES (CEC 2009)

Conclusion I

difference vector → discrete recomb., replace

individuals only in case of improvementparticle memory & neighborhood bests

build attractor for particle velocities

xi

xj

xi

xi

vi

t

vi

t 1xi

xr

3

xr

1

xr

2

c xr

2 xr

3

xi

v v

xr

1

27M. Kronfeld, A. Zell

• Where it gets even more interesting:

• Multi-modality in high dimensions

• Multi-objective target functions

• Dynamic target function

• Some open questions:

• How do multi-modal techniques relate to dynamic

adaptions?

• How can they be efficiently combined, e.g., to handle

dynamic constraints?

Conclusion II

28M. Kronfeld, A. Zell

• Storn&Price `95: Rainer Storn and Kenneth V. Price. Differential Evolution -

a Simple and Efficient Adaptive Scheme for Global Optimization over

Continuous Spaces", Technical Report TR-95-012, ICSI, 1995.

• Kennedy&Eberhart `95: James Kennedy and Russel Eberhart. Particle

swarm optimization. In Proceedings of the IEEE 1995 International

Conference on Neural Networks, pp. 1942-1948, 1995.

• Feoktistov `06: Vitaliy Feoktistov. Differential Evolution: In Search of

Solutions. Springer, Berlin, 2006.

• Clerc&Kennedy `02: Maurice Clerc and James Kennedy. The particle

swarm - explosion, stability, and convergence in a multidimensional

complex space. IEEE Transactions on Evolutionary Computation 6(1), pp.

58-73, 2002.

• Dräger `09: Andreas Dräger et al.: Modeling metabolic networks in C.

glutamicum: a comparison of rate laws in combination with various

parameter optimization strategies, BMC Systems Biology 2009, 3:5, 2009.

• CEC 2005: IEEE Congress on Evolutionary Computation. CEC 2005, 2-4

September 2005, Edinburgh, UK. IEEE 2005, ISBN 0-7803-9363-5

References

29M. Kronfeld, A. Zell