1

Comparing BIO Algorithms Under Meta-Optimization

Josh Jung

2

Outline

● Problem Overview● Review of Algorithms● Meta-Optimization● Experimental Methods● Results● Conclusions

3

The Optimization Problem

● Exploration(Diversification)

● Exploitation(Intensification)

(one last time)

4

Why so many Biologically-Inspired Algorithms?

5

Why so many Biologically-Inspired Algorithms?

6

Why so many Biologically-Inspired Algorithms?

7

Xin-She Yang’s BIO Algorithms

8

Xin-She Yang’s BIO Algorithms

9

Cuckoo AlgorithmHow is the initial population generated?Uniformly at random.

How exactly does a Lévy Flight work?As described, not as written.

How are new nests built?

Wasn’t this supposed to be the probability of discovery?

10

Firefly Algorithm

α (randomization factor) decays every iteration

Attractiveness calculation changed slightly

11

Particle Swarm Optimization (PSO)

● Positions are updated by velocities

● Velocities are updated by:– Vector to personal best

solution

– Vector to global best solution

12

Candidate BIO Algorithms: Overview

Particle Swarm Optimization

Cuckoo Alg.: Matlab

Cuckoo Alg.: Paper

Firefly Alg.: Matlab

Firefly Alg.: Paper

Firefly Algorithm

Cuckoo Algorithm

13

Meta-Optimization

14

Random Search

15

Grid Search

16

Grid Search

17

Grid Search

18

Optimizers Optimizing Optimizers

Problem Solution

Optimizer

Problem Solution

Problem Solution

Problem Solution

Optimizer

Problem Solution

Problem Solution

Meta-Optimizer

19

Optimizers Optimizing Optimizers

Problem Solution

Optimizer

Problem Solution

Problem Solution

Problem Solution

Optimizer

Problem Solution

Problem Solution

Meta-Optimizer

Meta-Meta-Optimizer

20

Optimizers Optimizing Optimizers

Problem Solution

Optimizer

Problem Solution

Problem Solution

Problem Solution

Optimizer

Problem Solution

Problem Solution

Meta-Optimizer

Meta-Meta-Optimizer

Meta-Meta-Meta-Optimizer

21

Test Functions

Michalewicz’s Function

22

Test Functions

Michalewicz’s Function

23

Test Functions

Easom’s Function De Jong’s Function

24

Test Functions

Rosenbrock’s Function Ackley’s Function

25

Experimental Methods

1) Write everything in Python

26

Experimental Methods

2) Run combinations of optimizers and meta- optimizers on all test functions

Meta-Optimizers● Grid● Random● Itself

Optimizers● PSO● Cuckoo: Matlab● Cuckoo: Paper● Firefly: Matlab● Firefly: Paper

27

Experimental Methods

3) Do repeated runs for best parameter settings

28

Results

29

Results: Success Rate

30

Results: Mean Evaluations

31

Results: Mean Runtime

32

(Preliminary) Conclusions

● Algorithms from papers << Algorithms from Matlab code

● Other algorithms are (at best) comparable to PSO

33

(Preliminary) Conclusions

● Algorithms from papers << Algorithms from Matlab code

● Other algorithms are (at best) comparable to PSO

Remaining Work● Finish tests for Firefly Algorithm● Publish code to GitHub

34

References

● Pohlheim, H. "Geatbx examples examples of objective functions (2006)." URL http://www. geatbx. com/download/GEATbx_ObjFunExpl_v37. pdf.

● Yang, Xin-She. "Firefly algorithms for multimodal optimization." International symposium on stochastic algorithms. Springer Berlin Heidelberg, 2009.

● Yang, Xin-She, and Suash Deb. "Cuckoo search via Lévy flights." Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on. IEEE, 2009.

35

Additional Image Sources● http://www.geatbx.com/download/GEATbx_ObjFunExpl_v37.pdf● http://smaree.com/2016/09/30/global-minimum-of-the-michalewicz-function/● https://www.theodysseyonline.com/hamster-wheels-river-styx● https://www.python.org/● http://www.digitalspacedigitalform.com/introduction-to-2d-autocad/● Pedersen, M.E.H., Tuning & Simplifying Heuristical Optimization, PhD Thesis, 2010, University of Southampton,

School of Engineering Sciences, Computational Engineering and Design Group. - Public Domain, https://commons.wikimedia.org/w/index.php?curid=9775862

● http://dsdeepdive.blogspot.com/2016/03/optimizations-of-gradient-descent.html● https://commons.wikimedia.org/w/index.php?curid=1215127● https://www.flickr.com/photos/kachnch/16364273038, CC BY 2.0, https://commons.wikimedia.org/w/index.php?

curid=38480628● Kevin Pluck - Flickr: The King., CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=755560● Quit007, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1433821● http://www.blm.gov/nv/st/en.html, Public Domain, https://commons.wikimedia.org/w/index.php?curid=192812● Aaron Siirila, CC BY-SA 2.5, https://commons.wikimedia.org/w/index.php?curid=3792946● Stephen Ausmus - https://commons.wikimedia.org/w/index.php?curid=10112924● https://en.wikipedia.org/wiki/Genetics#/media/File:DNA_Overview2.png● Alastair Rae, CC BY-SA 2.0, https://commons.wikimedia.org/w/index.php?curid=2229241

36

Results: Mean Fitness Values