1
Comparing BIO Algorithms Under Meta-Optimization
Josh Jung
2
Outline
● Problem Overview● Review of Algorithms● Meta-Optimization● Experimental Methods● Results● Conclusions
3
The Optimization Problem
● Exploration(Diversification)
● Exploitation(Intensification)
(one last time)
4
Why so many Biologically-Inspired Algorithms?
5
Why so many Biologically-Inspired Algorithms?
6
Why so many Biologically-Inspired Algorithms?
7
Xin-She Yang’s BIO Algorithms
8
Xin-She Yang’s BIO Algorithms
9
Cuckoo AlgorithmHow is the initial population generated?Uniformly at random.
How exactly does a Lévy Flight work?As described, not as written.
How are new nests built?
Wasn’t this supposed to be the probability of discovery?
10
Firefly Algorithm
α (randomization factor) decays every iteration
Attractiveness calculation changed slightly
11
Particle Swarm Optimization (PSO)
● Positions are updated by velocities
● Velocities are updated by:– Vector to personal best
solution
– Vector to global best solution
12
Candidate BIO Algorithms: Overview
Particle Swarm Optimization
Cuckoo Alg.: Matlab
Cuckoo Alg.: Paper
Firefly Alg.: Matlab
Firefly Alg.: Paper
Firefly Algorithm
Cuckoo Algorithm
13
Meta-Optimization
14
Random Search
15
Grid Search
16
Grid Search
17
Grid Search
18
Optimizers Optimizing Optimizers
Problem Solution
Optimizer
Problem Solution
Problem Solution
Problem Solution
Optimizer
Problem Solution
Problem Solution
Meta-Optimizer
19
Optimizers Optimizing Optimizers
Problem Solution
Optimizer
Problem Solution
Problem Solution
Problem Solution
Optimizer
Problem Solution
Problem Solution
Meta-Optimizer
Meta-Meta-Optimizer
20
Optimizers Optimizing Optimizers
Problem Solution
Optimizer
Problem Solution
Problem Solution
Problem Solution
Optimizer
Problem Solution
Problem Solution
Meta-Optimizer
Meta-Meta-Optimizer
Meta-Meta-Meta-Optimizer
21
Test Functions
Michalewicz’s Function
22
Test Functions
Michalewicz’s Function
23
Test Functions
Easom’s Function De Jong’s Function
24
Test Functions
Rosenbrock’s Function Ackley’s Function
25
Experimental Methods
1) Write everything in Python
26
Experimental Methods
2) Run combinations of optimizers and meta- optimizers on all test functions
Meta-Optimizers● Grid● Random● Itself
Optimizers● PSO● Cuckoo: Matlab● Cuckoo: Paper● Firefly: Matlab● Firefly: Paper
27
Experimental Methods
3) Do repeated runs for best parameter settings
28
Results
29
Results: Success Rate
30
Results: Mean Evaluations
31
Results: Mean Runtime
32
(Preliminary) Conclusions
● Algorithms from papers << Algorithms from Matlab code
● Other algorithms are (at best) comparable to PSO
33
(Preliminary) Conclusions
● Algorithms from papers << Algorithms from Matlab code
● Other algorithms are (at best) comparable to PSO
Remaining Work● Finish tests for Firefly Algorithm● Publish code to GitHub
34
References
● Pohlheim, H. "Geatbx examples examples of objective functions (2006)." URL http://www. geatbx. com/download/GEATbx_ObjFunExpl_v37. pdf.
● Yang, Xin-She. "Firefly algorithms for multimodal optimization." International symposium on stochastic algorithms. Springer Berlin Heidelberg, 2009.
● Yang, Xin-She, and Suash Deb. "Cuckoo search via Lévy flights." Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on. IEEE, 2009.
35
Additional Image Sources● http://www.geatbx.com/download/GEATbx_ObjFunExpl_v37.pdf● http://smaree.com/2016/09/30/global-minimum-of-the-michalewicz-function/● https://www.theodysseyonline.com/hamster-wheels-river-styx● https://www.python.org/● http://www.digitalspacedigitalform.com/introduction-to-2d-autocad/● Pedersen, M.E.H., Tuning & Simplifying Heuristical Optimization, PhD Thesis, 2010, University of Southampton,
School of Engineering Sciences, Computational Engineering and Design Group. - Public Domain, https://commons.wikimedia.org/w/index.php?curid=9775862
● http://dsdeepdive.blogspot.com/2016/03/optimizations-of-gradient-descent.html● https://commons.wikimedia.org/w/index.php?curid=1215127● https://www.flickr.com/photos/kachnch/16364273038, CC BY 2.0, https://commons.wikimedia.org/w/index.php?
curid=38480628● Kevin Pluck - Flickr: The King., CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=755560● Quit007, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1433821● http://www.blm.gov/nv/st/en.html, Public Domain, https://commons.wikimedia.org/w/index.php?curid=192812● Aaron Siirila, CC BY-SA 2.5, https://commons.wikimedia.org/w/index.php?curid=3792946● Stephen Ausmus - https://commons.wikimedia.org/w/index.php?curid=10112924● https://en.wikipedia.org/wiki/Genetics#/media/File:DNA_Overview2.png● Alastair Rae, CC BY-SA 2.0, https://commons.wikimedia.org/w/index.php?curid=2229241
36
Results: Mean Fitness Values