Mirrored Variants of the (1,2)-CMA-ES Compared on the Noisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop] Anne Auger, Dimo Brockhoff, and Nikolaus Hansen Projet TAO, INRIA Saclay—Ile-de-France LRI, Bât 490, Univ. Paris-Sud 91405 Orsay Cedex, France fi[email protected]ABSTRACT Derandomization by means of mirrored samples has been re- cently introduced to enhance the performances of (1,λ) and (1 + 2) Evolution-Strategies (ESs) with the aim of design- ing fast local search stochastic algorithms. In this paper, we investigate the impact of mirrored samples for noisy op- timization. Since elitist selection is detrimental for noisy optimization, we investigate non-elitist ESs only here. We compare on the BBOB-2010 noisy benchmark testbed two variants of the (1,2)-CMA-ES where mirrored samples are implemented with the baseline (1,2)-CMA-ES. Each algo- rithm implements a restart mechanism. A total budget of 10 4 D function evaluations per trial has been used, where D is the dimension of the search space. The experiments clearly show a ranking among the three algorithms: both mirroring variants have lower expected running times than the (1,2)-CMA-ES by at least 50% on 5 functions and they solve three additional functions in 20D that the (1,2)-CMA-ES cannot solve (or only with small probability). The comparison between the two mirroring variants is in favor of the algorithm employing a sequen- tial selection in addition—outperforming the algorithm with only mirrored samples on five functions by at least 17% whereas no statistically significant worsening can be ob- served. Both algorithms using mirrored samples also out- perform the function-wise best algorithm of the BBOB-2009 benchmarking on three (respectively four) functions com- prising Cauchy noise by up to 65%. Categories and Subject Descriptors G.1.6 [Numerical Analysis]: Optimization—global opti- mization, unconstrained optimization ; F.2.1 [Analysis of Algorithms and Problem Complexity]: Numerical Al- gorithms and Problems c ACM, 2010. This is the authors’ version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published at GECCO’10, July 7–11, 2010, Portland, OR, USA. http://doi.acm.org/10.1145/1830761.1830775 General Terms Algorithms Keywords Benchmarking, Black-box optimization 1. INTRODUCTION Evolution Strategies (ESs) are robust stochastic search al- gorithms for black-box optimization where the function to be minimized, f , maps the continuous search space R D into R. ESs evolve a population of candidate solutions that are created by sampling λ independent random vectors following a multivariate normal distribution. Recently, a new deran- domization technique replacing the independent sampling of new solutions (or offspring) by mirrored samples has been introduced to enhance the performances of ESs [1]. With mirrored sampling, a single sample N of a multivariate nor- mal distribution is used for two offspring of the same itera- tion. Denoting X the current solution, the two offspring will equal X +N and X -N respectively. The resulting offspring are thus symmetric or mirrored with respect to X and are thus negatively correlated. Mirrored samples have been im- plemented in the Covariance-Matrix-Adaptation Evolution- Strategy (CMA-ES), an ES whose characteristic is to adapt the full covariance matrix of the multivariate normal search distribution [6]. Another new concept called sequential se- lection was introduced together with mirrored samples [1]. Sequential selection, consists in performing sequential eval- uations of the offspring and breaking the evaluation loop as soon as an offspring is better than the current solution X and thus saving the remaining fitness evaluations. In this paper, we assess quantitatively the improvement that can be brought by mirrored samples and by mirrored samples coupled with sequential selection. We compare on the BBOB-2010 noisy testbed the (1,2)-CMA-ES with two variants: first the (1,2m)-CMA-ES where mirrored samples are used, and second the (1,2 s m )-CMA-ES that in addition to the mirrored samples uses sequential selection. The algo- rithms and the CPU timing experiments are described in a complementing paper in the same proceeding [2]. inria-00502436, version 1 - 14 Jul 2010 Author manuscript, published in "GECCO workshop on Black-Box Optimization Benchmarking (BBOB'2010) (2010) 1575-1582" DOI : 10.1145/1830761.1830775
Transcript
Mirrored Variants of the (1,2)-CMA-ES Compared on theNoisy BBOB-2010 Testbed
[Black-Box Optimization Benchmarking Workshop]
Anne Auger, Dimo Brockhoff, and Nikolaus HansenProjet TAO, INRIA Saclay—Ile-de-France
LRI, Bât 490, Univ. Paris-Sud91405 Orsay Cedex, France
ABSTRACTDerandomization by means of mirrored samples has been re-cently introduced to enhance the performances of (1, λ) and(1 + 2) Evolution-Strategies (ESs) with the aim of design-ing fast local search stochastic algorithms. In this paper,we investigate the impact of mirrored samples for noisy op-timization. Since elitist selection is detrimental for noisyoptimization, we investigate non-elitist ESs only here. Wecompare on the BBOB-2010 noisy benchmark testbed twovariants of the (1,2)-CMA-ES where mirrored samples areimplemented with the baseline (1,2)-CMA-ES. Each algo-rithm implements a restart mechanism. A total budget of104D function evaluations per trial has been used, where Dis the dimension of the search space.
The experiments clearly show a ranking among the threealgorithms: both mirroring variants have lower expectedrunning times than the (1,2)-CMA-ES by at least 50% on 5functions and they solve three additional functions in 20Dthat the (1,2)-CMA-ES cannot solve (or only with smallprobability). The comparison between the two mirroringvariants is in favor of the algorithm employing a sequen-tial selection in addition—outperforming the algorithm withonly mirrored samples on five functions by at least 17%whereas no statistically significant worsening can be ob-served. Both algorithms using mirrored samples also out-perform the function-wise best algorithm of the BBOB-2009benchmarking on three (respectively four) functions com-prising Cauchy noise by up to 65%.
Categories and Subject DescriptorsG.1.6 [Numerical Analysis]: Optimization—global opti-mization, unconstrained optimization; F.2.1 [Analysis ofAlgorithms and Problem Complexity]: Numerical Al-gorithms and Problems
1. INTRODUCTIONEvolution Strategies (ESs) are robust stochastic search al-
gorithms for black-box optimization where the function tobe minimized, f , maps the continuous search space RD intoR. ESs evolve a population of candidate solutions that arecreated by sampling λ independent random vectors followinga multivariate normal distribution. Recently, a new deran-domization technique replacing the independent sampling ofnew solutions (or offspring) by mirrored samples has beenintroduced to enhance the performances of ESs [1]. Withmirrored sampling, a single sample N of a multivariate nor-mal distribution is used for two offspring of the same itera-tion. Denoting X the current solution, the two offspring willequal X+N and X−N respectively. The resulting offspringare thus symmetric or mirrored with respect to X and arethus negatively correlated. Mirrored samples have been im-plemented in the Covariance-Matrix-Adaptation Evolution-Strategy (CMA-ES), an ES whose characteristic is to adaptthe full covariance matrix of the multivariate normal searchdistribution [6]. Another new concept called sequential se-lection was introduced together with mirrored samples [1].Sequential selection, consists in performing sequential eval-uations of the offspring and breaking the evaluation loop assoon as an offspring is better than the current solution Xand thus saving the remaining fitness evaluations.
In this paper, we assess quantitatively the improvementthat can be brought by mirrored samples and by mirroredsamples coupled with sequential selection. We compare onthe BBOB-2010 noisy testbed the (1,2)-CMA-ES with twovariants: first the (1,2m)-CMA-ES where mirrored samplesare used, and second the (1,2s
m)-CMA-ES that in additionto the mirrored samples uses sequential selection. The algo-rithms and the CPU timing experiments are described in acomplementing paper in the same proceeding [2].
inria
-005
0243
6, v
ersi
on 1
- 14
Jul
201
0Author manuscript, published in "GECCO workshop on Black-Box Optimization Benchmarking (BBOB'2010) (2010) 1575-1582"
2.1 Comparing (1,2)- and (1,2m)-CMA-ESResults from experiments comparing (1,2)-CMA-ES and
(1,2m)-CMA-ES according to [4] on the benchmark functionsgiven in [3, 5] are presented in Figures 1 and 2 and in Table 1.The expected running time (ERT), used in the figuresand table, depends on a given target function value, ft =fopt + ∆ft, and is computed over all relevant trials as thenumber of function evaluations executed during each trialwhile the best function value did not reach ft, summed overall trials and divided by the number of trials that actuallyreached ft [4, 7]. Statistical significance is tested withthe rank-sum test for a given target ∆ft using, for eachtrial, either the number of needed function evaluations toreach ∆ft (inverted and multiplied by −1), or, if the targetwas not reached, the best ∆f -value achieved, measured onlyup to the smallest number of overall function evaluations forany unsuccessful trial under consideration.
According to the experiments, the (1,2m)-CMA-ES clearlyoutperforms the baseline algorithm (1,2)-CMA-ES. Both in5D and in 20D, the (1,2m)-CMA-ES is on no function and forno target ≤ 1 worse than the (1,2)-CMA-ES. Moreover, sta-tistically significant improvements can be reported on 8 and11 functions in 20D and 5D respectively. The improvementfactors are ranging from about 1.8 for f101 and f103 over2–3 (f106 and f109) to a factor of more than 5 for the spherefunction with moderate uniform noise (f102, all in 20D). In20D, three functions can be solved for all 15 instances bythe (1,2m)-CMA-ES whereas the (1,2)-CMA-ES has a lower(on f106) or much lower success probability (only 1 out of 15instances were solved by the (1,2)-CMA-ES on f112 and 0instances were solved for f118). Similar results hold for 5D.
On f109 (in both 5D and 20D and for several difficult tar-gets), on f130 (in 20D and for a target of 10−7), as wellas on medium targets on f121, the (1,2m)-CMA-ES outper-forms the function-wise best algorithm of the BBOB-2009benchmarking. The largest improvement is achieved for theGallagher function with Cauchy noise (f130) where the ex-pected running time of the (1,2m)-CMA-ES is about 40%lower than for the best algorithm of BBOB-2009 for thatfunction of last year’s benchmarking.
2.2 (1,2m)- and (1,2sm)-CMA-ES
The results of this comparison are shown in Fig. 3 and 4and in Table 2.
In 5D, no statistically significant differences (except forf121 where the (1,2s
m)-CMA-ES is outperforming the (1,2m)-CMA-ES) can be observed. In 20D, however, the (1,2s
m)-CMA-ES clearly shows a better (statistically significant)performance than the (1,2m)-CMA-ES on 6 functions: onf101, the improvement is 17%, on f102 and f103 about 20%,on f109 25%, on f112 about 60%, and on f118 about 30%.
The (1,2sm)-CMA-ES outperforms the function-wise best
algorithm of BBOB-2009 on f103 by about 12%, on f109 andf112 by about 30%, and on f130 by about 65% for severallow target values in 20D.
2.3 Comparing (1,2)- and (1,2sm)-CMA-ES
The third comparison yields similar results for the (1,2sm)-
CMA-ES than for the (1,2m)-CMA-ES above when com-pared to the (1,2)-CMA-ES, except that the improvementover the (1,2)-CMA-ES is even larger—not surprisingly af-
ter the results of Sec. 2.2. In 20D (5D), the (1,2)-CMA-ESis outperformed by the (1,2s
m)-CMA-ES on 8 (12) functions(with statistical significance). The expected running timeof the (1,2s
m)-CMA-ES is always smaller than the one of the(1,2)-CMA-ES (with a few exceptions in 2D and 3D, cp. 3).Note that due to space limitations, we cannot show the plotsand tables of this comparison.
3. CONCLUSIONSThe idea behind derandomization by means of mirroring
introduced in [1] is to use only one random sample froma multivariate normal distribution to create two (negativelycorrelated or mirrored) offspring. Thereby, the first offspringis generated by adding a random sample to the parent solu-tion and the second offspring then equals the solution whichis symmetric to the first offspring with respect to the par-ent (by adding the negative sample to the parent). Here,this concept of mirroring has been integrated within twovariants of a simple (1,2)-CMA-ES (of which the (1,2s
m)-CMA-ES uses sequential selection [1] in addition and the(1,2m)-CMA-ES does not). The three algorithms are thencompared on the noisy BBOB-2010 testbed.
The experiments clearly show a ranking among the threealgorithms: both the (1,2m)-CMA-ES and the (1,2s
m)-CMA-ES are never worse than the (1,2)-CMA-ES and have lowerexpected running times than the baseline by at least 50%on 5 functions. Moreover, the two algorithms employingmirroring solve three additional functions in 20D which the(1,2)-CMA-ES cannot solve (or only with small probabil-ity). The comparison between the (1,2m)-CMA-ES and the(1,2s
m)-CMA-ES is in favor of the (1,2sm)-CMA-ES outper-
forming the (1,2m)-CMA-ES on five functions by at least17% whereas no statistically significant worsening can beobserved. Both the (1,2m)-CMA-ES and the (1,2s
m)-CMA-ES also outperform the function-wise best algorithm of theBBOB-2009 benchmarking on three (respectively four) func-tions comprising Cauchy noise by up to 65%.
AcknowledgmentsThis work receives support by the French national researchagency (ANR) within the SYSCOMM project ANR-08-SYSC-017 and within the COSINUS project ANR-08-COSI-007-12.
4. REFERENCES[1] A. Auger, D. Brockhoff, and N. Hansen. Mirrored
sampling and sequential selection for evolutionstrategies. Rapport de Recherche RR-7249, INRIASaclay—Ile-de-France, April 2010.
[2] A. Auger, D. Brockhoff, and N. Hansen. Mirroredvariants of the (1,2)-CMA-ES compared on the noiselessBBOB-2010 testbed. In GECCO (Companion), 2010.
[3] S. Finck, N. Hansen, R. Ros, and A. Auger.Real-parameter black-box optimization benchmarking2010: Presentation of the noisy functions. TechnicalReport 2009/21, Research Center PPE, 2010.
[4] N. Hansen, A. Auger, S. Finck, and R. Ros.Real-parameter black-box optimization benchmarking2010: Experimental setup. Technical Report RR-7215,INRIA, 2010.
[5] N. Hansen, S. Finck, R. Ros, and A. Auger.Real-parameter black-box optimization benchmarking
116 Ellipsoid 119 Sum of diff. powers 122 Schaffer F7 125 Griewank-Rosenbrock 128 Gallagher
Ga
uss
no
ise
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
117 Ellipsoid 120 Sum of diff. powers 123 Schaffer F7 126 Griewank-Rosenbrock 129 Gallagher
un
ifor
mn
ois
e
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 60
1
2
3
4
5
6
118 Ellipsoid 121 Sum of diff. powers 124 Schaffer F7 127 Griewank-Rosenbrock 130 Gallagher
Ca
uch
yn
ois
e
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 60
1
2
3
4
5
6
Figure 1: Expected running time (ERT in log10 of number of function evaluations) of (1,2m)-CMA-ES versus(1,2)-CMA-ES for 46 target values ∆f ∈ [10−8, 10] in each dimension for functions f101–f130. Markers on theupper or right edge indicate that the target value was never reached by (1,2m)-CMA-ES or (1,2)-CMA-ESrespectively. Markers represent dimension: 2:+, 3:O, 5:?, 10:◦, 20:2.
Figure 2: Empirical cumulative distributions (ECDF) of run lengths and speed-up ratios in 5-D (left) and20-D (right). Left sub-columns: ECDF of the number of necessary function evaluations divided by dimensionD (FEvals/D) to reached a target value fopt + ∆f with ∆f = 10k, where k ∈ {1,−1,−4,−8} is given by the firstvalue in the legend, for (1,2m)-CMA-ES (solid) and (1,2)-CMA-ES (dashed). Light beige lines show the ECDFof FEvals for target value ∆f = 10−8 of all algorithms benchmarked during BBOB-2009. Right sub-columns:ECDF of FEval ratios of (1,2m)-CMA-ES divided by (1,2)-CMA-ES, all trial pairs for each function. Pairswhere both trials failed are disregarded, pairs where one trial failed are visible in the limits being > 0 or < 1.The legends indicate the number of functions that were solved in at least one trial ((1,2m)-CMA-ES first).
Table 1: ERT in number of function evaluations divided by the best ERT measured during BBOB-2009 (givenin the respective first row) for the algorithms (1,2)-CMA-ES and (1,2m)-CMA-ES for different ∆f values forfunctions f101–f130. The median number of conducted function evaluations is additionally given in italics,if ERT(10−7) = ∞. #succ is the number of trials that reached the final target fopt + 10−8. Bold entries arestatistically significantly better compared to the other algorithm, with p = 0.05 or p = 10−k where k > 1 is thenumber following the ? symbol, with Bonferroni correction of 60.
116 Ellipsoid 119 Sum of diff. powers 122 Schaffer F7 125 Griewank-Rosenbrock 128 Gallagher
Ga
uss
no
ise
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
117 Ellipsoid 120 Sum of diff. powers 123 Schaffer F7 126 Griewank-Rosenbrock 129 Gallagher
un
ifor
mn
ois
e
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 60
1
2
3
4
5
6
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
118 Ellipsoid 121 Sum of diff. powers 124 Schaffer F7 127 Griewank-Rosenbrock 130 Gallagher
Ca
uch
yn
ois
e
2 3 4 52
3
4
5
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
0 1 2 3 4 5 60
1
2
3
4
5
6
Figure 3: Expected running time (ERT in log10 of number of function evaluations) of (1,2sm)-CMA-ES versus
(1,2m)-CMA-ES for 46 target values ∆f ∈ [10−8, 10] in each dimension for functions f101–f130. Markers on theupper or right edge indicate that the target value was never reached by (1,2s
Table 2: Expected running time (ERT in number of function evaluations) divided by the best ERT measuredduring BBOB-2009 as in Table 1 but now comparing (1,2m)-CMA-ES and (1,2s
[6] N. Hansen, S. D. Muller, and P. Koumoutsakos.Reducing the time complexity of the derandomizedevolution strategy with covariance matrix adaptation.Evolutionary Computation, 11(1):1–18, 2003.
[7] K. Price. Differential evolution vs. the functions of thesecond ICEO. In Proceedings of the IEEE InternationalCongress on Evolutionary Computation, pages 153–157,1997.
![Page 1: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/1.jpg)
![Page 2: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/2.jpg)
![Page 3: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/3.jpg)
![Page 4: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/4.jpg)
![Page 5: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/5.jpg)
![Page 6: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/6.jpg)
![Page 7: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/7.jpg)
![Page 8: [inria-00502436, v1] Mirrored Variants of the (1,2)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010g.pdfNoisy BBOB-2010 Testbed [Black-Box Optimization Benchmarking Workshop]](https://reader036.documents.pub/reader036/viewer/2022071215/6044ead37b7cad01793f8a94/html5/thumbnails/8.jpg)
![Benchmarking Exponential Natural Evolution Strategies on the …tom/publications/bbob-xnes.pdf · 2012. 6. 5. · Report 2009/21, Research Center PPE, 2010. [4] N. Fukushima, Y. Nagata,](https://static.documents.pub/doc/80x56/6044ea906b0d854dbd17f927/benchmarking-exponential-natural-evolution-strategies-on-the-tompublicationsbbob-xnespdf.jpg)
![Benchmarking the LGO Solver Suite within the COCO Test ......GECCO, the GEnetic and Evolutionary Computation Conference [11]) has been held since 2009 in every year: the BBOB workshop](https://static.documents.pub/doc/80x56/6044e73ad25d8f668a78bb68/benchmarking-the-lgo-solver-suite-within-the-coco-test-gecco-the-genetic.jpg)
![[inria-00502438, v1] Mirrored Variants of the (1,4)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010i.pdfCompared to the BBOB-2009 benchmarking, the (1,4 s m)-CMA-ES improves](https://static.documents.pub/doc/80x56/6044e73ad25d8f668a78bb65/inria-00502438-v1-mirrored-variants-of-the-14-cma-es-dimobrockhoffpublicationlistfilesabh2010ipdf.jpg)
