HAL Id: hal-01435449 https://hal.inria.fr/hal-01435449 Submitted on 14 Jan 2017 HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Benchmarking RM-MEDA on the Bi-objective BBOB-2016 Test Suite Anne Auger, Dimo Brockhoff, Nikolaus Hansen, Dejan Tušar, Tea Tušar, Tobias Wagner To cite this version: Anne Auger, Dimo Brockhoff, Nikolaus Hansen, Dejan Tušar, Tea Tušar, et al.. Benchmarking RM- MEDA on the Bi-objective BBOB-2016 Test Suite. GECCO 2016 - Genetic and Evolutionary Com- putation Conference, Jul 2016, Denver, CO, United States. pp.1241-1247, 10.1145/2908961.2931707. hal-01435449
Transcript
HAL Id: hal-01435449https://hal.inria.fr/hal-01435449
Submitted on 14 Jan 2017
HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.
Benchmarking RM-MEDA on the Bi-objectiveBBOB-2016 Test Suite
To cite this version:Anne Auger, Dimo Brockhoff, Nikolaus Hansen, Dejan Tušar, Tea Tušar, et al.. Benchmarking RM-MEDA on the Bi-objective BBOB-2016 Test Suite. GECCO 2016 - Genetic and Evolutionary Com-putation Conference, Jul 2016, Denver, CO, United States. pp.1241-1247, �10.1145/2908961.2931707�.�hal-01435449�
ABSTRACTIn this paper, we benchmark the Regularity Model-BasedMultiobjective Estimation of Distribution Algorithm (RM-MEDA) of Zhang et al. on the bi-objective bbob-biobj testsuite of the Comparing Continuous Optimizers (COCO) plat-form. It turns out that, starting from about 200 times di-mension many function evaluations, RM-MEDA shows a linearincrease in the solved hypervolume-based target values withtime until a stagnation of the performance occurs ratherquickly on all problems. The final percentage of solved hy-pervolume targets seems to decrease with the problem di-mension.
Categories and Subject DescriptorsG.1.6 [Numerical Analysis]: Optimization—global opti-mization, unconstrained optimization; F.2.1 [Analysis ofAlgorithms and Problem Complexity]: Numerical Al-gorithms and Problems
1. INTRODUCTIONMulti-objective optimization differs from single-objective
optimization most importantly in the type of the desired ap-proximation. In the single-objective case, a single solutionwith a function value as small as possible is sought. In themulti-objective case, one is interested in finding an approx-imation of the set of Pareto-optimal solutions. The qualityof this approximation is given explicitly or implicitly by anindicator function, such as the hypervolume indicator [1].
Despite this difference of aiming to converge either to aset or to a single solution, only few efforts have been pur-sued to specifically develop variation or solution generation
approaches for the multi-objective case. In most cases, stillthe established variation operators from single-objective op-timization are used. One proposal of such a specific variationmethod is the Regularity Model-Based Multiobjective Esti-mation of Distribution Algorithm (RM-MEDA) by Zhang et al.We will benchmark this approach on the bi-objective bbob-
biobj test suite [6] of the Comparing Continuous Optimizersplatform COCO [4].
Throughout the paper, n will denote the problem dimen-sion.
2. ALGORITHMThe main idea behind RM-MEDA is to approximate the Pareto
set of a multi-objective problem by an (n − 1)-dimensionalmanifold represented by a piecewise linear model. Thismodel is used to sample candidate solutions in the vicinity ofthe current approximation, where the selected solutions arein turn used to improve the sampling model. The numberof segments or clusters is a parameter of the algorithm andthe respective submodels are learned through the applica-tion of a linear principle component analysis (LPCA). Thesampling of the piecewise linear model is performed by ran-domly picking a segment of the model. The probability ofpicking a segment is relative to the segments’ volume1. Onthe segment, a point is chosen uniformly at random and arandom perturbation in terms of an isotropic n-dimensionalnormal distribution is added. The variance or step size ofthe perturbation depends on the variation of the solutionsof the current population assigned to the chosen segment.
For a more detailed algorithm description of RM-MEDA, werefer the interested reader to the original publication [7].
2.1 Parameter SettingsThe MATLAB implementation of RM-MEDA2 was run with
the default parameters [7]. The code was slightly adjusted tocope with the assumptions regarding the vector and variablerepresentation mady by the COCO platform. A populationsize N = 100 was set. The generator of the distribution forsampling new solutions was based on a linear principal com-ponent analysis (LPCA) with 5 clusters, 50 training steps,and an extension rate of 0.25. The budget of 105n was used
1In the bi-objective case, the lengths of the segments areused.2http://cswww.essex.ac.uk/staff/qzhang/code/RM-MEDA-Matlab v0.1.zip
to determine the number of generations. No restarts wereperformed.
3. CPU TIMINGIn order to evaluate the CPU timing of the algorithm, we
have run the RM-MEDA with restarts on the entire bbob-biobjtest suite for 100n function evaluations. The COCO Mat-lab/Octave code was run with Octave 4.0.0 on a Windows 7machine with Intel(R) Core(TM) i7-5600U CPU 2.60GHzwith 1 processor and 2 cores. The time per function evalua-tion for dimensions 2, 3, 5, 10, 20, and 40 equaled 4.7 ·10−4,5.0 · 10−4, 5.2 · 10−4, 5.8 · 10−4, 7.0 · 10−4, and 8.7 · 10−4
seconds respectively.
4. RESULTSResults of RM-MEDA from experiments according to [5], [3]
and [2] and on the benchmark functions given in [6] arepresented in Figures 1, 2, 3, and 4, and in Table 1. Theexperiments were performed with COCO [4], version 1.0.1and the plots were produced with version 1.1.
On almost all problems, three phases can be distinguishedin the ECDF plots of Figures 1, 2, and 3. The first phaseof initialization and learning of the piecewise representationtakes about 100..200n function evaluations. In this phase,none or almost none of the target precisions are reached.It coincides with the performance of a pure random searchwithin the region of interest [−100, 100]n (comparison re-sults with pure random search not shown here). The secondphase shows a linear convergence in which the performancedisplayed in the ECDFs does not differ much for changingdimension (ECDFs for different dimensions are almost par-allel). In this phase, the algorithm successfully exploits thesampling model. The last phase is characterized by a stag-nation of the approximation quality. The sampling modelcannot further be refined and the number of targets achievedremains constant. The actual percentages of solved prob-lems at the end of the runs depend on the dimension of theproblems.
For the linear convergence phase, on some functions, thealgorithm seems to be quicker for smaller problem dimen-sions (on the four functions 11, 12, 16, and 23) while onmost, the algorithm shows an increased performance withhigher dimension (on the 35 functions 1–10, 17, 21, 26, 28–30, 32–40, 44–50, 52, 54, 55, and on almost all functiongroups). This observation, however, depends on the relativequality of the reference sets and the corresponding hypervol-ume reference values underlying the performance assessmentof COCO. This limitation must be taken into account beforemaking more general statements.
This also holds for the performance of RM-MEDA in the lastphase, but to a smaller degree: it is expected that algorithmperformance is decreasing when the problem dimension in-creases as this is the case also for RM-MEDA on all functiongroups and most functions. Exceptions are f26 (Attractivesector/Schwefel), and functions for which pairs of dimen-sions show similar performance. Here, it is most likely thatthe reference sets have a larger impact on the display thanthe actual effect of the dimension.
5. CONCLUSIONAfter a short initialization phase, the Regularity Model-
Based Multiobjective Estimation of Distribution Algorithm
(RM-MEDA) of Zhang et al. showed a linear increase in thesolved hypervolume-based target values on the bi-objectivebbob-biobj test suite of the Comparing Continuous Opti-mizers (COCO) platform. However, after some time, a stag-nation of the performance occurred. The final percentage ofsolved hypervolume targets seems to decrease with the prob-lem dimension.
6. ACKNOWLEDGMENTSThis work was supported by the grant ANR-12-MONU-
0009 (NumBBO) of the French National Research Agency.
7. REFERENCES[1] A. Auger, J. Bader, D. Brockhoff, and E. Zitzler.
[2] D. Brockhoff, T. Tusar, D. Tusar, T. Wagner,N. Hansen, and A. Auger. Biobjective performanceassessment with the COCO platform. ArXiv e-prints,arXiv:1605.01746, 2016.
[3] N. Hansen, A. Auger, D. Brockhoff, D. Tusar, andT. Tusar. COCO: Performance assessment. ArXive-prints, arXiv:1605.03560, 2016.
[4] N. Hansen, A. Auger, O. Mersmann, T. Tusar, andD. Brockhoff. COCO: A platform for comparingcontinuous optimizers in a black-box setting. ArXive-prints, arXiv:1603.08785, 2016.
[5] N. Hansen, T. Tusar, O. Mersmann, A. Auger, andD. Brockhoff. COCO: The experimental procedure.ArXiv e-prints, arXiv:1603.08776, 2016.
[6] T. Tusar, D. Brockhoff, N. Hansen, and A. Auger.COCO: The bi-objective black-box optimizationbenchmarking (bbob-biobj) test suite. ArXiv e-prints,arXiv:1604.00359, 2016.
[7] Q. Zhang, A. Zhou, and Y. Jin. RM-MEDA: Aregularity model based multiobjective estimation ofdistribution algorithm. IEEE Transactions onEvolutionary Computation, 12(1):41–63, 2008.
Figure 1: Empirical cumulative distribution of simulated (bootstrapped) runtimes in number ofobjective function evaluations divided by dimension (FEvals/DIM) for the 58 targets {−10−4,−10−4.2,−10−4.4,−10−4.6,−10−4.8,−10−5, 0, 10−5, 10−4.9, 10−4.8, . . . , 10−0.1, 100} for functions f1 to f16 and all dimensions.
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f175 instances
1.1
17 sep. Ellipsoid/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f185 instances
1.1
18 sep. Ellipsoid/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f195 instances
1.1
19 sep. Ellipsoid/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f205 instances
1.1
20 Attractive sector/Attractive sector
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
5-D
3-D
2-D
10-Dbbob-biobj - f215 instances
1.1
21 Attractive sector/Rosenbrock
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f225 instances
1.1
22 Attractive sector/Sharp ridge
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f235 instances
1.1
23 Attractive sector/Different Powers
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f245 instances
1.1
24 Attractive sector/Rastrigin
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f255 instances
1.1
25 Attractive sector/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
5-D
10-D
3-D
2-Dbbob-biobj - f265 instances
1.1
26 Attractive sector/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
5-D
20-D
3-D
2-Dbbob-biobj - f275 instances
1.1
27 Attractive sector/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f285 instances
1.1
28 Rosenbrock/Rosenbrock
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f295 instances
1.1
29 Rosenbrock/Sharp ridge
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f305 instances
1.1
30 Rosenbrock/Different Powers
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
20-D
5-D
3-D
2-Dbbob-biobj - f315 instances
1.1
31 Rosenbrock/Rastrigin
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f325 instances
1.1
32 Rosenbrock/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f335 instances
1.1
33 Rosenbrock/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f345 instances
1.1
34 Rosenbrock/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
20-D
5-D
3-D
2-Dbbob-biobj - f355 instances
1.1
35 Sharp ridge/Sharp ridge
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f365 instances
1.1
36 Sharp ridge/Different Powers
Figure 2: Empirical cumulative distribution of simulated (bootstrapped) runtimes, measured in numberof objective function evaluations, divided by dimension (FEvals/DIM) for the targets as given in Fig. 1 forfunctions f17 to f36 and all dimensions.
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f375 instances
1.1
37 Sharp ridge/Rastrigin
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f385 instances
1.1
38 Sharp ridge/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f395 instances
1.1
39 Sharp ridge/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f405 instances
1.1
40 Sharp ridge/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f415 instances
1.1
41 Different Powers/Different Powers
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f425 instances
1.1
42 Different Powers/Rastrigin
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s20-D
10-D
5-D
3-D
2-Dbbob-biobj - f435 instances
1.1
43 Different Powers/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f445 instances
1.1
44 Different Powers/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
20-D
5-D
3-D
2-Dbbob-biobj - f455 instances
1.1
45 Different Powers/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
20-D
5-D
3-D
2-Dbbob-biobj - f465 instances
1.1
46 Rastrigin/Rastrigin
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f475 instances
1.1
47 Rastrigin/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
5-D
10-D
3-D
2-Dbbob-biobj - f485 instances
1.1
48 Rastrigin/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f495 instances
1.1
49 Rastrigin/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f505 instances
1.1
50 Schaffer F7/Schaffer F7
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f515 instances
1.1
51 Schaffer F7/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f525 instances
1.1
52 Schaffer F7/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
5-D
20-D
3-D
2-Dbbob-biobj - f535 instances
1.1
53 Schwefel/Schwefel
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f545 instances
1.1
54 Schwefel/Gallagher 101
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
20-D
3-D
5-D
2-Dbbob-biobj - f555 instances
1.1
55 Gallagher 101/Gallagher 101
Figure 3: Empirical cumulative distribution of simulated (bootstrapped) runtimes, measured in numberof objective function evaluations, divided by dimension (FEvals/DIM) for the targets as given in Fig. 1 forfunctions f37 to f55 and all dimensions.
multimodal-multimodal multimodal-weakstructure weakstructure-weakstructure all 55 functions
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f46, f47, f505 instances
1.1
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f48, f49, f51, f525 instances
1.1
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
10-D
20-D
5-D
3-D
2-Dbbob-biobj - f53-f555 instances
1.1
0 1 2 3 4 5 6 7 8log10 of (# f-evals / dimension)
0.0
0.2
0.4
0.6
0.8
1.0
Pro
port
ion o
f fu
nct
ion+
targ
et
pair
s
20-D
10-D
5-D
3-D
2-Dbbob-biobj - f1-f555 instances
1.1
Figure 4: Empirical cumulative distribution of simulated (bootstrapped) runtimes, measured in numberof objective function evaluations, divided by dimension (FEvals/DIM) for the 58 targets {−10−4,−10−4.2,−10−4.4,−10−4.6,−10−4.8,−10−5, 0, 10−5, 10−4.9, 10−4.8, . . . , 10−0.1, 100} for all function groups and all dimensions.The aggregation over all 55 functions is shown in the last plot.
![[inria-00502438, v1] Mirrored Variants of the (1,4)-CMA-ES ...dimo.brockhoff/publicationListFiles/abh2010i.pdfCompared to the BBOB-2009 benchmarking, the (1,4 s m)-CMA-ES improves](https://static.documents.pub/doc/80x56/6044e73ad25d8f668a78bb65/inria-00502438-v1-mirrored-variants-of-the-14-cma-es-dimobrockhoffpublicationlistfilesabh2010ipdf.jpg)
![[inria-00502441, v1] Benchmarking the (1,4)-CMA-ES With ...dimo.brockhoff/publicationListFiles/abh2010k.pdftial selection on the BBOB-2010 noisy testbed. Independent restarts are conducted](https://static.documents.pub/doc/80x56/6044e73ad25d8f668a78bb64/inria-00502441-v1-benchmarking-the-14-cma-es-with-dimobrockhoffpublicationlistfilesabh2010kpdf.jpg)
