Expert Systems With Applications 44 (2016) 1–12

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

An improved adaptive differential evolution algorithm for

continuous optimization

Wenchao Yi, Yinzhi Zhou1, Liang Gao, Xinyu Li∗, Jianhui Mou

State Key Laboratory of Digital Manufacturing Equipment & Technology, Huazhong University of Science and Technology, Wuhan 430074, PR China

a r t i c l e i n f o

Keywords:

Differential evolution

pbest retention mechanism

Adaptive parameter control

pbest roulette wheel selection

Real-world application problems

a b s t r a c t

A novel differential evolution algorithm based on adaptive differential evolution algorithm is proposed by

implementing pbest roulette wheel selection and retention mechanism. Motivated by the observation that

individuals with better function values can generate better offspring, we propose a fitness function value

based pbest selection mechanism. The generated offspring with better fitness function value indicates that

the pbest vector of current individual is suitable for exploitation, so the pbest vector should be retained into

the next generation. This modification is used to avoid the individual gather around the pbest vector, thus

diversify the population. The performance of the proposed algorithm is extensively evaluated both on the 25

famous benchmark functions and four real-world application problems. Experimental results and statistical

analyses show that the proposed algorithm is highly competitive when compared with other state-of-the-art

differential evolution algorithms.

© 2015 Elsevier Ltd. All rights reserved.

1

(

r

p

p

l

o

W

2

p

2

F

t

e

t

o

t

f

v

L

g

(

o

g

h

m

s

v

t

e

E

e

a

t

s

i

t

p

b

p

d

e

m

l

h

0

. Introduction

Differential evolution (DE) algorithm, proposed by Storn and Price

1997), has been proven to be simple yet effective evolutionary algo-

ithm (EA) (Bäck, 1996). What’s more, DE algorithm presents com-

etitive performance in diverse fields. DE algorithm has been ap-

lied successfully in fields including constrained optimization prob-

ems (Becerra & Coello, 2006; Huang, Wang, & He, 2007), multi-

bjective optimization problems (Gong & Cai, 2009; Tan, Jiao, Li, &

ang, 2012) and engineering design optimization problems (Liao,

010; Yildiz, 2013). However, recent research works showed that the

erformance of DE is related to mutation operators (Das & Abaham,

009; Epitropakis, Tasoulis, Pavlidis, Plagianakos, & Vrahatis, 2011;

an & Lampinen, 2003; Piotrowski, 2013; Zhou, Li, & Gao, 2013). Mu-

ation operator manipulates the balance between exploitation and

xploration. We will mainly focus on introducing some representa-

ive works on the improvement of the mutation operators and a series

f improvements on the efficient DE variant called adaptive differen-

ial evolution algorithm with optional external archive (JADE) in the

ollowing paragraph.

Some researchers have made some contributions to the inno-

ation of the mutation operator adopted in DE algorithm. Fan and

ampinen (2003) proposed the trigonometric region based mutation

∗ Corresponding author. Tel.: +86 27 87557742; fax: +86 27 87543074.

E-mail addresses: janety1989@gmail.com (W. Yi), leitezhou@gmail.com (Y. Zhou),

aoliang@mail.hust.edu.cn (L. Gao), lixinyu@mail.hust.edu.cn (X. Li), mjhcr@163.com

J. Mou).1 Co-first author.

r

S

p

t

T

ttp://dx.doi.org/10.1016/j.eswa.2015.09.031

957-4174/© 2015 Elsevier Ltd. All rights reserved.

perator, in which a trigonometric region was formed to limit the

enerated mutant vectors. Das and Abraham (2009) introduced the

ybrid DE mutation operator that was a linear combination of two

utation operators to balance exploration and exploitation ability. A

tatic ring topology was used to select the neighborhood best indi-

idual to enhance local search. Based on this, Piotrowski (2013) in-

roduced an improved version of neighborhood-based mutation op-

rator by splitting the mutation directly into global and local ones.

pitropakis et al. (2011) presented the proximity-based mutation op-

rator. By incorporating the information about neighboring individu-

ls, more specifically, by giving each individual selection probability

hat is inversely related to its distance from the base vector. An in-

ect mutation operator proposed by Zhou et al. (2013) divided the

ndividuals into better part and worse part. Based on the division,

he novel mutation operator lets the better part to exploit, the worse

art to explore. Most researchers used “DE/rand/1”, “DE/current-to-

est/1” and “DE/best/1” or their modified versions. “DE/rand/1” em-

hasizes on the exploration, while the last two are rarely used mainly

ue to their premature convergence to multimodal problems. How-

ver, “DE/current-to-best/1” and “DE/best/1” can obtain good perfor-

ances when dealing with the unimodal problems. So, it is a chal-

enge to combine their merits together.

Some researchers achieved excellent improvements in DE algo-

ithm, in which a DE variant called JADE presented by Zhang and

anderson (2009a) attracted wide attentions in recent years for its

romising results. A novel adaptive control parameter and a new mu-

ation operator called “DE/current-to-pbest” were proposed in JADE.

he control parameters of each individual are updated according to

2 W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12

D

w

t

{i

x

w

t

2

t

g

vw

a

t

g

w

v

2

s

c

u

w

g

t

[

2

x

a

t

x

w

x

the successful information of F and CR for each individual in the last

generation. The top 100∗p% (p is a predefined number between 0 and

1) individuals in the current generation are stored and then randomly

choose any of the top 100∗p% individual to replace role that the best

individual plays in “DE/current-to best/1”. JADE is a highly competi-

tive DE variant and some improvements based on the framework of

JADE are studied. Gong, Cai, Ling, and Li (2011) introduced the strat-

egy adaptive mechanism. The mechanism chooses a mutation oper-

ator at random if the generated random number is smaller than a

predefined parameter or uses the operator in the previous genera-

tion, where a family of different mutation operators used in JADE is

chosen to formulate the pool. Islam, Das, Ghosh, Roy, and Suganthan

(2012) proposed an improved JADE, in which pbest crossover oper-

ator was proposed and modified adaptation schemes were utilized.

Based on JADE, a repairing crossover rate technique based JADE was

proposed by Gong, Cai, and Wang (2014) The crossover rate is re-

paired by calculating its corresponding binary string, which can be

calculated and updated according to binary string for the each indi-

vidual in the last generation. This algorithm is one of the best DE algo-

rithms that obtain the best performances over CEC2005 competition

benchmark functions. From this series of research works, we found

that JADE is an efficient DE variant for global optimization problems.

However, as emphasized above, mutation operator has a great impact

on the DE algorithm’s performance. Although “DE/current-to-pbest”

can help the offspring gather around the better individuals, it will de-

crease the diversity to some extent. So there still exists some space for

further improvements.

Based on the above analysis, we present a modified algorithm

based on JADE. In JADE, one top vectors are chosen for mutation op-

eration for each individual randomly. However, this process does not

consider the function value of these alternative vectors. As Rank-JADE

algorithm shows, the terminal vector of the difference vectors plays

an important role in the DE variants’ performance. DE algorithm has

shown excellent performance in exploring the feasible region, the

motivation of our research is to enhance its exploitation ability by

making the better top vectors to be chosen to survive into the next

generation. The better here has two-fold meanings, one of which

means the top vector with better fitness function value, the other in-

dicates the top vector that can produce better offspring. Based on the

above motivation, a pbest roulette wheel selection operation is exe-

cuted according to the function value of top vectors. If this top vector

can achieve better trial vector, it can survive into the next genera-

tion, even it may not be included in the top vectors anymore. Then

we combined the improved mutation operators with the repairing

crossover rate mechanism. We call this algorithm as pbest roulette

wheel selection and retention mechanism based repairing crossover

rate in adaptive DE algorithm (pbestrr-JADE).

The remainder of the paper is organized as follows. Section 2 in-

troduces the classical DE algorithm and reviews the related works on

one of competitive DE variants called JADE. Then the proposed al-

gorithm is presented in Section 3. Experimental results on CEC2005

competition benchmark functions and real-world application prob-

lems are reported in Section 4. Finally, Section 5 concludes the paper.

To give a vivid description of the whole paper, the graphical abstract

is presented in the Appendix.

2. Related works of JADE

In this section, we will present the classical DE as a foundation.

Then one of the most competitive DE variants called JADE will be in-

troduced. In the following section, a recently proposed Rcr-JADE will

be presented.

2.1. Classical DE algorithm

In this section, an introduction of the classical DE algorithm [1]

will be presented, which facilitates the explanation of the improved

DE algorithm later.

DE that is an effective evolutionary algorithm utilizes NP

-dimensional individuals, i.e. xi,G = {x1i,G

, . . . , xDi,G

}, i = 1, . . . , NP,

here G denotes the number of generations. Each dimension of

he individual is constrained by xmin = {x1min

, . . . , xDmin

} and xmax =x1

max, . . . , xDmax}. Usually, the initial population is randomly generated

n the feasible region, which can be expressed as follows:

ji,0

= x jmin

+ rand(0, 1) ∗ (x jmax − x j

min) (1)

here rand(0,1) denotes a random number that is uniformly dis-

ributed and generated in the range of 0 and 1.

.1.1. Mutation

Then the mutation operator is utilized to generate the mutant vec-

ors vi,G, DE/rand/1 is the most commonly used operator, where the

enerated vi,G can be represented as:

i,G = xr1,G + F ∗ (xr2,G − xr3,G), r1 �= r2 �= r3 �= i (2)

here xr1 ,G, xr2 ,G, xr3 ,G that are chosen from the current population

nd individual i are four mutually different individuals. F is the mu-

ation control parameter to scale the difference vector. Similarly, we

ive another five frequently used mutation operators as follows:

(1) “DE/rand/2”

vi,G = xr1,G + F ∗ (xr2,G − xr3,G) + F ∗ (xr4,G − xr5,G),

r1 �= r2 �= r3 �= r4 �= r5 �= i (3)

(2) “DE/best/1”

vi,G = xbest,G + F ∗ (xr1,G − xr2,G), r1 �= r2 �= i (4)

(3) “DE/best/2”

vi,G = xbest,G + F ∗ (xr1,G − xr2,G) + F ∗ (xr3,G − xr4,G),

r1 �= r2 �= r3 �= r4 �= i (5)

(4) “DE/rand-to-best/1”

vi,G = xi,G + F ∗ (xbest,G − xi,G) + F ∗ (xr1,G − xr2,G),

r1 �= r2 �= i (6)

(5) “DE/current-to-best/1”

vi,G = xi,G + F ∗ (xr1,G − xi,G) + F ∗ (xr2,G − xr3,G),

r1 �= r2 �= r3 �= i (7)

here xbest, G defines the individual that has the best fitness function

alue at the G-th generation.

.1.2. CrossoverThen the binomial crossover operator that is often used can be

elected to generate the trail vector ui, G between xi, G and vi, G, which

an expressed by the formula below:

ji,G

={

v ji,G

, if randj ≤ CRi or j = nj

x ji,G

, otherwisei = 1, 2, . . . , NP; j = 1, 2, . . . , D

(8)

here randj is a random number that is uniformly distributed and

enerated within the range of [0, 1]. CRi ∈ (0, 1) is crossover con-

rol parameter and nj is a random integer generated within the range

1, D].

.1.3. Selection

Then a better individual between trail vector ui, G and target vector

i, G will be selected. The better one will survive into the next gener-

tion based on the comparison of the fitness value. The greedy selec-

ion is performed as shown below:

i,G+1 ={

ui,G, if f (ui,G) ≤ f (xi,G)

xi,G, otherwise(9)

here f(xi, G) and f(ui, G) are the fitness function value of target vector

i, G and trail vector ui, G.

W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12 3

2

J

w

n

c

t

a

f

2

t

D

a

p

s

v

w

t

i

g

A

A

i

b

v

w

P

t

2

d

s

o

o

μ

w

d

k

w

m

[

F

w

t

1

μ[

t

S{

Fig. 1. r1 selection procedure for Rank-JADE.

w

d

t

e

o

c

a

i

2

f

J

p

v

w

l

s

a

i

t

T

s

2

t

p

v

B

b

w

C

CD

.2. JADE

Zhang and Sanderson (2009a) proposed a novel DE algorithm,

ADE, in which a mutation operator called “DE/current-to-pbest/1”

ith optional archive and an adaptive control parameter mecha-

ism are proposed. “DE/current-to-pbest/1” with optional archive

an make a balance between global and local search ability. The adap-

ive control parameter mechanism can update the control parameters

ccording to the successful ones, which can adjust adaptively to dif-

erent problems.

.2.1. DE/current-to-pbest/1

Comparing with the mutation operators DE/rand/k focusing on

he exploration ability, greedy operators DE/current-to-best/k and

E/best/k emphasize on the exploitation ability. To obtain a fast

nd reliable convergence performance, DE/current-to-pbest/1 is pro-

osed by Zhang and Sanderson (2009a, 2009b), which can be de-

cribed as the formula below:

i,G = xi,G + Fi ∗ (xpbest,G − xi,G)

+ Fi ∗ (xr1,G − xr2,G)(without archive) (10)

here xpbest, G denotes the randomly selected individual among the

op 100∗p% individuals in the current population with p ∈ (0, 1) and Fi

s the mutation control parameter that associated with xi, G and is re-

enerated according to the control parameter adaptation mechanism.

n optional archive A can be implemented in DE/current-to-pbest/1.

rchive A is used to store the inferior solutions that fail to survive

nto the next generation in the selection operation. The formula can

e rewritten as follows:

i,G = xi,G + Fi ∗ (xpbest,G − xi,G)

+ Fi ∗ (xr1,G − x̃r2,G)(with archive) (11)

here xi, G, xpbest, G, and xr1, G are selected from the current population

. x̃r2,G is chosen from the union set P∪A. If the archive size exceeds

he maximum size, then remove some randomly chosen solutions.

.2.2. Control parameter adaptation mechanism

The control parameter Fi is generated according to the Cauchy

istribution and successful parameters in the last generation. As

tated by Zhang and Sanderson (2009a), the location parameter μF

f Cauchy distribution is initialized to be 0.5 and updated at the end

f each generation as the formula below:

F = (1 − c) ∗ μF + c ∗ meanL(SF ) (12)

here c is a positive number generated in the range of (0, 1), SF

enotes the set of successful mutation control parameters that is

ept from the last generation, and meanL(.) means the Lehmer mean,

hich can be calculated as follows:

eanL(SF ) =∑

F∈SFF 2∑

F∈SFF

(13)

The formula for generating Fi presented in Zhang and Sanderson

13] is as follows:

i = randci(μF , 0.1) (14)

here randci(x, y) is a Cauchy distribution with localization parame-

er x and scale parameter y. If Fi exceeds 1, then Fi is truncated to be

. If Fi is less than 0, then Fi should be regenerated [13].

Similarly, CRi is generated with a normal distribution of mean

CR and standard deviation that equals 0.1. CRi then is truncated to

0,1]. μCR is usually initialized to be 0.5 and then updated according

o the above equations. The formula of CRi proposed by Zhang and

anderson (2009a) can be expressed as follows:

CRi = randni(μCR, 0.1)

μCR = (1 − c) ∗ μCR + c ∗ meanA(SCR)(15)

here randni(x, y) means a normal distribution with mean x and stan-

ard deviation y. meanA(x) represents the arithmetic mean of x. SCR is

he set of all successful crossover control parameters in the last gen-

ration. c is a positive constant between 0 and 1.

Since JADE proposed, many researchers have made improvements

n it. In the following sections, we will introduce recently proposed

ompetitive JADE variants called ranking based mutation operator in

daptive DE algorithm (Rank-JADE) and repairing the crossover rate

n adaptive DE algorithm (Rcr-JADE), respectively.

.3. Rank-JADE

Gong and Cai (2013) proposed ranking based mutation operator

or the DE algorithm (Rank-DE). In the paper, a variant named Rank-

ADE was introduced. Rank-JADE with an improved DE/current-to-

best/1 operator version was presented:

i,G = xi,G + Fi ∗ (xpbest,G − xi,G) + Fi ∗ (xr1,G − xr2,G) (16)

here the terminal vector index r1 is selected according to the fol-

owing rule, which is based on the basic r1 selection rule that the clas-

ical DE algorithm adopted: Firstly, the whole population is sorted in

scending order. Then the selection probability is defined as follows:

pi = (NP − i)/NP i = 1, 2, . . . , NP (17)

Finally, if the random number generated within the range of [0,1]

s bigger than the selection probability or the index r1 is equal to i,

hen regenerating the index r1 till it meets the above requirement.

he r1 selection procedure can be summarized as follows (Fig. 1)

The Rank-JADE algorithm presents competitive experimental re-

ults in Gong & Cai (2013).

.4. Rcr-JADE

Gong et al. (2014) proposed the repairing crossover rate in adap-

ive DE algorithm (Rcr-JADE). Rcr-JADE adopted the “DE/rand-to-

best/1 with archive” operator:

i,G = xr1,G + Fi ∗ (xpbest − xr1,G)

+ Fi ∗ (xr2,G − x̃r3,G)(with archive) (18)

The repairing technique modifies the CRi according to the set SCR.efore introducing the repairing technique, bi, j is defined as follows:

i, j ={

1, if (rand < CRi or j = nj)

0, otherwisei = 1, 2, . . . , NP; j = 1, 2, . . . , D

(19)

here D denotes the dimension of the variable b. Then the repairing

Ri is calculated as:

R′i =

∑Dj=1 bi, j

i = 1, 2, . . . , NP; j = 1, 2, . . . , D (20)

4 W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12

Fig. 2. The pseudo code of improved DE/current-to-pbest/1 operator.

Fig. 3. The pseudo code of pbest retention mechanism.

F

3

a

(

b

t

h

t

p

f

p

We can conclude from the introduction part that the mutation op-

erator has a great impact on the performance of DE algorithm. Al-

though Rcr-JADE is one of the most competitive variants of DE algo-

rithm by far, the mutation operator that Rcr-JADE adopted still has

room for improvement.

3. The proposed pbestrr-JADE

In JADE and Rcr-JADE, each individual randomly chooses one in-

dividual among top 100∗p% group in the population. However, this

process does not take the function value of these top individuals into

consideration. In this paper, we proposed a fitness-based elite selec-

tion process to deal with this situation. Based on the above work, the

pbestrr-JADE will be proposed in this section.

3.1. Power of individuals

For minimization objective function problem, we define the nor-

malized cost of each individual by:

F(xi) = maxNPi=1{ f (xi)} − f (xi) (21)

where maxNPi=1

{ f (xi)} means the maximum function value in the pop-

ulation. Then we can define the normalized power of each individualby the following equation:

proi ={∣∣F(xi)/

∑NPj=1 F(xi)

∣∣, if∑NP

j=1 F(xi) �= 0

1/NP, otherwisei, j = 1, 2, . . . , NP

(22)

So the normalized power of each individual is determined by func-

tion value. The better individuals have high probabilities.

3.2. Improved DE/current-to-pbest/1 operator

In JADE, pbest is randomly selected from the top 100∗p% individu-

als. These pbest individuals in top 100∗p% group are named as elites.

In this paper, these elites are chosen according to the normalized

power of each elite.

proi =∣∣∣∣∣F(xi)

/N∑

j=1

F(x j

)∣∣∣∣∣ i = 1, 2, . . . , NP; j = 1, 2, . . . , N (23)

where F(xi) = maxNi=1

{ f (xi)} − f (xi), N is the number of elites, equals

to 100∗p. The cumulative probability of each elite is defined as:

cproi =∣∣∣∣∣ i∑

j=1

proi

/N∑

j=1

proj

∣∣∣∣∣ i = 1, 2, . . . , NP; j = 1, 2, . . . , N (24)

All individuals in the population will choose one elite as pbest in

formula (11) using roulette wheel selection, i.e. the elite with better

individuals will have a higher chance to be chosen as pbest.

The pseudo code of improved DE/current-to-pbest/1 operator is

presented in Fig. 2.

3.3. pbest retention mechanism

The pbest individuals are updated in each generation. Usually,

these they may differ from those in the last generation. Some pbest

individuals that produce better trail vector ui,G+1 may lose their po-

sitions in the next generation. However, generated trail vector ui,G+1

survives into the next iteration means that pbesti tends to generate

individuals that are more likely to survive. So pbest retention mech-

anism is used in the proposed algorithm, which means pbest in-

dividuals that generate better individuals can survive into the next

generation.

To sum up, pbesti for the individual i can be calculated as:

pbesti ={

pbesti, if f (ui,G+1) ≤ f (xi,G)elite j, if f (ui,G+1) > f (xi,G) and cpro j−1 ≤ rand < cpro j

i = 1, 2, . . . , NP; j = 1, 2, . . . , N

(25)

The pseudo code of pbest retention mechanism is presented in

ig. 3.

.4. Overall implementation

The pseudo code of the proposed pbestrr-JADE algorithm is given

s follows:

To the best of our knowledge, Rank-DE proposed by Gong and Cai

2013) was one of the most competitive attempt to provide a ranking

ased selection on the specific vector in the mutation operator. Due

o the similarity that both the Rank-JADE and the proposed algorithm

ave some improvement on the selection on the specific vector in

he mutation operator, we will take Rank-JADE with DE/current-to-

best/1 that proposed by Gong and Cai (2013) as the representative

or further comparison. Furthermore, we will state that the proposed

bestrr-JADE differs from Rank-JADE in the following major aspects.

(1) pbestrr-JADE focuses on the selection of vector xpbest, while

vector xr2 and x̃r3 are randomly selected in each specific set.

While Rank-JADE focuses on the selection of vector xr2 and

vector xpbest and x̃r3 are randomly selected. The reason that

we choose xpbest vector to improve is xpbest plays an important

role in DE/current-to-pbest/1 mutation operator than xr2 do.

We will make further comparison in the following section.

(2) The proposed pbestrr-JADE uses the roulette wheel selection

method to select the specific vector. While Rank-JADE uses the

method that is similar to a liner ranking fitness assignment to

W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12 5

Fig. 4. The pseudo code of the proposed pbestrr-JADE algorithm.

c

b

t

4

m

d

e

m

f

p

f

a

b

c

e

t

t

(

b

d

f

4

m

r

m

a

e

w

v

h

r

c

l

f

t

select the specific vector. In general, the selection probability

formula is different.

(3) In pbestrr-JADE, the pbest retention mechanism is proposed.

This mechanism is designed to enhance the diversity of the

population and is aiming at archiving those promising pbest

individuals that can produce better offsprings, while Rank-

JADE do not have this mechanism.

Despite of the above three major aspects, we will make further

omparison in the following sections both on the CEC 2005 test

enchmark functions and four real-world application problems with

he proposed algorithm to demonstrate the difference between them.

. Experimental results

To evaluate the performance of the proposed approach, 25 fa-

ous benchmark functions from CEC2005 contest are adopted. The

etailed description of these functions can be referred in Suganthan

t al. (2005). According to Suganthan et al. (2005), all these bench-

ark functions can be divided into four diverse groups: unimodal

unctions (F01–F05), basic multimodal functions (F06–F12), ex-

anded multimodal functions (F13–F14), and hybrid composition

unctions (F15–F25).

In the experiments, the following parameters for JADE, Rcr-JADE

nd proposed algorithm are adopted primarily (later a discussion will

e conducted on the population size of the proposed algorithm after

hoosing a representative mutation operator). Population size (NP)

quals to 100; elite size equals to 10; initial distribution parame-

ers: μF = 0.5 and μCR = 0.5; weight c = 0.1. These parameter set-

ings are suggested by Gong et al. (2014) and Zhang and Sanderson

2009a). As suggested by Suganthan et al. (2005), the maximal num-

er of fitness evaluation equals to D × 10000, where D represents the

imension of each function and all the experimental results are taken

rom 25 independent runs.

.1. Performance comparisons of the new algorithm with different

utation operators

To choose a suitable mutation operator for the proposed algo-

ithm, four modified “DE/rand-to-best/1” and “DE/current-to-best/1”

utation operators proposed in Zhang and Sanderson (2009a, 2009b)

re tested. The four mutation operators that also adopted by Gong

t al. (2014) are listed below:

(1) “DE/current-to-pbest/1 (without archive)”:

vi = xi + Fi ∗ (xpbest − xi) + Fi ∗ (xr2 − xr3) (26)

(2) “DE/rand-to-pbest/1(without archive)”:

vi = xr1 + Fi ∗ (xpbest − xr1) + Fi ∗ (xr2 − xr3) (27)

(3) “DE/current-to-pbest/1(with archive)”:

vi = xi + Fi ∗ (xpbest − xi) + Fi ∗ (xr2 − x̃r3) (28)

(4) “DE/rand-to-pbest/1 (with archive)”:

vi = xr1 + Fi ∗ (xpbest − xr1) + Fi ∗ (xr2 − x̃r3) (29)

here the archive refers to an archive A that stores the inferior indi-

iduals recently explored in the evolutionary process. Other indexes

ave been explained in Section 2 . We use O1, O2, O3, and O4 rep-

esent these four operators in short. The proposed algorithm will be

ompared with JADE, Rcr-JADE with four different mutation operators

isted above, respectively. So, there are four JADEs, four Rcr-JADEs and

our pbestrr-JADEs based on four mutation operators. We denoted

hem as:

• JADE-O1, Rcr-JADE-O1 and pbestrr-JADE-O1 based “DE/current-to-

pbest/1(without archive)”• JADE-O2, Rcr-JADE-O2 and pbestrr-JADE-O2 based “DE/rand-to-

pbest/1 (without archive)”

6 W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12

Fig. 5. Best rankings of JADE, Rcr-JADE and pbestrr-JADE with diverse operators for all

functions with D = 30.

Fig. 6. Best rankings of JADE, Rcr-JADE and pbestrr-JADE with diverse operators for all

functions with D = 50.

Fig. 7. Best rankings of pbestrr-JADE with different operators for all functions at D = 50.

p

w

f

4

t

t

i

e

u

A

v

d

s

o

A

t

t

t

s

e

p

f

t

1

t

m

e

4

S

l

a

J

o

b

a

l

T

g

• JADE-O3, Rcr-JADE-O3 and pbestrr-JADE-O3 based “DE/current-

to-pbest/1 (with archive)”• JADE-O4, Rcr-JADE-O4 and pbestrr-JADE-O4 based “DE/rand-to-

pbest/1 (with archive)”

The CEC2005 benchmark functions with dimension D = 30 and

D = 50 are used and the experimental results are given in Table 1.

All the results are taken from 25 independent runs and are presented

in Tables 1 and 2; the overall best results in 12 algorithms are high-

lighted in gray boldface. Moreover, the number of best results ob-

tained by each algorithm with dimension D = 30 and D = 50 are plot-

ted in Figs. 5, and 6. An additional comparison is made by comparing

the number of the best results achieved in pbestrr-JADE with different

mutation operators with dimension D = 50 in Fig. 7. The best pbestrr-

JADE operator among the four pbestrr-JADEs will be selected.

According to comparisons made in Table 1 and Fig. 5, pbestrr-

JADE-O3 obtains the best results over 25 benchmark functions at

D = 30, which achieves an overwhelming 11 the best results over the

comparison with other 11 algorithms. While in Table 2 and Fig. 6, it

is difficult to find an overwhelming one among the 12 algorithms at

D = 50, so we make a further comparison among pbestrr-JADE algo-

rithms. From Fig. 7, we can conclude that pbestrr-JADE-O3 has ad-

vantages over other three pbestrr-JADE algorithms. Robustness plays

an important role in evaluating an algorithm. Although pbestrr-JADE-

O4 obtains most the best results in Fig. 6, it can be concluded from

Figs. 6 and 7 that pbestrr-JADE-O3 is the most robust one among the

roposed four pbestrr-JADE algorithms. So in the following section,

e take pbestrr-JADE-O3 as the representative one to carry on the

urther experiments.

.2. Study on the population size

We have studied the influence of diverse mutation operators on

he proposed framework in the previous section. It is worth men-

ioning that in the selected pbestrr-JADE-O3, the elites, which is pbest

ndividuals, may have a relationship with the population size (as the

lites percentage is given). So, we will study the influence of the pop-

lation size that may have on the proposed algorithm in this section.

s we have chosen pbestrr-JADE-O3 as the representative in the pre-

ious section, population size with 50, 100, 150, and 200 will be con-

ucted on the selected pbestrr-JADE-O3 algorithm to find the best

etting of the population size. 25 independent runs are conducted

n 25 benchmark functions at D = 30 (with D∗10,000 FES per run).

ll the other parameters that are needed for the pbestrr-JADE-O3 are

he same as we stated at the beginning of Section 4. The experimen-

al results are presented in Table 3 and the gray boldface indicates

he best results that are achieved among the four population size

ettings. The last column presents the number of the best results that

ach algorithm achieved.

With population size 200, the pbestrr-JADE-O3 can achieve

romising experimental results on the first seven test benchmark

unctions. While with population size 100, pbestrr-JADE-O3 can ob-

ain the best results on the hybrid complex benchmark functions and

7 best results are achieved. From the Table 3, we can conclude that

he proposed pbestrr-JADE-O3 algorithm can achieve the best perfor-

ance with population size 100. Hence, we will conduct the further

xperiments with population size 100.

.3. Comparison with several improved DE algorithms

In this section, five successful improved DE algorithms namely

aDE, JADE, CoDE, Rank-JADE, and Rcr-JADE are compared in the fol-

owing experiment. The results of SaDE, JADE and CoDE in Table 4

re adopted from Wang, Cai, and Zhang (2011). The results of Rank-

ADE in Table 4 are adopted from Gong and Cai (2013). The results

f Rcr-JADE in Table 4 are taken from Gong et al. (2014). The gray

oldface indicate the best results among these algorithms. To make

fair comparison, Wilcoxon signed-rank test at the 0.05 significance

evel is applied to ensure statistically sound conclusion in Table 5. In

able 5, the same parameter settings are utilized for these five al-

orithms as that in their firstly presented papers. The number of

W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12 7

Table 1

Comparisons between JADE, Rcr-JADE and its corresponding pbestrr-JADE for CEC2005 at D = 30.

JADE-O1 Rcr-JADE-O1 pbestrr-JADE-O1 JADE-O2 Rcr-JADE-O2 pbestrr-JADE-O2

1 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

2 1.22E−27 ± 1.20E-27 8.26E−28 ± 4.54E-28 2.74E−18 ± 9.37E-18 1.35E−27 ± 2.65E-27 6.38E−28 ± 3.92E−28 2.67E−20 ± 4.92E-20

3 1.55E+04 ± 1.06E+04 1.60E+04 ± 1.04E+04 2.67E+04 ± 1.76E+04 2.82E+04 ± 1.53E+04 2.22E+04 ± 1.71E+04 3.64E+04 ± 1.79E+04

4 3.88E−09 ± 1.65E−08 3.47E−08 ± 1.41E−07 4.19E−07 ± 1.42E−07 1.02E+03 ± 2.46E+03 2.78E−07 ± 1.02E−06 1.97E−06 ± 4.08E−06

5 1.69E+01 ± 3.90E+01 4.28E+01 ± 1.14E+02 3.09E+00 ± 1.08E+01 9.61E+01 ± 1.55E+02 1.33E+02 ± 2.18E+02 4.06E+01 ± 1.19E+02

6 1.77E+01 ± 3.53E+01 8.77E−01 ± 1.67E+00 1.19E+00 ± 1.82E+00 5.78E+00 ± 2.10E+01 4.78E−01 ± 1.31E+00 7.42E+00 ± 5.23E+00

7 1.29E−02 ± 9.11E−03 1.50E−02 ± 1.32E−02 1.82E−02 ± 1.99E−02 1.33E−02 ± 1.01E−02 1.48E−02 ± 1.36E−02 2.03E−02 ± 1.35E−02

8 2.09E+01 ± 1.39E−01 2.02E+01 ± 3.28E−01 2.01E+01 ± 3.02E−01 2.09E+01 ± 1.37E−01 2.02E+01 ± 3.61E−01 2.02E+01 ± 3.87E−01

9 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

10 3.53E+01 ± 5.74E+00 2.38E+01 ± 4.88E+00 1.86E+01 ± 3.75E+00 3.36E+01 ± 9.82E+00 2.73E+01 ± 8.69E+00 2.12E+01 ± 5.66E+00

11 2.74E+01 ± 1.57E+00 2.71E+01 ± 1.81E+00 2.61E+01 ± 1.75E+00 1.69E+01 ± 3.48E+00 1.70E+01 ± 3.43E+00 1.98E+01 ± 4.40E+00

12 4.99E+03 ± 4.32E+03 1.70E+03 ± 2.09E+03 6.26E+03 ± 6.67E+03 1.16E+03 ± 1.88E+03 1.50E+03 ± 2.17E+03 7.48E+03 ± 7.59E+03

13 1.87E+00 ± 1.52E−01 1.52E+00 ± 1.23E−01 1.59E+00 ± 1.17E−01 2.18E+00 ± 1.77E−01 1.71E+00 ± 1.08E−01 1.71E+00 ± 1.36E−01

14 1.26E+01 ± 2.21E−01 1.22E+01 ± 3.17E−01 1.20E+01 ± 4.19E−01 1.27E+01 ± 2.44E−01 1.10E+01 ± 9.76E−01 1.08E+01 ± 1.19E+00

15 3.69E+02 ± 9.08E+01 3.46E+02 ± 1.16E+02 3.92E+02 ± 6.40E+01 3.48E+02 ± 9.31E+01 3.50E+02 ± 7.35E+01 3.76E+02 ± 87.9E+01

16 7.20E+01 ± 5.46E+01 7.78E+01 ± 1.06E+02 3.53E+01 ± 4.21E+00 9.32E+01 ± 1.04E+02 6.39E+01 ± 7.30E+01 7.72E+01 ± 1.15E+02

17 1.35E+02 ± 8.02E+01 8.72E+01 ± 5.94E+01 8.25E+01 ± 7.19E+01 8.17E+01 ± 8.38E+01 8.55E+01 ± 1.15E+02 9.42E+01 ± 1.32E+02

18 8.96E+02 ± 3.93E+01 8.80E+02 ± 5.27E+01 9.04E+02 ± 1.15E+01 9.00E+02 ± 3.37E+01 9.02E+02 ± 3.05E+01 9.05E+02 ± 1.30E+00

19 8.89E+02 ± 4.49E+01 8.92E+02 ± 4.36E+01 9.05E+02 ± 1.04E+01 9.06E+02 ± 2.19E+01 9.09E+02 ± 1.59E+01 9.05E+02 ± 1.11E+00

20 8.93E+02 ± 4.12E+01 8.92E+02 ± 4.37E+01 9.05E+02 ± 9.54E−01 9.01E+02 ± 3.02E+01 9.07E+02 ± 2.22E+01 9.05E+02 ± 1.27E+00

21 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 1.93E−13 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 1.80E−13

22 9.10E+02 ± 1.04E+01 9.01E+02 ± 1.77E+01 8.72E+02 ± 1.64E+01 9.10E+02 ± 9.17E+00 8.87E+02 ± 1.80E+01 8.45E+02 ± 1.87E+01

23 5.34E+02 ± 7.89E−05 5.50E+02 ± 7.97E+01 5.34E+02 ± 1.71E−04 5.42E+02 ± 5.46E+01 5.34E+02 ± 2.34E−03 5.34E+02 ± 4.17E−04

24 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 1.02E−12 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 8.59E−13

25 2.12E+02 ± 1.33E−01 2.11E+02 ± 2.03E−01 2.11E+02 ± 1.06E+00 2.10E+02 ± 4.24E−01 2.10E+02 ± 2.04E−01 2.11E+02 ± 1.02E+00

JADE-O3 Rcr-JADE-O3 pbestrr-JADE-O3 JADE-O4 Rcr-JADE-O4 pbestrr-JADE-O4

1 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

2 4.77E−28 ± 1.84E−28 3.74E−28 ± 1.19E−28 1.17E−28 ± 1.38E−28 4.35E−28 ± 2.60E−28 3.78E−28 ± 1.98E−28 2.94E−28 ± 1.48E−28

3 9.45E+03 ± 7.33E+03 1.06E+04 ± 8.05E+03 7.82E+03 ± 6.06E+03 1.65E+04 ± 1.28E+04 1.50E+04 ± 1.29E+04 1.38E+04 ± 1.01E+04

4 2.28E−14 ± 1.34E−13 2.89E−12 ± 1.75E−11 1.42E−15 ± 3.24E−15 8.29E+02 ± 2.14E+03 6.37E−11 ± 3.17E−10 2.69E−14 ± 7.58E−14

5 3.97E−02 ± 1.34E−01 1.85E−01 ± 6.42E−01 2.46E−09 ± 5.83E−09 5.60E+00 ± 2.77E+01 2.04E−01 ± 8.02E−01 5.64E−08 ± 2.23E−07

6 7.08E+00 ± 2.65E+01 7.18E−01 ± 1.55E+00 1.59E−01 ± 7.97E−01 2.34E+00 ± 1.29E+01 1.59E−01 ± 7.89E−01 1.59E−01 ± 7.97E−01

7 7.83E−03 ± 8.86E−03 7.63E−03 ± 7.65E−03 8.37E−03 ± 8.92E−03 4.83E−03 ± 5.56E−03 5.12E−03 ± 6.94E−03 5.72E−03 ± 5.16E−03

8 2.09E+01 ± 6.23E−02 2.03E+01 ± 4.46E−01 2.03E+01 ± 4.40E−01 2.09E+01 ± 6.14E−02 2.04E+01 ± 4.56E−01 2.04E+01 ± 4.54E−01

9 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

10 3.20E+01 ± 8.31E+00 2.28E+01 ± 5.15E+00 1.65E+01 ± 3.27E+00 3.13E+01 ± 8.62E+00 2.47E+01 ± 9.35E+00 2.35E+01 ± 6.75E+00

11 2.18E+01 ± 6.88E+00 2.05E+01 ± 6.83E+00 2.59E+01 ± 1.49E+00 1.51E+01 ± 3.32E+00 1.60E+01 ± 3.25E+00 18.9E+01 ± 6.19E+00

12 3.76E+03 ± 4.16E+03 2.37E+03 ± 3.09E+03 3.26E+03 ± 4.25E+03 1.14E+03 ± 1.40E+03 1.51E+03 ± 2.77E+03 3.99E+03 ± 5.24E+03

13 1.82E+00 ± 1.57E−01 1.55E+00 ± 1.18E−01 1.58E+00 ± 1.21E−01 2.16E+00 ± 1.48E−01 1.69E+00 ± 1.11E−01 1.71E+00 ± 1.22E−01

14 1.25E+01 ± 2.40E−01 1.20E+01 ± 3.41E−01 1.19E+01 ± 4.87E−01 1.27E+01 ± 1.98E−01 1.12E+01 ± 1.02E+00 1.09E+01 ± 8.37E−01

15 3.54E+02 ± 9.73E+01 3.64E+02 ± 1.06E+02 3.72E+02 ± 9.79E+01 3.40E+02 ± 8.33E+01 3.48E+02 ± 6.46E+01 3.45E+02 ± 1.24E+01

16 6.86E+01 ± 5.47E+01 7.88E+01 ± 1.09E+02 4.24E+01 ± 1.85E+01 7.57E+01 ± 8.21E+01 5.60E+01 ± 5.53E+01 4.95E+01 ± 2.75E+01

17 1.62E+02 ± 1.20E+02 1.14E+02 ± 1.15E+02 5.72E+01 ± 8.05E+00 8.15E+01 ± 8.72E+01 8.75E+01 ± 1.12E+02 8.49E+01 ± 1.01E+02

18 8.88E+02 ± 4.45E+01 8.91E+02 ± 4.29E+01 9.04E+02 ± 2.20E+00 9.07E+02 ± 1.56E+01 9.10E+02 ± 2.20E+00 9.04E+02 ± 1.57E−01

19 8.99E+02 ± 3.35E+01 9.06E+02 ± 2.21E+01 9.04E+02 ± 9.68E-01 9.07E+02 ± 1.56E+01 9.10E+02 ± 2.49E+00 9.04E+02 ± 2.87E−01

20 8.99E+02 ± 3.35E+01 9.07E+02 ± 2.21E+01 9.04E+02 ± 8.16E−01 9.07E+02 ± 1.56E+01 9.10E+02 ± 2.49E+00 9.04E+02 ± 4.95E−01

21 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 1.18E−13 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 1.18E−13

22 9.06E+02 ± 1.19E+01 8.92E+02 ± 1.48E+01 8.61E+02 ± 2.39E+01 9.00E+02 ± 8.73E+00 8.63E+02 ± 1.47E+01 8.38E+02 ± 23.7E+01

23 5.50E+02 ± 7.76E+01 5.42E+02 ± 5.70E+01 5.34E+02 ± 3.71E−04 5.34E+02 ± 3.51E−04 5.34E+02 ± 3.71E−04 5.34E+02 ± 3.59E−04

24 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00

25 2.12E+02 ± 1.05E−01 2.10E+02 ± 3.85E−01 2.10E+02 ± 8.21E−01 2.09E+02 ± 1.32E−01 2.09E+02 ± 8.67E−02 2.10E+02 ± 6.73E−01

f

a

t

(

b

s

v

p

c

p

r

p

e

t

i

R

o

f

b

p

f

s

p

p

2

g

s

a

c

e

a

s

p

unction evaluations (FES) of these six algorithms was set to 300,000

t D = 30, which are the same as pbestrr-JADE and all the experimen-

al results are taken from 25 independent runs as Suganthan et al.

2005) suggested.

From Tables 4 and 5, we can see that new algorithm shares the

est performance on unimodal functions (F01–F05) with JADE. For

even basic multimodal functions, pbestrr-JADE-O3 shows great ad-

antage over JADE and Rank-JADE; compared with CoDE, the pro-

osed algorithm performs similarly; but, it shows some weakness

ompared with Rcr-JADE. One point should be mentioned that the

roposed algorithm can perform much better than other five algo-

ithms for F10. For two expanded multimodal functions, the pro-

osed algorithm is better than some algorithms but worse than oth-

rs. There is not one algorithm that can beat all other algorithm in

hese two functions.

For hybrid composition functions, the proposed algorithm shows

ts advantage over other six algorithms. Compared with SaDE and

ank-JADE, pbestrr-JADE-O3 obtain better or at least similar results

n these eleven functions. Compared with JADE, they perform equally

or five functions; for F16 and F17, the proposed algorithm is much

etter; for F22 and F25, the proposed algorithm show minor im-

rovement. Compared with CoDE, the proposed algorithm outper-

orms over eight functions; for F23 and F25, the proposed algorithm

hows similar performance with CoDE. Compared with Rcr-JADE, the

roposed algorithm performs better; especially for F18–F20, the pro-

osed algorithm shows minor improvement.

In summary, from the results of Wilcoxon signed-rank test over

5 benchmark functions, we can conclude that the proposed al-

orithm presents the best performance compared with other five

tate-of the-art algorithms. The comparison shows that the proposed

lgorithm has better global search ability while without losing the lo-

al search ability. Separately speaking, pbestrr-JADE-O3 enhances the

xploration ability of JADE, and not only improves the local search

bility of Rcr-JADE for first five functions but also increases its global

earch ability. The recently proposed CoDE, Rcr-JADE and Rank-JADE

erforms better than previous algorithms with its excellent global

8 W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12

Table 2

Comparison between JADE, Rcr-JADE and its corresponding pbestrr-JADE for CEC2005 at D = 50.

JADE-O1 Rcr-JADE-O1 pbestrr-JADE-O1 JADE-O2 Rcr-JADE-O2 pbestrr-JADE-O2

1 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

2 7.69E−21 ± 1.50E−20 1.09E−20 ± 2.17E−20 1.50E−10 ± 2.17E−10 5.18E+03 ± 8.61E+03 5.08E−19 ± 1.62E−18 1.48E−11 ± 2.37E−11

3 1.95E+04 ± 9.19E+03 2.31E+04 ± 1.06E+04 6.36E+04 ± 2.61E+04 1.45E+06 ± 7.01E+06 3.26E+04 ± 1.38E+04 6.35E+04 ± 2.79E+04

4 1.12E+01 ± 1.87E+01 2.76E+01 ± 4.24E+01 4.44E+01 ± 7.32E+01 1.43E+04 ± 1.90E+04 6.18E+02 ± 4.21E+03 1.52E+01 ± 1.57E+01

5 2.48E+03 ± 4.87E+02 2.50E+03 ± 4.55E+02 2.15E+03 ± 4.12E+02 2.65E+03 ± 5.87E+02 2.54E+03 ± 3.71E+02 2.49E+03 ± 4.07E+02

6 3.97E+00 ± 1.39E+01 2.07E+00 ± 2.01E+00 4.29E+01 ± 4.39E+01 3.61E+00 ± 1.53E+01 1.28E+00 ± 1.88E+00 5.71E+01 ± 3.60E+01

7 6.78E−03 ± 1.15E−02 8.90E−03 ± 1.27E−02 5.41E−03 ± 9.30E−03 1.77E−03 ± 4.14E−03 2.46E−03 ± 9.37E−03 7.18E−03 ± 1.24E−02

8 2.11E+01 ± 2.71E−01 2.03E+01 ± 5.06E−01 2.03E+01 ± 4.74E−01 2.11E+01 ± 2.52E−01 2.05E+01 ± 5.38E−01 2.05E+01 ± 5.20E−01

9 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 7.81E−16 ± 1.15e-15 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 3.55E−16 ± 7.25E−16

10 6.57E+01 ± 1.06E+01 6.49E+01 ± 1.16E+01 4.18E+01 ± 5.99E+00 5.15E+01 ± 1.07E+01 4.87E+01 ± 1.29E+01 3.80E+01 ± 1.07E+01

11 5.26E+01 ± 2.44E+00 5.30E+01 ± 2.30E+00 5.17E+01 ± 2.88E+00 5.28E+01 ± 8.21E+00 4.80E+01 ± 1.20E+01 4.42E+01 ± 1.08E+01

12 1.56E+04 ± 1.76E+04 5.96E+03 ± 7.43E+03 1.80E+04 ± 1.37E+04 2.81E+04 ± 2.67E+04 9.10E+03 ± 1.12E+04 1.90E+04 ± 1.68E+04

13 2.65E+00 ± 1.91E−01 2.77E+00 ± 2.20E−01 2.90E+00 ± 1.66E−01 2.89E+00 ± 1.74E−01 3.06E+00 ± 1.72E-01 3.13E+00 ± 1.74E-01

14 2.17E+01 ± 3.24E−01 2.14E+01 ± 3.96E−01 2.16E+01 ± 3.59E−01 2.19E+01 ± 9.25E−01 2.11E+01 ± 1.08E+00 2.02E+01 ± 1.53E+00

15 3.34E+02 ± 9.20E+01 3.25E+02 ± 9.54E+01 3.84E+02 ± 5.37E+01 3.26E+02 ± 9.43E+01 3.04E+02 ± 1.07E+02 3.44E+02 ± 9.16E+01

16 7.55E+01 ± 7.40E+01 5.66E+01 ± 5.17E+01 3.56E+01 ± 3.34E+00 9.88E+01 ± 1.25E+02 6.28E+01 ± 7.38E+01 3.67E+01 ± 9.71E+00

17 1.11E+02 ± 4.96E+01 1.11E+02 ± 6.57E+01 8.23E+01 ± 1.02E+01 6.60E+01 ± 4.22E+01 7.58E+01 ± 9.85E+01 4.16E+01 ± 7.70E+00

18 9.40E+02 ± 3.10E+01 9.34E+02 ± 3.64E+01 9.12E+02 ± 4.26E+01 9.39E+02 ± 8.35E+00 9.36E+02 ± 2.94E+01 9.26E+02 ± 2.68E+01

19 9.40E+02 ± 2.28E+01 9.39E+02 ± 1.90E+01 8.99E+02 ± 5.10E+01 9.39E+02 ± 8.73E+00 9.43E+02 ± 7.45E+00 9.31E+02 ± 6.35E+00

20 9.39E+02 ± 2.27E+01 9.41E+02 ± 1.70E+01 9.24E+02 ± 4.57E+01 9.39E+02 ± 9.06E+00 9.42E+02 ± 7.41E+00 9.26E+02 ± 5.38E+00

21 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00

22 9.48E+02 ± 9.61E+00 9.50E+02 ± 8.82E+00 8.97E+02 ± 1.76E+01 9.25E+02 ± 2.09E+01 9.19E+02 ± 1.31E+01 9.03E+02 ± 1.60E+01

23 5.59E+02 ± 1.04E+02 5.46E+02 ± 4.94E+01 5.39E+02 ± 3.47E-01 5.39E+02 ± 5.21E-03 5.39E+02 ± 1.75E-02 5.39E+02 ± 1.38E+00

24 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00

25 2.14E+02 ± 8.67E−01 2.14E+02 ± 7.00E−01 2.19E+02 ± 2.80E+00 2.15E+02 ± 7.66E−01 2.15E+02 ± 8.85E−01 2.19E+02 ± 4.73E+00

JADE-O3 Rcr-JADE-O3 pbestrr-JADE-O3 JADE-O4 Rcr-JADE-O4 pbestrr-JADE-O4

1 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

2 1.09E−26 ± 7.27E−27 1.15E−26 ± 5.27E−27 1.95E−22 ± 4.35E−22 6.98E+03 ± 9.70E+03 2.14E−26 ± 1.64E−26 9.48E−24 ± 1.28E−23

3 1.70E+04 ± 1.01E+04 1.57E+04 ± 7.74E+03 2.84E+04 ± 1.11E+04 3.24E+06 ± 8.22E+06 2.46E+04 ± 1.35E+04 3.29E+04 ± 1.19E+04

4 3.79E+00 ± 1.70E+01 2.97E+00 ± 1.05E+01 5.49E+00 ± 2.37E+01 1.15E+04 ± 1.69E+04 8.21E+02 ± 5.80E+03 5.48E+00 ± 1.53E+01

5 1.89E+03 ± 3.98E+02 1.81E+03 ± 4.43E+02 1.30E+03 ± 5.52E+02 2.08E+03 ± 9.91E+02 1.74E+03 ± 3.74E+02 1.23E+03 ± 4.40E+02

6 1.12E+00 ± 1.81E+00 1.67E+00 ± 1.99E+00 1.11E+00 ± 1.82E+00 3.99E-01 ± 1.21E+00 5.58E-01 ± 1.40E+00 1.27E+00 ± 1.89E+00

7 4.92E−03 ± 9.15E−03 3.20E−03 ± 5.95E−03 4.14E−03 ± 6.10E−03 4.38E−03 ± 7.43E−03 1.87E−03 ± 5.36E−03 4.53E−03 ± 7.06E−03

8 2.11E+01 ± 2.69E−01 2.07E+01 ± 5.33E−01 2.05E+01 ± 5.85E−01 2.11E+01 ± 2.72E−01 2.07E+01 ± 5.51E−01 2.05E+01 ± 5.75E−01

9 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 2.13E−16 ± 5.89E−16 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

10 6.42E+01 ± 8.91E+00 5.61E+01 ± 9.72E+00 3.69E+01 ± 5.20E+00 4.90E+01 ± 1.13E+01 5.12E+01 ± 1.18E+01 3.90E+01 ± 1.33E+01

11 5.23E+01 ± 2.20E+00 5.24E+01 ± 2.27E+00 5.22E+01 ± 2.98E+00 5.53E+01 ± 7.90E+00 4.32E+01 ± 1.15E+01 3.98E+01 ± 7.77E+00

12 2.09E+04 ± 2.24E+04 8.87E+03 ± 1.45E+04 1.06E+04 ± 1.46E+04 3.00E+04 ± 2.67E+04 6.89E+03 ± 1.15E+04 1.41E+04 ± 1.90E+04

13 2.69E+00 ± 1.90E−01 2.86E+00 ± 1.66E−01 2.93E+00 ± 1.92E−01 2.94E+00 ± 1.69E−01 3.04E+00 ± 2.05E−01 3.17E+00 ± 1.50E−01

14 2.17E+01 ± 3.53E−01 2.15E+01 ± 4.86E−01 2.14E+01 ± 4.55E−01 2.17E+01 ± 1.03E+00 2.08E+01 ± 1.24E+00 1.96E+01 ± 1.56E+00

15 3.46E+02 ± 8.80E+01 3.22E+02 ± 9.51E+01 3.31E+02 ± 9.48E+01 3.06E+02 ± 9.77E+01 3.10E+02 ± 1.04E+02 3.32E+02 ± 8.99E+01

16 6.73E+01 ± 6.99E+01 6.27E+01 ± 7.12E+01 4.34E+01 ± 2.71E+01 5.21E+01 ± 5.22E+01 5.02E+01 ± 2.47E+01 5.32E+01 ± 7.24E+01

17 1.17E+02 ± 6.24E+01 9.79E+01 ± 2.67E+01 7.84E+01 ± 9.08E+01 8.08E+01 ± 6.91E+01 6.33E+01 ± 7.27E+01 9.85E+01 ± 1.35E+02

18 9.33E+02 ± 3.60E+01 9.29E+02 ± 4.06E+01 9.19E+02 ± 4.20E+00 9.31E+02 ± 2.03E+01 9.30E+02 ± 2.78E+01 9.20E+02 ± 2.78E+00

19 9.36E+02 ± 2.30E+01 9.38E+02 ± 2.98E+01 9.19E+02 ± 7.04E+00 9.29E+02 ± 2.78E+01 9.35E+02 ± 2.29E+01 9.20E+02 ± 3.00E+00

20 9.35E+02 ± 2.26E+01 9.36E+02 ± 2.97E+01 9.15E+02 ± 2.44E+01 9.28E+02 ± 2.77E+01 9.35E+02 ± 2.24E+01 9.21E+02 ± 2.33E+00

21 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 7.02E+02 ± 2.52E+02 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00

22 9.48E+02 ± 9.49E+00 9.44E+02 ± 1.12E+01 8.94E+02 ± 1.97E+01 9.21E+02 ± 2.63E+01 9.05E+02 ± 1.33E+01 8.72E+02 ± 2.30E+01

23 5.39E+02 ± 3.26E−03 5.39E+02 ± 7.48E−03 7.09E+02 ± 2.32E+02 5.39E+02 ± 6.38E−03 5.39E+02 ± 8.89E−03 5.39E+02 ± 3.41E−01

24 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00

25 2.14E+02 ± 9.13E−01 2.14E+02 ± 6.18E−01 2.18E+02 ± 2.86E+00 2.14E+02 ± 9.23E−01 2.14E+02 ± 5.07E−01 2.16E+02 ± 1.51E+00

Fig. 8. The convergence curve of different state-of-the-art DE algorithms on the

function f3.

search ability, but pbestrr-JADE-O3 can present a better performance

when comparing with them.

Then we give four representative types of benchmark function’s

convergence speed curve of five algorithms in Figs. 8–11. From

Figs. 8–11, we can conclude that the proposed algorithm not only can

achieve good experimental results, but also can obtain a fast conver-

gence speed.

In terms of the limitation of the proposed pbestrr-JADE-O3, it can-

not obtain a more precise solution on the last eleven hybrid compo-

sition functions, although it can provide promising results than other

state-of-the-art DE based algorithms. The reason is the proposed al-

gorithm is still under the framework of JADE algorithm, so it is hard

to improve the results in a magnitude level.

4.4. Real-world applications

From the previous experimental results that are obtained on

the CEC2005 benchmark functions, the proposed pbestrr-JADE-O3 is

W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12 9

Table 3

Comparison on the diverse population size setting for the pbestrr-JADE-O3 algorithm.

Population size=50 Population size=100 Population size=150 Population size=200

1 8.08E−30 ± 4.04E−29 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

2 5.96E−28 ± 3.50E−28 1.17E−28 ± 1.38E−28 1.50E−29 ± 3.39E−29 3.03E−30 ± 8.37E−30

3 8.20E+03 ± 6.78E+03 7.82E+03 ± 6.06E+03 3.00E+03 ± 2.68E+03 1.57E+02 ± 7.82E+02

4 1.14E-04 ± 3.91E−04 1.42E−15 ± 3.24E−15 9.27E−21 ± 2.08E−20 5.75E−25 ± 2.82E−24

5 7.54E−01 ± 1.40E+00 2.46E−09 ± 5.83E−09 1.78E−09 ± 5.11E−09 3.90E−10 ± 1.11E−09

6 9.57E−01 ± 1.74E+00 1.59E−01 ± 7.97E−01 1.59E−01 ± 7.97E−01 1.59E−01 ± 7.97E−01

7 1.49E−02 ± 1.00E−02 8.37E−03 ± 8.92E−03 4.83E−03 ± 4.47E−03 3.45E−03 ± 4.93E−03

8 2.03E+01 ± 4.27E−01 2.03E+01 ± 4.40E−01 2.06E+01 ± 4.42E−01 2.09E+01 ± 2.33E−01

9 3.55E−16 ± 7.25E−16 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

10 2.66E+01 ± 7.84E+00 1.65E+01 ± 3.27E+00 2.51E+01 ± 4.18E+00 3.30E+01 ± 4.35E+00

11 2.68E+01 ± 1.97E+00 2.59E+01 ± 1.49E+00 2.59E+01 ± 1.89E+00 2.63E+01 ± 2.05E+00

12 3.86E+03 ± 3.97E+03 3.26E+03 ± 4.25E+03 2.89E+03 ± 3.82E+03 2.28E+03 ± 3.40E+03

13 1.24E+00 ± 1.28E−01 1.57E+00 ± 1.21E−01 2.06E+00 ± 1.45E−01 2.38E+00 ± 1.70E−01

14 1.22E+01 ± 3.92E−01 1.19E+01 ± 4.87E-01 1.22E+01 ± 2.35E−01 1.23E+01 ± 210E−01

15 3.25E+02 ± 1.09E+02 3.72E+02 ± 9.79E+01 3.80E+02 ± 1.16E+02 3.96E+02 ± 6.11E+001

16 1.58E+02 ± 1.72E+02 4.24E+01 ± 1.85E+01 8.59E+01 ± 9.97E+01 5.74E+01 ± 2.41E+01

17 1.62E+02 ± 1.51E+02 5.72E+01 ± 8.05E+00 1.05E+02 ± 9.16E+01 1.15E+02 ± 8.75E+01

18 9.06E+02 ± 1.22E+00 9.04E+02 ± 2.20E+00 9.04E+02 ± 9.46E−01 9.04E+02 ± 1.86E−01

19 9.06E+02 ± 1.35E+00 9.04E+02 ± 9.68E−01 9.04E+02 ± 7.62E−01 9.04E+02 ± 6.07E−01

20 9.05E+02 ± 1.19E+00 9.04E+02 ± 8.16E−01 9.04E+02 ± 4.33E−01 9.04E+02 ± 1.81E−01

21 5.13E+02 ± 6.44E+01 5.00E+02 ± 1.18E−13 5.00E+02 ± 9.71E−14 5.00E+02 ± 8.12E−14

22 8.84E+02 ± 2.56E+01 8.61E+02 ± 2.39E+01 8.70E+02 ± 2.06E+01 8.67E+02 ± 1.85E+01

23 5.53E+02 ± 8.08E+01 5.34E+02 ± 3.71E−04 5.34E+02 ± 4.35E−13 5.34E+02 ± 3.32E−13

24 2.61E+02 ± 2.11E+02 2.00E+02 ± 0.00E+00 2.00E+02 ± 2.90E−14 2.00E+02 ± 2.90e−14

25 2.26E+02 ± 5.73E+01 2.10E+02 ± 8.21E−01 2.10E+02 ± 7.63E−01 2.10E+02 ± 5.81E−01

Total 3 17 12 16

Table 4

Indirect comparison among different state-of-the-art DE algorithms for all functions at D = 30.

SaDE JADE CoDE Rank-JADE Rcr-JADE pbestrr-JADE-O3

1 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

2 8.26E−06 ± 1.65E−05 1.07E−28 ± 1.00E−28 1.69E−15 ± 3.95E−15 2.99E−28 ± 1.25E−28 3.78E−28 ± 1.98E−28 1.17E−28 ± 1.38E−28

3 4.27E+05 ± 2.08E+05 8.42E+03 ± 7.26E+03 1.05E+05 ± 6.25E+04 7.67E+03 ± 6.70E+03 1.50E+04 ± 1.29E+04 7.82E+03 ± 6.06E+03

4 1.77E+02 ± 2.67E+02 1.73E−16 ± 5.43E−16 5.81E−03 ± 1.38E−02 5.61E−16 ± 3.03E−15 6.37E−11 ± 3.17E−10 1.42E−15 ± 3.24E−15

5 3.25E+03 ± 5.90E+02 8.59E−08 ± 5.23E−07 3.31E+02 ± 3.44E+02 4,.77E−02 ± 1.59E−01 2.04E−01 ± 8.02E−01 2.46E−09 ± 5.83E−09

6 5.31E+01 ± 3.25E+01 1.02E+01 ± 2.96E+01 1.60E−01 ± 7.85E−01 7.74E−01 ± 3.87E+00 1.59E−01 ± 7.89E−01 1.59E−01 ± 7.97E−01

7 1.57E−02 ± 1.38E−02 8.07E−03 ± 7.42E−03 7.46E−03 ± 8.55E−03 6.06E−03 ± 7.82E−03 5.12E−03 ± 6.94E−03 8.37E−03 ± 8.92E−03

8 2.09E+01 ± 4.95E−02 2.09E+01 ± 1.68E−01 2.01E+01 ± 1.41E−01 2.09E+01 ± 1.43E−01 2.04E+01 ± 4.56E−01 2.03E+01 ± 4.40E−01

9 2.39E−01 ± 4.33E−01 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00 0.00E+00 ± 0.00E+00

10 4.72E+01 ± 1.01E+01 2.41E+01 ± 4.61E+00 4.15E+01 ± 1.16E+01 2.48E+01 ± 4.66E+00 2.47E+01 ± 9.35E+00 1.65E+01 ± 3.27E+00

11 1.65E+01 ± 2.42E+00 2.53E+01 ± 1.65E+00 1.18E+01 ± 3.40E+00 2.55E+01 ± 1.58E+00 1.60E+01 ± 3.25E+00 2.59E+01 ± 1.49E+00

12 3.02E+03 ± 2.33E+03 6.15E+03 ± 4.79E+03 3.05E+03 ± 3.80E+03 3.91E+03 ± 3.88E+03 1.51E+03 ± 2.77E+03 3.26E+03 ± 4.25E+03

13 3.94E+00 ± 2.81E−01 1.49E+00 ± 1.09E−01 1.57E+00 ± 3.27E−01 1.47E+00 ± 1.08E−01 1.69E+00 ± 1.11E−01 1.57E+00 ± 1.21E−01

14 1.26E+01 ± 2.83E−01 1.23E+01 ± 3.11E−01 1.23E+01 ± 4.81E−01 1.22E+01 ± 3.29E−01 1.12E+01 ± 1.02E+00 1.19E+01 ± 4.87E−01

15 3.76E+02 ± 7.83E+01 3.51E+02 ± 1.28E+02 3.88E+02 ± 6.85E+01 3.56E+02 ± 9.29E+01 3.48E+02 ± 6.46E+01 3.72E+02 ± 9.79E+01

16 8.57E+01 ± 6.94E+01 1.01E+02 ± 1.24E+02 7.37E+01 ± 5.13E+01 8.83E+01 ± 1.12E+02 5.60E+01 ± 5.53E+01 4.24E+01 ± 1.85E+01

17 7.83E+01 ± 3.76E+01 1.47E+02 ± 1.33E+02 6.67E+01 ± 2.12E+01 1.05E+02 ± 8.49E+01 8.75E+01 ± 1.12E+02 5.72E+01 ± 8.05E+00

18 8.68E+02 ± 6.23E+01 9.04E+02 ± 1.03E+00 9.04E+02 ± 1.04E+00 9.02E+02 ± 2.62E−01 9.10E+02 ± 2.20E+00 9.04E+02 ± 2.20E+00

19 8.74E+02 ± 6.22E+01 9.04E+02 ± 8.40E−01 9.04E+02 ± 9.42E−01 9.05E+02 ± 2.18E−01 9.10E+02 ± 2.49E+00 9.04E+02 ± 9.68E−01

20 8.78E+02 ± 6.03E+01 9.04E+02 ± 8.47E−01 9.04E+02 ± 9.01E−01 8.95E+02 ± 3.57E+01 9.10E+02 ± 2.49E+00 9.04E+02 ± 8.16E−01

21 5.52E+02 ± 1.82E+02 5.00E+02 ± 4.67E−13 5.00E+02 ± 4.88E−13 5.00E+02 ± 0.00E+00 5.00E+02 ± 0.00E+00 5.00E+02 ± 1.18E−13

22 9.36E+02 ± 1.83E+01 8.66E+02 ± 1.91E+01 8.63E+02 ± 2.43E+01 8.90E+02 ± 1.37E−01 8.63E+02 ± 1.47E+01 8.61E+02 ± 2.39E+01

23 5.34E+02 ± 3.57E−03 5.50E+02 ± 8.05E+01 5.34E+02 ± 4.12E−04 5.34E+02 ± 2.82E−03 5.34E+02 ± 3.71E−04 5.34E+02 ± 3.71E−04

24 2.00E+02 ± 6.20E−13 2.00E+02 ± 2.85E-14 2.00E+02 ± 2.85E−14 2.00E+00 ± 0.00E+00 2.00E+02 ± 0.00E+00 2.00E+02 ± 0.00E+00

25 2.14E+02 ± 2.00E+00 2.11E+02 ± 7.92E−01 2.11E+02 ± 9.02E−01 2.09E+02 ± 1.05E−01 2.09E+02 ± 2.51E−01 2.10E+02 ± 8.21E−01

h

a

s

f

l

a

a

r

m

p

p

p

a

1

r

s

r

p

S

w

w

t

ighly competitive when comparing with other state-of-the-art DE

lgorithms. To evaluate the performance of the pbestrr-JADE-O3 in

olving real-world problems, four real-world problems are collected

rom CEC2011 contest on real world numerical optimization prob-

ems (Das & Suganthan, 2010) and SaDE, JADE, CoDE, Rank-JADE,

nd Rcr-JADE are chosen to make a comparison with the proposed

lgorithm. As stated by Das and Suganthan [21], the selected four

eal-world problems are: (P01) parameter estimation for frequency-

odulated sound waves (D = 6); (P02) spread spectrum radar polly

hase code design (D = 20); (P03) transmission network expansion

lanning problem (D = 7); and (P04) circular antenna array design

roblem (D = 12).

The parameter settings are the same in previous sections for

ll the algorithms. The maximum function evaluation (maxFES) is

50,000 for all the problems. The results after executing the algo-

ithms for 50,000, 100,000, 150,000 function evaluations are pre-

ented. The experimental results are obtained over 25 independent

uns and described in Table 6.

According to the results in Table 6, we can see that the pro-

osed pbestrr-JADE is still highly competitive when comparing with

aDE, JADE, Rank-JADE, and Rcr-JADE algorithms. When comparing

ith CoDE, the proposed algorithm shows a little inferior at maxFES,

hich obtains three similar results and one inferior result over the

ested four real-world problems. According to the requirement that

10 W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12

Table 5

Direct comparison among different state-of-the-art DE algorithms for all functions at D = 30.

SaDE JADE CoDE Rank-JADE Rcr-JADE pbestrr-JADE-O3

1 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00

2 4.71E-06 ± 8.10E−06 + 1.00E-28 ± 8.53E−29 = 1.39E-15 ± 2.05E−15 + 7.69E-29 ± 9.80E−29 = 3.44E-28 ± 1.83E−28 + 8.59E-29 ± 7.61E−29

3 4.09e+05 ± 1.64E+05 + 6.06E+03 ± 4.46E+03 = 1.31E+05 ± 6.66E+04 + 7.28E+03 ± 5.32E+03 = 1.50E+04 ± 1.09E+04 + 7.36E+03 ± 5.13E+03

4 1.64E+02 ± 1.49E+02 + 9.38E-15 ± 3.11E−14 = 3.40E-03 ± 5.65E−03 + 3.76E-15 ± 1.26E−14 = 7.51E-13 ± 2.23E−12 + 1.42E-15 ± 3.24E−15

5 3.51E+03 ± 6.35E+02 + 2.13E-07 ± 9.77E−07 = 6.60E+02 ± 4.94E+02 - 2.56E-02 ± 5.83E−02 + 1.29E-07 ± 5.57E−07 = 2.46E-09 ± 5.83E−09

6 4.64E+01 ± 3.28E+01 + 1.28E+01 ± 3.56E+01 = 1.32E-10 ± 4.19E−10 + 9.50E+00 ± 2.51E+01 + 1.59E-01 ± 7.97E−01 = 1.59E-01 ± 7.97E−01

7 2.66E-02 ± 1.38E−02 + 7.59E-03 ± 6.70E−03 = 1.20E+00 ± 1.46E−01 + 1.07E-02 ± 9.47E−03 + 5.80E-03 ± 6.40E−03 = 8.37E-03 ± 8.92E−03

8 2.09E+01 ± 7.21E−02 + 2.09E+01 ± 1.94E−01 + 2.01E+01 ± 1.62E−01 = 2/09E+01 ± 7.39E−02 + 2.03E-01 ± 4.68E−01 = 2.03E+01 ± 4.27E−01

9 7.96E-02 ± 2.75E−01 + 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00 = 0.00E+00 ± 0.00E+00

10 4.78E+01 ± 9.95E+00 + 2.39E+01 ± 4,27E+00 + 4.15E+01 ± 1.38E+01 + 2.46E+01 ± 4.60E+00 + 2.08E+01 ± 5.57E+00 + 1.88E+01 ± 3.51E+00

11 1.64E+01 ± 2.79E+00 − 2.49E+01 ± 1.94E+00 − 1.25E+01 ± 3.85E+00 − 2.52E+01 ± 1.17E+00 = 1.87E+01 ± 4.54E+00 − 2.58E+01 ± 1.79E+00

12 3.65E+03 ± 2.87E+03 = 7.05E+03 ± 4.61E+03 + 3.36E+03 ± 3.81E+03 = 7.37E+03 ± 4.33E+03 + 2.21e+03 ± 3.45E+03 = 3.26E+03 ± 4.25E+03

13 3.88E+00 ± 3.66E−01 + 1.47E+00 ± 1.02E−01 − 1.60E+00 ± 2.58E−01 = 1.49E+00 ± 1.28E−01 = 1.66E+00 ± 1.16E−01 + 1.56E+00 ± 1.11E−01

14 1.26E+01 ± 2.99E−01 + 1.22E+01 ± 3.25E−01 + 1.24E+01 ± 5.51E−01 + 1.24E+01 ± 2.72E−01 + 1.09E+01 ± 7.80E−01 − 1.20E+01 ± 4.20E−01

15 4.16E+02 ± 6.23E+01 + 3.76E+02 ± 1.01E+02 = 4.04E+02 ± 5.39E+01 = 3.60E+02 ± 1.00E+02 = 3.64E+02 ± 7.00E+01 = 3.28E+02 ± 1.21E+02

16 9.86E+01 ± 1.08E+02 + 1.69E+02 ± 1.73E+02 = 6.38E+01 ± 1.20E+01 + 9.54E+01 ± 1.29E+02 = 1.14E+02 ± 1.43E+02 + 4.24E+01 ± 1.85E+01

17 1.21E+02 ± 1.26E+02 + 1.28E+02 ± 1.21E+02 + 8.43E+01 ± 7.77E+01 + 9.40E+01 ± 7.08E+01 + 1.43E+02 ± 1.43E+02 = 5.72E+01 ± 8.05E+00

18 8.72E+02 ± 6.10E+01 = 9.04E+02 ± 4.05E−01 = 9.05E+02 ± 1.18E+00 + 9.09E+02 ± 2.23E+00 + 9.04E+02 ± 5.07E−01 = 9.04E+02 ± 2.20E+00

19 8.73E+02 ± 6.14E+01 = 9.04E+02 ± 8.66E−01 + 9.04E+02 ± 8.40E−01 + 9.01E+02 ± 3.05E+01 + 9.04E+02 ± 6.98E-01 = 9.04E+02 ± 9.68E−01

20 8.84E+02 ± 5.90E+01 = 9.04E+02 ± 5.97E−01 + 9.04E+02 ± 1.11E+00 + 8.96E+02 ± 3.64E+01 + 9.05E+02 ± 1.13E+00 + 9.04E+02 ± 8.16E−01

21 5.00E+02 ± 2.09E−13 = 5.00E+02 ± 6.14E−14 = 5.12E+02 ± 6.00E+01 + 5.12E+02 ± 6.00E+01 + 5.00E+02 ± 8.84E−14 = 5.00E+02 ± 1.18E−13

22 9.32E+02 ± 1.51E+01 + 8.69E+02 ± 2.43E+01 + 8.66E+02 ± 2.02E+01 + 8.98E+02 ± 1.15E+01 + 8.45E+02 ± 2.69E+01 = 8.61E+02 ± 2.39E+01

23 5.34E+02 ± 4.30E−03 = 5.34E+02 ± 1.58E−04 − 5.34E+02 ± 4.45E−04 = 5.34E+02 ± 1.12E-04 = 5.34E+02 ± 4.24E−04 = 5.34E+02 ± 1.72E−04

24 2.00E+02 ± 6.21E−13 = 2.00E+02 ± 2.90E−14 = 2.00E+02 ± 2.90E−14 = 2.00E+02 ± 2.90E−14 = 2.00E+02 ± 2.90E−14 = 2.00E+02 ± 2.90E−14

25 2.13E+02 ± 1.89E+00 + 2.11E+02 ± 7.69E−01 + 2.35E+02 ± 5.55E+00 + 2.11E+02 ± 7.77E−01 = 2.11E+02 ± 9.30E−01 + 2.10E+02 ± 8.21E−01

+ 16 9 15 13 8

= 8 13 8 12 14

− 1 3 2 0 3

“+”, “−”, and “=” indicate the proposed algorithm is respectively better than, worse than, or similar to its competitor according to the Wilcoxon signed-rank test at α = 0.05.

Fig. 9. The convergence curve of different state-of-the-art DE algorithms on the

function f10.

Fig. 10. The convergence curve of different state-of-the-art DE algorithms on the

function f13.

Fig. 11. The convergence curve of different state-of-the-art DE algorithms on the

function f16∗In Figures 8-11, pbestrr-JADE is short for pbestrr-JADE-O3

D

1

t

p

p

J

W

t

5

a

w

n

r

w

a

as and Suganthan (2010) written, the intermediate results at 50,000,

00,000 function evaluations are also presented in Table 6. The in-

ermediate results at 50,000 function evaluations show that the pro-

osed algorithm can obtain highly competitive results except for

roblem P02 comparing with CoDE. We can see that the pbestrr-

ADE-O3 can achieve promising results at small function evaluations.

e can conclude that the proposed algorithm still effective in solving

hese four real-world problems.

. Conclusions and future work

Many works have been done in adopting diverse mutation oper-

tors. To enhance the performance of JADE algorithm, in this paper,

e propose a pbest roulette wheel selection and retention mecha-

ism based repairing crossover control parameter adaptive DE algo-

ithm, i.e. by modifying the pbest selection mechanism into roulette

heel selection and preserving the pbest individual that gener-

tes better offspring. Four mutation operators are tested, which are

W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12 11

Table 6

Comparison between different state-of-the-art DE algorithms and pbestrr-JADE on four real-world applications.

SaDE JADE CoDE Rank-JADE Rcr-JADE pbestrr-JADE

P01 5 × 104 Best 0.00E+00 + 1.14E−01 + 2.43E−25 + 9.10E−02 + 0.00E+00 + 0.00E+00

Median 4.45E−03 4.69E+00 2.02E−10 2.34E+00 7.21E−14 0.00E+00

Worst 1.21E+01 1.33E+01 1.25E+01 1.20E+01 1.25E+01 1.23E+01

Mean 2.49E+00 5.23E+00 2.02E+00 4.18E+00 2.73E+00 9.40E−01

Std 3.86E+00 4.18E+00 4.20E+00 3.69E+00 4.88E+00 3.23E+00

1 × 105 Best 0.00E+00 − 6.61E−07 + 0.00E+00 = 1.74E−02 + 0.00E+00 + 0.00E+00

Median 0.00E+00 2.30E−01 0.00E+00 3.11E−01 0.00E+00 0.00E+00

Worst 0.00E+00 1.17E+01 1.14E+01 1.98E+00 1.14E+01 1.17E+01

Mean 0.00E+00 1.19E+00 9.03E−01 4.49E−01 9.03E−01 8.75E−01

Std 0.00E+00 2.55E+00 3.13E+00 4.64E−01 3.13E+00 2.98E+00

1.5 × 105 Best 0.00E+00 = 2.03E−07 + 0.00E+00 = 1.63E−04 + 0.00E+00 = 0.00E+00

Median 0.00E+00 4.86E−02 0.00E+00 3.54E−02 0.00E+00 0.00E+00

Worst 0.00E+00 5.34E−01 0.00E+00 1.95E+00 0.00E+00 0.00E+00

Mean 0.00E+00 1.12E−01 0.00E+00 1.74E−01 0.00E+00 0.00E+00

Std 0.00E+00 1.35E−01 0.00E+00 4.00E−01 0.00E+00 0.00E+00

P02 5 × 104 Best 1.25E+00 = 1.12E+00 = 5.02E-01 − 9.86E-01 + 1.17E+00 = 5.79E−01

Median 1.48E+00 1.36E+00 1.24E+00 1.42E+00 1.38E+00 1.54E+00

Worst 1.78E+00 1.49E+00 1.62E+00 1.34E+00 1.53E+00 2.05E+00

Mean 1.47E+00 1.34E+00 1.19E+00 1.40E+00 1.36E+00 1.49E+00

Std 1.05E−01 8.40E-02 2.70E−01 1.13E−01 9.80E−01 4.21E−01

1 × 105 Best 8.35E−01 + 1.05E+00 + 5.00E−01 − 9.23E−01 + 9.91E−01 + 5.00E−01

Median 1.37E+00 1.25E+00 7.36E−01 1.24E+00 1.27E+00 7.23E−01

Worst 1.55E+00 1.34E+00 9.05E−01 1.37E+00 1.35E+00 1.80E+00

Mean 1.32E+00 1.23E+00 7.07E−01 1.22E+00 1.23E+00 8.39E−01

Std 1.46E−01 8.26E−02 1.35E−01 1.12E−01 8.86E−02 3.50E−01

1.5 × 105 Best 5.41E−01 + 1.03E+00 + 5.00E−01 = 8.75E−01 + 9.87E−01 + 5.00E−01

Median 1.24E+00 1.18E+00 6.26E−01 1.19E+00 1.14E+00 6.05E−01

Worst 1.39E+00 1.34E+00 8.85E−01 1.34E+00 1.33E+00 1.37E+00

Mean 1.10E+00 1.18E+00 6.43E−01 1.17E+00 1.15E+00 7.03E−01

Std 2.61E−01 7.46E−02 1.16E−01 1.11E−01 7.77E−02 2.56E−01

P03 5 × 104 Best 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02

Median 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Worst 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Mean 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Std 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00

1 × 105 Best 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02

Median 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Worst 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Mean 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Std 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00

1.5 × 105 Best 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02 = 2.20E+02

Median 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Worst 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Mean 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02 2.20E+02

Std 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00 0.00E+00

P04 5 × 104 Best −2.18E+01 + −2.18E+01 + −2.18E+01 = −2.18E+01 = −2.18E+01 + −2.16E+01

Median −2.14E+01 −2.14E+01 −2.16E+01 −2.16E+01 −2.14E+01 −2.16E+01

Worst −1.93E+01 −1.57E+01 −2.12E+01 −2.14E+01 −1.71E+01 −2.14E+01

Mean −2.14E+01 −2.12E+01 −2.16E+01 −2.16E+01 −2.12E+01 −2.16E+01

Std 4.60E−01 1.178E+00 1.72E-01 1.47E−01 8.80E−01 9.465E−02

1 × 105 Best −2.17E+01 + −2.18E+01 + −2.18E+01 − −2.18E+01 = −2.18E+01 + −2.18E+01

Median −2.14E+01 −2.15E+01 −2.16E+01 −2.16E+01 −2.16E+01 −2.16E+01

Worst −1.93E+01 −2.11E+01 −2.14E+01 −2.14E+01 −1.95E+01 −2.14E+01

Mean −2.14E+01 −2.15E+01 −2.17E+01 −2.16E+01 −2.13E+01 −2.16E+01

Std 4.67E-01 1.676E−01 1.52E−01 1.40E−01 5.02E−01 1.16E−01

1.5 × 105 Best −2.18E+01 + −2.18E+01 + −2.18E+01 − −2.18E+01 = −2.18E+01 + −2.18E+01

Median −2.15E+01 −2.15E+01 −2.16E+01 −2.16E+01 −2.14E+01 −2.16E+01

Worst −1.93E+01 −2.12E+01 −2.14E+01 −2.14E+01 −1.97E+01 −2.14E+01

Mean −2.15E+01 −2.15E+01 −2.17E+01 −2.16E+01 −2.14E+01 −2.16E+01

Std 4.69E−01 1.78E−01 1.50E−01 1.38E+01 3.76E−01 1.37E−01

+ (at maxFES) 2 3 0 2 2

= (at maxFES) 2 1 3 2 2

− (at maxFES) 0 0 1 0 0

e

p

f

c

o

m

J

p

R

e

f

t

r

s

mbedded and compared with JADE, Rcr-JADE and the proposed

bestrr-JADE, respectively. One of them, i.e. pbestrr-JADE-O3 per-

orms the best over 12 compared algorithms. Then a further dis-

ussion on the population size is conducted and the better setting

f population size is selected. Based on the above experiments, we

ake a further comparison to test the effectiveness of the pbestrr-

ADE-O3 algorithm at D = 30. Compared with other recently pro-

osed five state-of-the-art DE algorithms, namely, SaDE, JADE, CoDE,

ank-JADE, Rcr-JADE, the proposed algorithm presents competitive

xperimental results over the tested benchmark functions and other

our selected benchmark functions. The proposed algorithm ob-

ains the better or at least comparable results to other five algo-

ithms, as for final achieved solutions’ quality and the convergence

peed.

12 W. Yi et al. / Expert Systems With Applications 44 (2016) 1–12

G

G

H

I

L

M

P

P

Q

R

S

S

T

T

T

W

W

W

Y

Y

Z

Z

Z

Z

Considering the future work directions, some works indicated that

ensemble mutation operators can improve the performance of DE al-

gorithm(Pan, Suganthan, Wang, Gao, & Mallipeddi, 2011; Qin, Huang,

& Suganthan, 2009; Wang & Huang, 2010), in the future we will try to

embed more mutation operators in the proposed framework.

It is also a trend to solve large-scale continuous optimization

problems. Some researchers have made some promising works in this

field (Weber, Neri, & Tirronen, 2011; Yang et al. 2011; Zhao, Sugan-

than, & Das, 2011). Thus, another future work can apply pbestrr-JADE-

O3 algorithm solving large-scale continuous optimization problems.

Moreover, it is also an interesting work to embed pbestrr-JADE-O3 al-

gorithm to solve more complex engineering optimization problems in

real-world (Das & Abraham, 2009; García-Domingo, Carmona, Rivera-

Rivas, del Jesus, & Aguilera, 2015; Tang, Zhao, & Liu, 2014; Mokhtari &

Salmasnia, 2015; Rao & Kalyankar, 2013; Tenne, 2012).

Acknowledgments

This research work is supported by the Natural Science Foun-

dation of China (NSFC) under Grant nos. 51421062, 51435009 and

51375004, and Youth Science & Technology Chenguang Program of

Wuhan under Grant no. 2015070404010187. The authors would like

to thank Dr. Wang Yong and Dr. Gong Wenyin for making their code

available online.

References

Bäck, T. (1996). Evolutionary algorithms in theory and practice: evolution strategies, evo-lutionary programming, genetic algorithms. Oxford University Press.

Becerra, R. L., & Coello, C. A. C. (2006). Cultured differential evolution for constrained

optimization. Computer Methods in Applied Mechanics and Engineering, 195, 4303–4322.

Das, S., & Abraham, A. (2009). Differential evolution using a neighborhood-based mu-tation operator. IEEE Transactions on Evolutionary Computation, 13, 526–553.

Das, S., & Suganthan, P.N. (2010). Problem definitions and evaluation criteria for CEC2011 competition on testing evolutionary algorithms on real world optimization

problems. URL http://www.ntu.edu.sg/home/EPNSugan

Epitropakis, M. G., Tasoulis, D. K., Pavlidis, N. G., Plagianakos, V. P., & Vrahatis, M. N.(2011). Enhancing differential evolution utilizing proximity-based mutation oper-

ators. IEEE Transactions on Evolutionary Computation, 15, 99–119.Fan, H. Y., & Lampinen, J. (2003). A trigonometric mutation operation to differential

evolution. Journal of Global Optimization, 27, 105–129.García-Domingo, B., Carmona, C. J., Rivera-Rivas, A. J., del Jesus, M. J., & Aguilera, J.

(2015). A differential evolution proposal for estimating the maximum power de-

livered by CPV modules under real outdoor conditions. Expert Systems with Appli-cations, 42(13), 5452–5462.

Gong, W. Y., & Cai, Z. H. (2009). An improved multiobjective differential evolution basedon pareto-adaptive ε-dominance and orthogonal design. The European Journal of

Operational Research, 198, 576–601.Gong, W. Y., Cai, Z. H., Ling, C. X., & Li, H. (2011). Enhanced differential evolution with

adptive strategies for numerical optimization. IEEE Transactions on Systems, Man,

and Cybernetics, 41, 397–413.

ong, W. Y., & Cai, Z. H. (2013). Differential evolution with ranking-based mutationoperators. IEEE Transactions on Systems, Man, and Cybernetics, 46, 2066–2081.

ong, W. Y., Cai, Z. H., & Wang, Y. (2014). Repairing the crossover rate in adaptive dif-ferential evolution. Applied Soft Computing, 15, 149–168.

uang, F. Z., Wang, L., & He, Q. (2007). An effective co-evolutionary differential evolu-tion for constrained optimization. Applied Mathematics and Computation, 186, 340–

356.slam, S. M., Das, S., Ghosh, S., Roy, S., & Suganthan, P. N. (2012). An adaptive differential

evolution algorithm with novel mutation and crossover strategies for global nu-

merical optimization. IEEE Transactions on Evolutionary Computation, 42, 482–500.iao, T. W. (2010). Two hybrid differential evolution algorithms for engineering design

optimization. Applied Soft Computing, 10, 1188–1199.okhtari, H., & Salmasnia, A. (2015). A Monte Carlo simulation based chaotic differ-

ential evolution algorithm for scheduling a stochastic parallel processor system.Expert Systems With Applications, 42(20), 7132–7147.

an, Q. K., Suganthan, P. N., Wang, L., Gao, L., & Mallipeddi, R. (2011). A differential

evolution algorithm with self-adapting strategy and control parameters. Computers& Operations Research, 38, 394–408.

iotrowski, A. P. (2013). Adaptive memetic differential evolution with global and localneighborhood-based mutation operators. Information Science, 241, 164–194.

in, A. K., Huang, V. L., & Suganthan, P. N. (2009). Differential evolution algorithm withstrategy adaptation for global numerical optimization. IEEE Transactions on Evolu-

tionary Computation, 13, 398–417.

ao, R. V., & Kalyankar, V. D. (2013). Parameter optimization of modern machining pro-cess using teaching-learning-based optimization algorithm. Engineering Applica-

tions of Artificial Intelligence, 26, 524–531.torn, R., & Price, K. (1997). Differential evolution – A simple and efficient heuristic for

global optimization over continuous spaces. The Journal of Global Optimization, 11,341–359.

uganthan, P. N., Hansen, N., Liang, J. J., Deb, K., Chen, Y. P., Auger, A., & Tiwari, S. (2005).

Problem definitions and evaluation criteria for the CEC2005 special session on real-parameter optimization. URL http://www.ntu.edu.sg/home/EPNSugan

an, Y. Y., Jiao, Y. C., Li, H., & Wang, X. K. (2012). A modification to MOEA/D-DE formultiobjective optimization problems with complicated Pareto sets. Information

Science, 213, 14–38.ang, L. X., Zhao, Y., & Liu, J. Y. (2014). An improved differential evolution algorithm for

practical dynamic scheduling in steelmaking-continuous casting production. IEEE

Transactions on Evolutionary Computation, 18(2), 209–225.enne, Y. (2012). A computational intelligence algorithm for expensive engineering op-

timization algorithms. Engineering Applications of Artificial Intelligence, 25, 1009–1021.

ang, L., & Huang, F. Z. (2010). Parameter analysis based on stochastic model for differ-ential evolution algorithm. Applied Mathematics and Computation, 217, 3263–3273.

ang, Y., Cai, Z. X., & Zhang, Q. F. (2011). Differential evolution with composite trail vec-

tor generation strategies and control parameters. IEEE Transactions on EvolutionaryComputation, 15, 55–66.

eber, M., Neri, F., & Tirronen, V. (2011). Shuffle or updated parallel differential evolu-tion for large-scale optimization. Soft Computing, 15, 2089–2107.

ang, Z. Y., Tang, K., & Yao, X. (2011). Scalability of generalized adaptive differentialevolution for large-scale continuous optimization. Soft Computing, 15, 2141–2155.

ildiz, A. R. (2013). Hybrid taguchi-differential evolution algorithm for optimization ofmulti-pass turning operation. Applied Soft Computing., 13, 1433–1439.

hang, J. Q., & Sanderson, A. C. (2009a). JADE: Adaptive differential evolution algorithm

with optional external archive. IEEE Transactions on Evolutionary Computation, 13,945–958.

hang, J. Q., & Sanderson, A. C. (2009b). Adaptive differential evolution: a robust approachto multimodal problem optimization. Berlin: Spinger-Verlag.

hao, S. Z., Suganthan, P. N., & Das, S. (2011). Self-adaptive differential evolution withmulti-trajectory search for large-scale optimization. Soft Computing, 15, 2175–2185.

hou, Y. Z., Li, X. Y., & Gao, L. (2013). A differential evolution algorithm with intersect

mutation operator. Applied Soft Computing, 13, 390–401.