Date post: | 24-Oct-2014 |
Category: |
Documents |
Upload: | arash-norouzzadeh |
View: | 29 times |
Download: | 0 times |
Appl IntellDOI 10.1007/s10489-011-0328-6
LADPSO: using fuzzy logic to conduct PSO algorithm
Mohammad Sadegh Norouzzadeh ·Mohammad Reza Ahmadzadeh · Maziar Palhang
© Springer Science+Business Media, LLC 2011
Abstract Optimization plays a critical role in human mod-ern life. Nowadays, optimization is used in many aspects ofhuman modern life including engineering, medicine, agri-culture and economy. Due to the growing number of opti-mization problems and their growing complexity, we needto improve and develop theoretical and practical optimiza-tion methods. Stochastic population based optimization al-gorithms like genetic algorithms and particle swarm opti-mization are good candidates for solving complex prob-lems efficiently. Particle swarm optimization (PSO) is anoptimization algorithm that has received much attention inrecent years. PSO is a simple and computationally inex-pensive algorithm inspired by the social behavior of birdflocks and fish schools. However, PSO suffers from pre-mature convergence, especially in high dimensional multi-modal functions. In this paper, a new method for improv-ing PSO has been introduced. The Proposed method whichhas been named Light Adaptive Particle Swarm Optimiza-tion is a novel method that uses a fuzzy control system toconduct the standard algorithm. The suggested method usestwo adjunct operators along with the fuzzy system in or-der to improve the base algorithm on global optimizationproblems. Our approach is validated using a number of com-mon complex uni-modal/multi-modal benchmark functionsand results have been compared with the results of Stan-dard PSO (SPSO2011) and some other methods. The sim-ulation results demonstrate that results of the proposed ap-proach is promising for improving the standard PSO algo-
M.S. Norouzzadeh (�) · M.R. Ahmadzadeh · M. PalhangElectrical & Computer Engineering Department, IsfahanUniversity of Technology, Isfahan, Irane-mail: [email protected]
M.R. Ahmadzadeh (�)e-mail: [email protected]
rithm on global optimization problems and also improvingperformance of the algorithm.
Keywords Particle swarm optimization · Fuzzy control ·Random search · Numerical function optimization ·Premature convergence
1 Introduction
There exists systems in nature which are composed of sim-ple agents. Each agent in these systems looks to be simpleand unintelligent but, the behavior of the entire system sur-prisingly seems intelligent. In such systems, no central con-trol or other types of coordinators exist but the collectivebehavior of the system is purposeful and smart. Scientistsnamed this behavior as swarm intelligence. In other words,in a swarm intelligent system each agent does a simple taskand interacts locally with the environment and other agents,but the collective behavior of the entire system that resultsfrom these simple tasks is intelligent. Many swarm intel-ligent systems exist in nature, for example ants that allo-cate their tasks dynamically without any coordinator. As an-other example birds in a flock and fishes in a school organizethemselves in optimal geometrical patterns.
Computer scientists have been inspired by swarm intelli-gent systems in nature and have tried to imitate the behav-ior of such systems by inventing computational swarm in-telligent models. The aim of such computational models ismaking powerful methods for solving problems which com-posed of simple and computationally inexpensive agents in-stead of devising complex concentrated systems. Workingwith many simple and understandable agents is easier thanworking with a very complex system. Particle Swarm Op-timization (PSO) algorithm is an example of such compu-
M.S. Norouzzadeh et al.
tational models which tries to simulate behavior of the birdswarms.
PSO is a population based numerical optimization al-gorithm inspired by the social behavior of bird swarms.Kennedy and Eberhart first introduced PSO algorithm in1995 [1]. Like evolutionary algorithms, PSO maintains agroup of candidate solutions. In PSO each candidate’s so-lutions is called particle and the entire population is calledswarm. In order to simulating the behavior of natural birdswarms, each particle within PSO swarm does two simpletasks. First, it constantly desires to come back to its ownprevious best position and second, it follows its successfulneighbors by flying to their successful positions. The overallbehavior of the swarm results from these two simple tasks isswarm’s rapid focusing on promising areas of search space.
Although PSO is a speedy and robust optimization al-gorithm, it suffers from premature convergence especiallyin problems with many variables. Premature convergence isdefined as converging of the algorithm to a suboptimal so-lution rather than locating the global optimum solution. Todate, many researchers have examined various approaches tosolve this problem. We will discuss this in some importantprevious works on this paper.
To avoid converging to local optima in PSO, an idea ofconducting the base algorithm by a fuzzy control system andalso utilizing exploration ability of random search and diver-sification of a type of mutation operator is addressed in thispaper.
This paper is organized as follows. In Sect. 2, we havetalked about the basic PSO algorithm. In Sect. 3, we havereviewed previous attempts for improving PSO algorithmon global optimization problems. In Sect. 4, we have intro-duced our method and have described it in detail. In Sect. 5,we have explained benchmark functions and their settingsthen we have compared the simulation results with those ofstandard PSO, and some other methods. Finally, Sect. 6 isthe conclusion and suggestion for some future works.
2 Particle swarm optimization
Particle swarm optimization is a stochastic optimization toolthat stores a population of possible solutions to solve prob-lems. The current solution of each particle is called posi-tion of the particle. The population initialized by randomlygenerated particles. The historical PSO algorithm uses uni-form random numbers for initialization. During each iter-ation, particles within the swarm update their position insearch space based on two types of knowledge. First theirown personal experiences and second the experiences oftheir neighbors. It means that in each iteration of the algo-rithm, each particle flies to the direction of the best positionof itself and the best position of its neighbors. The neighbors
of each particle are usually determined before the start of thealgorithm, basic PSO algorithm could be classified as localbest PSO or global best PSO base on social structure of theswarm. In PSO algorithm, positions of particles are updatedby their velocity vectors. It means that the position of eachparticle in the next iteration is calculated by adding its veloc-ity vector to its current position (1). Consider that we havean optimization problem in a d-dimensional search space.Let Xi = (xi1, xi2, . . . , xid) and Vi = (vi1, vi2, . . . , vid) arethe ith particle’s position vector and velocity vector re-spectively. Also suppose that Pi = (pi1,pi2, . . . , pid) repre-sents best previously visited position of the ith particle andPgi = (pg1,pg2, . . . , pgd) represents the global best posi-tion of the swarm. In PSO algorithm, optimization is done bythe velocity equation. The velocity of each particle is com-puted according to (2).
Xt+1i = Xt
i + V ti (1)
V t+1i = wV t
i + c1r1(Pti − Xt
i ) + c2r2(Ptg − Xt
i ) (2)
where d ∈ {1,2, . . . ,D}, i ∈ {1,2, . . . ,N} and N is theswarm size and the superscript t is the iteration number.In (2) w is the inertia weight which prevents particles tosuddenly change their directions, r1 and r2 are two randomvectors with values in the range [0,1] used to keep somediversity and c1 and c2 are the cognitive and social scalingparameters which are positive constants. The proportion ofc1 and c2 determine how much each particle relies on itsexperience and how much it relies on others experiences.Usually c1 and c2 are equal and set to about 2.
Empirical simulations showed that in basic PSO, if wedo not limit the amount of velocity to some predeter-mined value, then the magnitude of velocity of particlesincrease rapidly. In such conditions particles locations in-crease rapidly too and therefore the swarms is unable to dooptimization [2]. Constriction coefficient model [3] is an-other popular model of PSO algorithm which does not needvelocity clamping. Moreover if certain conditions are met,the constriction coefficient model could guarantee conver-gence of the swarm. In constriction coefficient model thevelocity equation change to (3). Where χ called the con-striction factor.
V t+1i = χ[V t
i + ϕ1(Pti − Xt
i ) + ϕ2(Ptg − Xt
i )] (3)
where
χ = 2k
|2 − ϕ − √ϕ(ϕ − 4)| ,
ϕ = ϕ1 + ϕ2, ϕ1 = c1r1, ϕ2 = c2r2 and k ∈ [0,1]The parameter k, in (3) controls the exploration and ex-ploitation abilities of the swarm. For k ≈ 0, fast convergenceis obtained with local exploitation. The swarm exhibits analmost hill-climbing behavior. On the other hand, k ≈ 1 re-sults in slow convergence with a high degree of exploration.
LADPSO: using fuzzy logic to conduct PSO algorithm
Usually, k is set to a constant value. However it can be de-creased during the execution of algorithm.
In this paper, we have used the current standard versionof the PSO (SPSO-2011 [12]) as the base algorithm for ourmethod. SPSO is a standard version of PSO approved byresearchers of the field, as a base method for comparison ofvariations of the algorithm.
3 Related works
In optimization theory we are looking for an optimizationalgorithm that can locate the global optimum point on ev-ery problem regardless of its difficulty and algorithm’s start-ing point. In 1997 Wolpert and Macready [4] proved that“averaged over all possible problems or cost functions, theperformance of all search algorithms is exactly the same”.This means that no algorithm is better on average than otheralgorithms. Although reaching to the most excellent algo-rithm for all problems is impossible, we are not coping withall possible problems or cost functions, therefore we are stilllooking for an algorithm with high accuracy on common anduseful problems.
Any optimization algorithm must do two tasks, explo-ration and exploitation. Exploration means that searching re-gions of search space to find promising regions and exploita-tion means that concentrating on a promising area to refinethe solution. Some problems needs more exploration andless exploitation while other problems needs more exploita-tion and less exploration. Every good optimization methodmust be able to balance these two contradictive tasks. Usu-ally algorithms are trying to explore search space in earlysteps and then they try to exploit hopeful areas.
Up to now many attempts have been made to deal withPSO algorithm’s premature convergence. Some of these at-tempts were successful to improve PSO algorithm on globaloptimization problems. Previous works have tried differentapproaches for improving basic PSO algorithm. In this sec-tion we introduce studies about important previous attemptsand classify them according to the changes made on the ba-sic algorithm. In this section we have examined seven typesof approaches for dealing with premature convergence.
3.1 Changing the initialization method of the algorithm
Some methods have tried to improve the initializationmethod of the PSO in order to have a better explorationin early iterations of the algorithms. Pant et al. [5] haveused different probability distributions for initializing theswarm. In another work Pant et al. utilized low discrep-ancy sequences (discrepancy is deviation of the random se-quence from the true uniform distribution, low discrepancy
sequences are less random but more uniform than pseudo-random numbers) for better initializing particles [6]. Plow-ing PSO is another approach for initializing PSO effectively,which uses a kind of random search for improving initializa-tion [7].
3.2 Changing the neighborhood pattern of the swarm
Some other methods have changed the neighborhood struc-ture of the swarm in order to get to a compromise be-tween exploration and exploitation. Whatever neighborhoodof particles in swarm is more coupled, the flow of experi-ences between particles become faster and therefore prema-ture convergence is more probable. Various methods havebeen suggested for changing the neighborhood structure. Forexample in fully informed PSO [8] each particle not onlyimpressed by best of its neighbor but also impressed by allof its neighbors. Suganthan have used Euclidian distance forselecting neighbors [9]. Kennedy and Mendes in [10] haveexamined the impact of various social topology graphs onPSO performance. In basic PSO algorithm each particle con-verges to median of its own and its neighbor’s best position,Kennedy in barebones PSO [11] has used randomly gen-erated positions around the middle of these two positionsinstead of particle’s best position and best swarm positionin velocity equation. Standard PSO (SPSO-07, SPSO-2011)[12] is an improved version and current standard of PSOwhich uses a random topology for choosing the local bestparticles adaptively.
3.3 Adjustment of basic algorithm parameters
Some researchers have tried to achieve a compromise be-tween exploration and exploitation by adjusting historicalPSO algorithm parameters [13–16]. For example whateverthe value of inertia weight is greater, the exploration abilityof the swarm is more and vice versa. So some methods ad-just inertia weight so that exploration in early steps is highand decrease it during running of algorithm. Constriction co-efficient model [3] itself is another method that tries to adjusthistorical PSO parameters for improving it.
3.4 Hybrid methods
There exists various studies that have mixed PSO algorithmby other optimization methods [17–19] or other method’sheuristics and concepts. For example Chen et al. have mixedPSO with Extremal optimization [17]. Angeline [20] hashybridized PSO with the selection operator of genetic al-gorithm and some other methods have combined PSO withcrossover and mutation operators of the genetic algorithm[21–25]. The proposed method in this paper could be classi-fied as hybrid method because it uses the mutation conceptfrom genetic algorithms to improve the PSO algorithm.
M.S. Norouzzadeh et al.
3.5 Multi-swarm methods
Some techniques have tried the idea of utilizing more thanone swarm for doing optimization. These swarms may co-operate or compete for doing their tasks. For example co-operative split PSO [26] has a number of sub-swarms thateach one is responsible for optimization of a subset of thesolution. In predator-prey PSO [27], Silva has inspired bypredator-prey relationship in nature and introduced predatorswarm to PSO which is useful for keeping diversity withinthe swarm. The predator swarm follows global best positionof the (prey) swarm so the preys fear and do not get closeto global best position very much so the swarm keeps itsdiversity.
3.6 Multi-start methods
One of the main reasons of premature convergence is thelack of diversity within the swarm. In PSO when a parti-cle reaches to an optima, it rapidly attracts its neighborsand therefore, the swarm concentrates on the founded op-timum. Many techniques [28–30] have been suggested forkeeping diversity within swarm by injecting diversity to theswarm by reinitializing particles or their velocities. Thesetechniques are differ based on when they inject diversityto the swarm and how they do diversification. For examplesome methods randomize velocity vectors while the othersrandomize the position of the particles. We utilized randomrestart technique in our proposed algorithm for injecting di-versity to the swarm.
3.7 Other methods
Several methods exist that cannot simply be classified asabove types. For example Blackwell et al. [31] have intro-duced charged PSO in which repulsion force has been in-troduced to PSO. In charged PSO each particle has an elec-trostatic charge that has impact on the behavior of the parti-cle. Particles with the same electrostatic charge repulse eachother and so we can ensure of the existence of diversity inthe swarm.
4 The proposed method
Our proposed algorithm is based on the Standard PSO al-gorithm. It means that our suggested algorithm does all ofStandard PSO computations in every iterations, moreoverwe have utilized two operators and a simple fuzzy con-trol system in order to improve Standard PSO algorithm onglobal optimization problems. The first operator or plow op-erator is used for efficient initializing of the algorithm. Thesecond operator or mutation operator is used for escaping
Fig. 1 Pseudo code of plow operator
from local optima and finally the fuzzy control system isused for conducting the search and effectively using the mu-tation operator, with the aim of facilitating exploration andexploitation. We will explain details of these operators andcontrol system in addition to description of our algorithmwhich we named Light Adaptive PSO (LADPSO) in the fol-lowing sub-sections. Note that all pseudo codes are assum-ing minimization problems.
4.1 Plow operator
The plow operator has utilized the exploration ability ofblind random search for making initialization of the PSOmore effective [7]. Plow operator performs on one particleof the swarm. Plow operator tries to reach a better solutionby iteratively changing the variable of one dimension in acandidate solution by uniform random numbers while otherdimensions remain fixed. If the new generated position isbetter than the previous one, then it keeps the new positionand drops it otherwise. This procedure gets done for each di-mension k times. Since this process is like plowing in agri-culture and prepares the algorithm to reach to a better solu-tion, this process has been named as plow. The pseudo codeof plow operator is represented in Fig. 1.
As we mentioned earlier, plow operator changes the valueof each dimension k times. The original paper [7] uses theplow operator only for global best location of the swarmwith k = 50, but we use the plow operator for all particlesonce at the beginning of the algorithm with k = 20. How-ever plow operator may be used several times during execu-tion of the algorithm for various reasons. For example plowoperator may be used to facilitate exploration, helping thealgorithm to escape from local optima or even injecting di-versity to the swarm. Usefulness of the plow operator hasbeen reported on [7].
4.2 Mutation operator
Up to now mutation has been combined with PSO in manyprevious works. We have referenced to some of them in pre-
LADPSO: using fuzzy logic to conduct PSO algorithm
Fig. 2 Pseudo code of the adaptive mutation operator
vious section. Several methods including uniform mutation,Gaussian mutation, Levy mutation and adaptive mutationwhich choose the type of mutation based on problems condi-tions have been suggested for performing mutation in PSO.In our work, we have added a Gaussian mutation to the par-ticles. The amplitude of the Gaussian mutation is controlledby the fuzzy system. The control strategy designed so thatthe amplitude of the Gaussian noise is high in the early stepsto help the algorithm to explore more effectively and it willbe decreased to facilitate exploitation in the last phases.
Like the plow operator, mutation operator changes thevalue of one dimension k times while other variables remainconstant. This change is performed by replacing the value ofthe variable by a random number with normal distributionwith average of variable’s old value and a determined vari-ance. The value of variance and k is set automatically by thefuzzy control system. Mutation operator acts on global bestlocation of the entire swarm. Like multi restart techniques,in our algorithm after each application of mutation operator,we have reinitialized some of population randomly (exceptthe global best particle) to inject diversity to the swarm; thenumber of particles to reinitialize is determined by fuzzycontrol system too.
The pseudo code of the mutation operator has been de-picted in Fig. 2 where “randn” indicates a normal randomnumber with zero mean and unity standard deviation.
4.3 Proposed algorithm (LADPSO)
We have utilized two above operators in order to improv-ing standard PSO algorithm. As we mentioned earlier theplowing operator has been used for effective initializationand the mutation operator has been used for escaping fromlocal optima. In LADPSO algorithm plowing is used onlyonce at beginning of the algorithm, on the other hand themutation operator acts when a fuzzy control system triggersit. We have used control system with the aim of helping thealgorithm to converge to global optimum and optimal useof computational resources. Usually on similar works ad-junct operators have been used regularly for example onetime on every few iterations. This approach does not seemefficient because on some conditions the algorithm uses op-erators unduly or the algorithm could not use them when it
Fig. 3 Pseudo code of the LADPSO algorithm
needs to use them. In this paper, we have used a compu-tationally inexpensive fuzzy control system for determiningthe exact timing of operator’s usage and configuration of theoperator. This control system specifies whether we must useor not use the mutation operator based on swarm’s condi-tions. Also the fuzzy system specifies the type of mutationoperator. The details of our control system will be explainedin next section.
In LADPSO, in each iteration, if the control system de-cides that we need to perform the mutation operator, thenit will be done. When the swarm starts to converge we usethe mutation operator to help the swarm to escape from lo-cal optima. The proposed algorithm first tries to reach to agood start point by performing the plow operator on the en-tire swarm. In consequent iterations, if the swarm starts toconverge then the fuzzy control system decides to do muta-tion, during the mutation the algorithm tries to escape fromlocal optimum and to inject some diversity to the swarm. Ifafter some mutations the swarm was unable to get better re-sult, the amplitude of Gaussian mutation is decreased to helpthe algorithm to do more exploitation. In fact the aim of thefuzzy system is to help both the exploration and exploitationwhen the algorithm needs to them.
Pseudo code and the flowchart of the LADPSO are shownin Figs. 3 and 6.
4.4 Fuzzy control system
We have used a fuzzy approach to cope with premature con-vergence in this paper. Our fuzzy control system tries to de-tect premature convergence by examining swarm conditionsand use necessary reaction (mutation operator timing andsettings) to deal with it. In this section we will explain thedetails of our fuzzy control. Our control system has beendesigned based on fuzzy approach; we have defined two in-put variables for our system. The first one is swarm radius;
M.S. Norouzzadeh et al.
Fig. 4 Membership function of swarm radius
Fig. 5 Membership of stagnation
swarm radius is mean of the standard deviations of the di-mensions of particles within swarm. The swarm radius hasbeen computed based on following equations (4) and (5).The second input variable is the number of iterations whichthe swarm does not success to improve the solution or thenumber of stagnant iterations.
radius(D) = STD(D) (4)
Swarm_Radius =∑dim
D=1 STD(D)
dim(5)
To deal with normal variables for all problems we have de-fined the membership functions of input variables as showedon Figs. 4 and 5.
We have designed a table-based inference engine for ourproposed fuzzy system. The inference table which definesour system strategy is depicted on Table 1. We have plannedtwo setting for the mutation operator. These setting are de-signed so that it could help exploration in early phases andalso assist exploitation on the last phases. The details ofthese settings are shown on Table 2, where k is the k param-eter on the mutation operator, sigma is multiplied to searchspace boundaries (upper limit-lower limit) and produces thevariance of the Gaussian mutation. The reinitialize propor-tion is the proportion of the swarm that must be randomlyreinitialized. Our suggested algorithm does not have any ex-tra parameters for setting by user. The values of decisiontable and membership functions of fuzzy variables are de-termined based on our experimental simulations. We havetried to tune these parameters accurately, but by using more
Table 1 Decision table of fuzzy controller
Stagnation Swarm radius
Very low Low Mid High
Low M1 M1 NOP NOP
Mid M2 M1 M1 NOP
High M2 M2 M1 M1
Table 2 Parameters of mutation
k Sigma Reinitialize proportion
M1 10 0.05 0.05
M2 10 0.01 0.00
computational resources it is possible to do a Comprehen-sive experimental simulation to set these parameters moreaccurately. Note that in Table 1, NOP means that do not anyoperation; just leave the standard PSO to go further.
5 Results and discussion
We have organized our simulations in two sections. Onthe first section we have evaluated the algorithm on firstten functions designed for the special session on real opti-mization of CEC 2005 [32]. These benchmark functions arelargely used and result of many various algorithm are avail-able for them. Also we have compared the results with theresults of standard PSO 2011. On the second section we haveapplied our algorithm on a real life optimization problemand compared it with results of some other methods. Notethat all simulations of the proposed algorithm were run onIntel Pentium Dual core with 1.7 GHz CPU and 2 GB ofmemory.
5.1 CEC 2005 functions
For this section, we have set the number of dimensions to(D =)10 and (D =)30, the swarm size is set to 40, andmaximum function evaluation number is set to 10000 ∗ D,the other parameters of algorithm is like standard PSOalgorithm[12]. We have reported the results of standard PSOalgorithm on Tables 4 and 5. Also we have shown the resultsof our method on Tables 6 and 7. We have reported the resultbased on settings suggested by [32]. Our results are compet-itive with some other algorithms which they tested on thesebenchmark functions for example [33–35] we have quotedresults of [33] as an example for comparison on Tables 8and 9. Also we have reported Algorithm Complexity basedon [32] in Table 8, where T0 is the Computing time for thecode on Fig. 7, T1 is computing time of the function 3 for200000 evaluations and T̂2 is the average of the complete
LADPSO: using fuzzy logic to conduct PSO algorithm
Fig. 6 Flowchart of LADPSOalgorithm
Fig. 7 The code for evaluating T0
computing time for the algorithm with 200000 evaluationsfor 5 times.
From the results above, it could be said that, our methodis promising for improving the standard algorithm, and itwas able to locate the global minimum on most of problems,with acceptable accuracy. Also from the results of Table 3 itis clear that our algorithm is more computationally efficientthan its standard version
5.2 Real life problem
In this section we have reported the results of applyingour algorithm to the Lennard-Jones potential problem. TheLennard-Jones potential (L-J potential or 6-12 potential) is amathematically simple model that describes the interactionbetween a pair of neutral atoms or molecules. Minimizingthe Lennard-Jones potential is an unconstrained global opti-
mization problem with many local optimum points. Becauseof its many local optimums the results of algorithms on min-imizing this function is very discriminant for comparison ofalgorithms. In minimizing this function we are seeking to ageometrical arrangement of n atoms with the lowest poten-tial energy. We have reported the average results of our al-gorithm for 5 to 15 atoms on Table 10 along with the resultsof some other methods like [36] and its global minimum forcomparison. The swarm size is 40 and the function evalua-tion count is set to 5000 + 3000 ∗ n ∗ (n − 1). We have re-ported the average of results for running the test for 20 times.
Also in Table 11 we have compared the results of ouralgorithm to one of the most comprehensive studies aboutLennard-Jones function optimization [37]. In Table 11 thebest value achieved over 10 runs, for various number ofatoms has been reported. As we could see, our method has agood performance on Lennard-Jones potential problem.
5.3 Analyzing the results
We have used Mann–Whitney–Wilcoxon (MWW) method[38, 39] to compare the results of SPSO and LADPSO al-gorithms. MWW is a non-parametric statistical analysis forexamining whether one of two samples of independent ob-servations have a tendency to have smaller values than the
M.S. Norouzzadeh et al.
Tabl
e4
Err
orva
lues
ofSP
SO20
11fo
rpr
oble
ms
1-10
(10D
)
FEs/
func
tion
12
34
56
78
910
103
min
1.15
39E
+02
1.15
09E
+03
3.11
60E
+06
5.32
83E
+02
1.87
04E
+03
3.93
46E
+05
1.39
00E
+01
2.04
23E
+01
4.21
29E
+01
3.25
57E
+01
73.
4795
E+
021.
8323
E+
036.
7645
E+
061.
9885
E+
033.
4304
E+
037.
7151
E+
054.
7139
E+
012.
0689
E+
014.
9606
E+
015.
3311
E+
01
134.
1273
E+
022.
3802
E+
031.
0634
E+
073.
0851
E+
033.
7821
E+
031.
4911
E+
066.
1102
E+
012.
0792
E+
015.
4903
E+
016.
1824
E+
01
194.
5542
E+
022.
8692
E+
031.
6214
E+
074.
3814
E+
034.
3999
E+
032.
7670
E+
066.
9930
E+
012.
0842
E+
015.
6695
E+
016.
7036
E+
01
max
6.21
31E
+02
3.23
08E
+03
2.46
06E
+07
6.99
24E
+03
5.32
74E
+03
6.42
35E
+06
1.30
45E
+02
2.09
30E
+01
6.73
48E
+01
7.21
16E
+01
mea
n3.
9501
E+
022.
3480
E+
031.
1482
E+
073.
2882
E+
033.
8881
E+
032.
0175
E+
066.
1812
E+
012.
0748
E+
015.
3631
E+
015.
9921
E+
01
std
1.22
93E
+02
6.18
74E
+02
6.53
27E
+06
1.72
12E
+03
8.17
26E
+02
1.62
13E
+06
2.62
67E
+01
1.36
32E
-01
5.55
98E
+00
9.88
30E
+00
104
min
4.83
39E
-07
1.63
06E
-05
4.18
56E
+04
1.26
36E
-03
4.29
29E
-03
4.40
28E
+00
5.60
55E
-02
2.01
50E
+01
5.53
83E
+00
5.78
30E
+00
77.
3028
E-0
74.
7178
E-0
41.
3289
E+
058.
9430
E-0
31.
4527
E-0
28.
5542
E+
001.
6503
E-0
12.
0351
E+
019.
8777
E+
001.
1343
E+
01
138.
1489
E-0
79.
7080
E-0
41.
8476
E+
051.
9373
E-0
21.
9517
E-0
21.
0229
E+
022.
4216
E-0
12.
0375
E+
011.
3254
E+
011.
4659
E+
01
198.
6275
E-0
72.
2922
E-0
33.
7931
E+
053.
7432
E-0
24.
0156
E-0
28.
2742
E+
022.
9198
E-0
12.
0523
E+
011.
8500
E+
011.
7670
E+
01
max
9.83
29E
-07
1.34
13E
-02
9.77
53E
+05
1.15
59E
+00
8.50
52E
-02
6.73
80E
+03
4.69
71E
-01
2.06
11E
+01
2.53
48E
+01
2.38
27E
+01
mea
n7.
8889
E-0
72.
2670
E-0
32.
7539
E+
057.
7618
E-0
22.
8589
E-0
21.
1759
E+
032.
3943
E-0
12.
0390
E+
011.
4226
E+
011.
4858
E+
01
std
1.25
51E
-07
3.30
99E
-03
2.05
03E
+05
2.28
65E
-01
2.11
55E
-02
2.05
25E
+03
1.07
00E
-01
1.34
79E
-01
5.25
04E
+00
5.07
32E
+00
105
min
5.39
71E
-07
4.48
58E
-07
4.33
65E
+03
6.41
67E
-07
6.96
08E
-07
1.68
95E
-02
9.62
10E
-03
2.01
12E
+01
1.98
99E
+00
2.06
68E
+00
77.
3741
E-0
77.
8312
E-0
72.
1273
E+
048.
3495
E-0
79.
1386
E-0
73.
0600
E-0
22.
9562
E-0
22.
0201
E+
013.
9798
E+
003.
7657
E+
00
137.
9384
E-0
78.
7540
E-0
73.
5677
E+
049.
3487
E-0
79.
4535
E-0
74.
3523
E-0
24.
6796
E-0
22.
0236
E+
014.
7817
E+
005.
2163
E+
00
199.
0362
E-0
79.
7090
E-0
75.
2422
E+
049.
7272
E-0
79.
6729
E-0
71.
0220
E+
025.
6552
E-0
22.
0302
E+
015.
9698
E+
006.
2968
E+
00
max
9.78
47E
-07
9.98
12E
-07
9.02
86E
+04
9.99
80E
-07
9.99
42E
-07
2.78
00E
+02
5.22
46E
-01
2.03
68E
+01
1.59
40E
+01
9.83
10E
+00
mea
n8.
0304
E-0
78.
5995
E-0
73.
7918
E+
048.
9410
E-0
79.
2681
E-0
75.
0578
E+
016.
7519
E-0
22.
0241
E+
015.
6251
E+
005.
1169
E+
00
std
1.28
02E
-07
1.37
76E
-07
2.24
58E
+04
1.03
23E
-07
6.89
07E
-08
8.68
87E
+01
9.82
47E
-02
7.11
37E
-02
3.16
87E
+00
2.06
06E
+00
LADPSO: using fuzzy logic to conduct PSO algorithm
Tabl
e5
Err
orva
lues
ofSP
SO20
11fo
rpr
oble
ms
1-10
(30D
)
FEs/
func
tion
12
34
56
78
910
3×
103
min
3.12
32E
+02
1.20
07E
+04
2.47
22E
+07
1.81
95E
+04
6.14
91E
+03
1.94
61E
+06
1.45
21E
+02
2.09
98E
+01
1.72
00E
+02
1.80
29E
+02
77.
0195
E+
021.
7960
E+
043.
6941
E+
072.
2750
E+
047.
7983
E+
036.
2680
E+
062.
4781
E+
022.
1101
E+
012.
1750
E+
022.
2878
E+
02
138.
5861
E+
021.
8943
E+
044.
2360
E+
072.
5047
E+
048.
6130
E+
031.
2253
E+
073.
1822
E+
022.
1156
E+
012.
2487
E+
022.
4412
E+
02
191.
1197
E+
032.
0358
E+
046.
2464
E+
072.
8751
E+
049.
5131
E+
032.
7609
E+
073.
9816
E+
022.
1189
E+
012.
3701
E+
022.
6437
E+
02
max
1.72
06E
+03
2.77
07E
+04
8.77
08E
+07
3.86
13E
+04
1.16
67E
+04
5.96
57E
+07
8.29
58E
+02
2.12
37E
+01
2.59
97E
+02
2.91
69E
+02
mea
n9.
1242
E+
021.
9402
E+
044.
7832
E+
072.
5673
E+
048.
6904
E+
031.
8587
E+
073.
2889
E+
022.
1144
E+
012.
2512
E+
022.
4327
E+
02
std
3.18
47E
+02
4.10
23E
+03
1.66
73E
+07
4.90
48E
+03
1.52
65E
+03
1.63
49E
+07
1.42
70E
+02
6.25
36E
-02
1.78
61E
+01
2.85
78E
+01
3×
104
min
8.01
26E
-07
1.76
29E
+01
1.34
34E
+06
4.12
80E
+03
2.96
19E
+03
2.48
56E
+01
4.62
81E
-02
2.08
07E
+01
3.58
18E
+01
3.58
18E
+01
78.
7817
E-0
73.
2873
E+
012.
0793
E+
065.
3854
E+
034.
3196
E+
032.
0192
E+
021.
3710
E-0
12.
0920
E+
014.
6825
E+
015.
9632
E+
01
139.
5457
E-0
76.
4617
E+
012.
5578
E+
066.
3613
E+
034.
9577
E+
034.
3300
E+
022.
1258
E-0
12.
0958
E+
016.
1966
E+
019.
1185
E+
01
199.
7326
E-0
78.
7396
E+
012.
9987
E+
068.
2697
E+
035.
3113
E+
031.
1023
E+
033.
0652
E-0
12.
1007
E+
019.
5180
E+
011.
2468
E+
02
max
9.99
03E
-07
1.60
48E
+02
4.45
25E
+06
1.17
01E
+04
6.17
53E
+03
1.03
10E
+04
9.08
20E
-01
2.10
76E
+01
1.56
28E
+02
1.82
10E
+02
mea
n9.
2528
E-0
76.
6370
E+
012.
5926
E+
066.
8501
E+
034.
9010
E+
031.
7795
E+
032.
5857
E-0
12.
0957
E+
017.
3631
E+
019.
5081
E+
01
std
6.13
35E
-08
3.80
12E
+01
7.52
00E
+05
2.06
14E
+03
8.16
84E
+02
2.87
54E
+03
2.01
78E
-01
7.39
35E
-02
3.39
02E
+01
4.18
87E
+01
3×
105
min
7.89
82E
-07
9.32
96E
-07
7.92
07E
+04
7.39
03E
+00
3.38
13E
+03
1.71
40E
+01
9.94
53E
-03
2.07
22E
+01
2.48
74E
+01
2.78
59E
+01
79.
0581
E-0
79.
8250
E-0
71.
8243
E+
052.
3050
E+
014.
2821
E+
032.
0745
E+
019.
9990
E-0
32.
0820
E+
014.
3778
E+
014.
4773
E+
01
139.
4793
E-0
79.
8978
E-0
72.
2671
E+
053.
6253
E+
014.
6013
E+
031.
4877
E+
022.
4573
E-0
22.
0850
E+
015.
6635
E+
015.
1738
E+
01
199.
8749
E-0
79.
9408
E-0
73.
0744
E+
056.
9441
E+
015.
1330
E+
037.
2051
E+
023.
6894
E-0
22.
0891
E+
016.
5667
E+
015.
6713
E+
01
max
9.99
24E
-07
9.98
47E
-07
5.07
08E
+05
2.31
62E
+02
6.31
36E
+03
4.40
44E
+03
8.10
87E
-02
2.10
05E
+01
1.80
47E
+02
7.66
12E
+01
mea
n9.
3626
E-0
79.
8438
E-0
72.
4853
E+
055.
4290
E+
014.
6923
E+
034.
6342
E+
022.
5243
E-0
22.
0856
E+
016.
4551
E+
015.
3327
E+
01
std
5.46
211E
-08
1.59
4E-0
810
9334
.06
48.4
1229
771
2.72
878
892.
1820
10.
0179
063
0.06
7497
534
.209
586
11.9
0755
5
M.S. Norouzzadeh et al.
Tabl
e6
Err
orva
lues
ofL
AD
PSO
for
prob
lem
s1-
10(1
0D)
FEs/
func
tion
12
34
56
78
910
103
min
3.14
74E
+00
5.97
16E
+03
6.80
73E
+06
5.69
97E
+03
2.53
19E
+03
3.90
80E
+04
7.45
02E
+02
2.02
53E
+01
4.71
94E
+00
4.74
08E
+01
78.
3217
E+
001.
0305
E+
041.
0182
E+
071.
2180
E+
044.
6451
E+
032.
7180
E+
051.
1178
E+
032.
0452
E+
017.
1692
E+
006.
3064
E+
01
131.
6061
E+
011.
3250
E+
041.
3308
E+
071.
6530
E+
045.
6153
E+
033.
8131
E+
051.
3390
E+
032.
0494
E+
018.
7066
E+
006.
8110
E+
01
194.
2378
E+
011.
5201
E+
041.
7371
E+
072.
0073
E+
046.
5441
E+
034.
6673
E+
051.
3571
E+
032.
0551
E+
019.
3167
E+
007.
1814
E+
01
max
8.38
00E
+01
1.79
89E
+04
2.99
86E
+07
2.15
86E
+04
7.42
98E
+03
6.58
48E
+05
1.36
99E
+03
2.06
19E
+01
1.03
83E
+01
7.45
07E
+01
mea
n2.
8667
E+
011.
2582
E+
041.
4375
E+
071.
5536
E+
045.
5389
E+
033.
5928
E+
051.
2207
E+
032.
0493
E+
018.
4098
E+
006.
5700
E+
01
std
2.78
69E
+01
3.49
82E
+03
5.97
45E
+06
4.91
48E
+03
1.30
67E
+03
1.70
65E
+05
1.98
38E
+02
9.71
79E
-02
1.41
91E
+00
7.85
44E
+00
104
min
3.86
99E
-01
7.40
20E
+01
7.14
49E
+05
1.71
05E
+02
6.70
14E
+01
1.97
21E
+02
1.06
79E
+00
2.02
60E
+01
3.79
01E
+00
1.75
71E
+01
71.
1261
E+
001.
3100
E+
021.
6784
E+
063.
4252
E+
021.
3931
E+
021.
1367
E+
031.
2441
E+
002.
0482
E+
015.
4865
E+
003.
2576
E+
01
131.
8927
E+
002.
0350
E+
022.
5234
E+
064.
9128
E+
022.
7293
E+
021.
7516
E+
031.
3780
E+
002.
0548
E+
016.
2237
E+
003.
9626
E+
01
194.
0834
E+
003.
1862
E+
023.
9012
E+
066.
1154
E+
023.
8794
E+
023.
0385
E+
031.
5380
E+
002.
0571
E+
016.
7404
E+
004.
5093
E+
01
max
5.14
83E
+00
5.13
25E
+02
5.00
82E
+06
8.22
93E
+02
6.01
62E
+02
6.55
63E
+03
2.53
11E
+00
2.06
23E
+01
8.44
83E
+00
5.05
36E
+01
mea
n2.
4352
E+
002.
3931
E+
022.
7851
E+
064.
6938
E+
022.
9016
E+
022.
2883
E+
031.
4854
E+
002.
0512
E+
016.
1684
E+
003.
7599
E+
01
std
1.59
64E
+00
1.36
29E
+02
1.22
17E
+06
1.96
34E
+02
1.59
89E
+02
1.83
26E
+03
3.76
02E
-01
9.27
11E
-02
1.16
28E
+00
8.97
23E
+00
105
min
2.56
05E
-07
4.04
98E
-07
5.58
84E
+03
7.86
60E
-07
7.20
06E
-07
4.31
30E
-01
9.99
67E
-03
2.00
48E
+01
3.91
19E
-03
1.98
99E
+00
77.
7901
E-0
79.
1245
E-0
72.
6539
E+
049.
4924
E-0
78.
1613
E-0
71.
6660
E+
002.
4590
E-0
22.
0304
E+
017.
5708
E-0
12.
9854
E+
00
138.
6851
E-0
79.
5076
E-0
73.
8194
E+
049.
7118
E-0
79.
3372
E-0
73.
5934
E+
004.
4283
E-0
22.
0334
E+
012.
4443
E+
003.
9798
E+
00
199.
2751
E-0
79.
8094
E-0
77.
1675
E+
049.
8974
E-0
79.
7750
E-0
72.
3480
E+
016.
8866
E-0
22.
0365
E+
013.
3726
E+
005.
9697
E+
00
max
9.51
70E
-07
9.86
20E
-07
8.48
03E
+04
9.99
55E
-07
9.96
02E
-07
5.60
94E
+01
7.87
75E
-02
2.03
81E
+01
3.88
97E
+00
6.96
51E
+00
mea
n8.
1585
E-0
79.
0734
E-0
74.
7083
E+
049.
4849
E-0
78.
9888
E-0
71.
5314
E+
014.
4214
E-0
22.
0316
E+
012.
0849
E+
004.
2733
E+
00
std
1.65
48E
-07
1.34
57E
-07
2.59
74E
+04
6.10
95E
-08
8.99
36E
-08
1.79
95E
+01
2.25
14E
-02
7.19
50E
-02
1.42
13E
+00
1.58
01E
+00
LADPSO: using fuzzy logic to conduct PSO algorithm
Tabl
e7
Err
orva
lues
ofL
AD
PSO
for
prob
lem
s1-
10(3
0D)
FEs/
func
tion
12
34
56
78
910
3×
103
min
2.07
79E
+01
5.66
83E
+03
8.79
51E
+06
7.49
08E
+03
1.59
29E
+04
3.49
28E
+06
4.67
81E
+03
2.07
73E
+01
3.46
84E
+01
3.23
97E
+02
73.
5562
E+
019.
2440
E+
031.
6708
E+
071.
0049
E+
041.
9252
E+
044.
3747
E+
065.
1484
E+
032.
0972
E+
013.
8951
E+
013.
5364
E+
02
134.
8565
E+
011.
1642
E+
042.
0291
E+
071.
1824
E+
042.
1243
E+
046.
1217
E+
065.
1837
E+
032.
1014
E+
014.
0360
E+
013.
7186
E+
02
195.
3055
E+
011.
2758
E+
042.
2463
E+
071.
3230
E+
042.
2523
E+
047.
2519
E+
065.
2024
E+
032.
1047
E+
014.
2480
E+
013.
9072
E+
02
max
1.01
23E
+02
1.69
71E
+04
2.79
74E
+07
1.56
12E
+04
2.42
08E
+04
8.34
99E
+06
5.23
95E
+03
2.10
73E
+01
4.38
98E
+01
4.07
74E
+02
mea
n4.
7321
E+
011.
1217
E+
041.
9784
E+
071.
1639
E+
042.
0927
E+
045.
9427
E+
065.
1536
E+
032.
1002
E+
014.
0496
E+
013.
7342
E+
02
std
1.72
08E
+01
2.79
11E
+03
4.74
31E
+06
2.13
48E
+03
2.35
93E
+03
1.57
31E
+06
1.13
60E
+02
6.38
00E
-02
2.21
45E
+00
2.28
32E
+01
3×
104
min
4.36
90E
-01
3.44
90E
+01
7.02
54E
+05
9.46
90E
+02
4.57
95E
+03
2.91
43E
+02
1.30
42E
+00
2.08
47E
+01
9.00
18E
+00
3.67
60E
+01
77.
3626
E-0
15.
4928
E+
011.
3057
E+
061.
3878
E+
035.
3579
E+
034.
8023
E+
021.
9922
E+
002.
0985
E+
011.
9135
E+
015.
6085
E+
01
139.
9280
E-0
17.
0801
E+
011.
7703
E+
061.
7559
E+
035.
5637
E+
037.
1231
E+
023.
0081
E+
002.
1016
E+
012.
3402
E+
018.
1358
E+
01
191.
1016
E+
007.
6666
E+
012.
2772
E+
061.
9259
E+
036.
5134
E+
031.
4329
E+
033.
4617
E+
002.
1044
E+
012.
7773
E+
018.
9431
E+
01
max
1.94
13E
+00
9.64
10E
+01
2.90
27E
+06
2.28
41E
+03
7.16
84E
+03
6.52
88E
+03
4.83
18E
+00
2.10
88E
+01
3.68
74E
+01
9.71
08E
+01
mea
n1.
0498
E+
006.
6754
E+
011.
7700
E+
061.
6702
E+
035.
9192
E+
031.
6087
E+
032.
7810
E+
002.
1011
E+
012.
3740
E+
017.
2203
E+
01
std
4.11
62E
-01
1.65
25E
+01
6.20
00E
+05
4.00
73E
+02
7.69
00E
+02
1.91
46E
+03
9.31
92E
-01
5.50
64E
-02
6.56
01E
+00
1.89
60E
+01
3×
105
min
7.18
95E
-07
8.26
68E
-07
4.02
45E
+04
3.53
02E
+00
2.53
40E
+03
4.17
11E
+00
9.93
42E
-03
2.07
50E
+01
8.86
36E
-03
2.48
74E
+01
79.
3850
E-0
71.
5025
E-0
66.
2722
E+
045.
4150
E+
003.
8358
E+
038.
0561
E+
019.
9972
E-0
32.
0870
E+
011.
2714
E+
013.
6813
E+
01
139.
7278
E-0
71.
9957
E-0
67.
1059
E+
047.
4276
E+
004.
1526
E+
039.
4653
E+
019.
9998
E-0
32.
0897
E+
011.
5968
E+
014.
2783
E+
01
199.
8254
E-0
72.
6610
E-0
68.
3794
E+
049.
1478
E+
004.
4287
E+
031.
3062
E+
022.
4573
E-0
22.
0939
E+
011.
8953
E+
015.
1738
E+
01
max
9.91
93E
-07
4.14
96E
-06
1.08
88E
+05
1.23
13E
+01
5.08
47E
+03
1.63
40E
+02
2.95
52E
-02
2.09
66E
+01
2.36
95E
+01
6.56
68E
+01
mea
n9.
5249
E-0
72.
1342
E-0
67.
3452
E+
047.
4768
E+
004.
0612
E+
039.
4867
E+
011.
6313
E-0
22.
0896
E+
011.
4859
E+
014.
5014
E+
01
std
5.60
76E
-08
9.30
99E
-07
1.71
99E
+04
2.47
64E
+00
6.12
98E
+02
4.88
63E
+01
7.76
82E
-03
5.35
80E
-02
6.23
13E
+00
1.12
63E
+01
M.S. Norouzzadeh et al.
Table 8 Error values of [32] for problems 1-10 (10D)
FES Prob. 1 2 3 4 5 6 7 8 9 10
103 min 1.88E-3 6.22E+00 1.07E+06 8.86E+01 6.71E+00 1.12E+01 6.51E-1 2.05E+01 5.02E+00 6.11E+00
7th 7.56E-3 2.40E+01 5.73E+06 5.81E+02 1.25E+01 1.27E+02 7.28E-1 2.07E+01 1.43E+01 3.63E+01
med. 1.34E-2 4.35E+01 1.14E+07 1.58E+03 2.16E+01 1.91E+03 9.06E-1 2.08E+01 2.75E+01 4.56E+01
19th 2.46E-2 8.36E+01 2.39E+07 4.28E+03 3.76E+01 3.15E+03 9.74E-1 2.08E+01 4.70E+01 5.14E+01
max 4.68E-2 1.70E+02 5.67E+07 1.75E+04 6.71E+01 9.89E+05 1.11E+00 2.09E+01 5.91E+01 6.37E+01
mean 1.70E-2 5.83E+01 1.68E+07 3.00E+03 2.81E+01 4.30E+04 8.69E-1 2.08E+01 3.07E+01 4.17E+01
std 1.20E-2 4.74E+01 1.59E+07 3.83E+03 1.85E+01 1.97E+05 1.38E-1 9.93E-2 1.72E+01 1.57E+01
104 min 1.84E-9 2.21E-9 2.21E-9 1.71E-9 2.46E-9 3.29E-9 9.31E-10 2.03E+01 1.99E+00 2.98E+00
7th 3.75E-9 3.27E-9 4.61E-9 3.85E-9 5.02E-9 4.49E-9 2.81E-9 2.05E+01 4.97E+00 4.97E+00
med. 5.65E-9 4.53E-9 5.51E-9 4.78E-9 6.33E-9 7.37E-9 5.46E-9 2.06E+01 5.97E+00 6.96E+00
19th 6.42E-9 5.71E-9 6.58E-9 6.46E-9 8.60E-9 3.99E+00 7.77E-9 2.06E+01 7.96E+00 7.96E+00
max 9.34E-9 7.67E-9 9.66E-9 7.80E-9 9.84E-9 2.63E+02 1.48E-2 2.07E+01 1.09E+01 1.59E+01
mean 5.20E-9 4.70E-9 5.60E-9 5.02E-9 6.58E-9 1.17E+01 2.27E-3 2.05E+01 6.21E+00 7.16E+00
std 1.94E-9 1.56E-9 1.93E-9 1.71E-9 2.17E-9 5.24E+01 4.32E-3 8.62E-2 2.10E+00 3.12E+00
105 min 1.84E-9 2.21E-9 2.21E-9 1.71E-9 2.46E-9 1.44E-9 6.22E-10 2.00E+01 1.52E-10 1.50E-10
7th 3.75E-9 3.27E-9 4.61E-9 3.85E-9 5.02E-9 3.81E-9 1.65E-9 2.00E+01 3.46E-10 3.34E-10
med. 5.65E-9 4.53E-9 5.51E-9 4.78E-9 6.33E-9 4.69E-9 2.84E-9 2.00E+01 6.14E-10 5.64E-10
19th 6.42E-9 5.71E-9 6.58E-9 6.46E-9 8.60E-9 5.67E-9 5.46E-9 2.00E+01 3.50E-9 1.08E-9
max 9.34E-9 7.67E-9 9.66E-9 7.80E-9 9.84E-9 8.13E-9 7.77E-9 2.00E+01 9.95E-1 9.95E-1
mean 5.20E-9 4.70E-9 5.60E-9 5.02E-9 6.58E-9 4.87E-9 3.31E-9 2.00E+01 2.39E-1 7.96E-2
std 1.94E-9 1.56E-9 1.93E-9 1.71E-9 2.17E-9 1.66E-9 2.02E-9 3.89E-3 4.34E-1 2.75E-1
Table 9 Error values of [32] for problems 1-10 (30D)
FES Prob. 1 2 3 4 5 6 7 8 9 10
3 × 103 min 4.49 E+2 1.12 E+5 3.84 E+8 6.13 E+5 6.21 E+3 3.26 E+6 4.10 E+1 2.12 E+1 2.19 E+2 2.43 E+2
7th 5.48 E+2 1.73 E+5 8.00 E+8 1.12 E+6 9.57 E+3 7.31 E+6 9.39 E+1 2.12 E+1 2.45 E+2 2.65 E+2
med. 7.40 E+2 2.35 E+5 1.00 E+9 1.44 E+6 1.09 E+4 1.23 E+7 1.20 E+2 2.12 E+1 2.50 E+2 2.74 E+2
19th 1.04 E+3 2.94 E+5 1.38 E+9 1.87 E+6 1.24 E+4 1.99 E+7 1.59 E+2 2.13 E+1 2.66 E+2 2.88 E+2
max 1.61 E+3 3.83 E+5 2.07 E+9 3.29 E+6 1.42 E+4 6.81 E+7 3.26 E+2 2.13 E+1 2.87 E+2 3.08 E+2
mean 8.16 E+2 2.39 E+5 1.07 E+9 1.55 E+6 1.07 E+4 1.77 E+7 1.39 E+2 2.12 E+1 2.53 E+2 2.77 E+2
std 3.01 E+2 7.80 E+4 4.43 E+8 6.15 E+5 2.13 E+3 1.62 E+7 7.17 E+1 4.35 E-2 1.65 E+1 1.90 E+1
3 × 104 min 3.98 E-9 2.29 E-3 1.24 E+6 4.88 E+2 5.00 E-2 1.77 E+1 3.93 E-9 2.10 E+1 2.39 E+1 3.08 E+1
7th 4.70 E-9 1.60 E-2 3.41 E+6 1.46 E+3 1.00 E+3 2.28 E+1 4.85 E-9 2.11 E+1 4.28 E+1 4.38 E+1
med. 5.20 E-9 2.57 E-2 4.90 E+6 3.51 E+3 1.32 E+3 2.58 E+1 5.69 E-9 2.11 E+1 4.88 E+1 5.27 E+1
19th 6.10 E-9 3.99 E-2 8.21 E+6 5.18 E+4 2.04 E+3 2.22 E+2 6.95 E-9 2.11 E+1 5.47 E+1 5.87 E+1
max 7.51 E-9 7.49 E-2 1.42 E+7 2.88 E+5 3.20 E+3 2.66 E+3 2.46 E-2 2.12 E+1 7.96 E+1 8.26 E+1
mean 5.42 E-9 2.73 E-2 6.11 E+6 4.26 E+4 1.51 E+3 4.60 E+2 1.77 E-3 2.11 E+1 4.78 E+1 5.14 E+1
std 9.80 E−10 1.79 E-2 3.79 E+6 7.43 E+4 8.82 E+2 8.29 E+2 5.52 E-3 4.04 E-2 1.15 E+1 1.25 E+1
3 × 105 min 3.98 E-9 4.48 E-9 4.07 E-9 6.06 E-9 7.15 E-9 4.05 E-9 1.76 E-9 2.00 E+1 2.98 E+0 9.95 E-1
7th 4.70 E-9 5.59 E-9 4.78 E-9 8.75 E-9 8.06 E-9 5.31 E-9 4.59 E-9 2.02 E+1 4.97 E+0 5.97 E+0
med. 5.20 E-9 6.13 E-9 5.44 E-9 1.93 E+1 8.61 E-9 6.32 E-9 5.41 E-9 2.09 E+1 6.96 E+0 6.96 E+0
19th 6.10 E-9 6.85 E-9 6.16 E-9 2.72 E+3 9.34 E-9 7.52 E-9 6.17 E-9 2.10 E+1 8.95 E+0 8.95 E+0
max 7.51 E-9 8.41 E-9 8.66 E-9 1.57 E+5 2.51 E-6 3.99 E+0 7.81 E-9 2.11 E+1 1.19 E+1 1.09 E+1
mean 5.42 E-9 6.22 E-9 5.55 E-9 1.27 E+4 1.08 E-7 4.78 E-1 5.31 E-9 2.07 E+1 6.89 E+0 6.96 E+0
std 9.80 E-10 8.95 E−10 1.09 E-9 3.59 E+4 4.99 E-7 1.32 E+0 1.41 E-9 4.28 E-1 2.22 E+0 2.45 E+0
LADPSO: using fuzzy logic to conduct PSO algorithm
Table 3 Computational complexity of LADPSO
Dim T0 T1 T̂2 (T̂2 − T1)/T0
Method – – SPSO LADPSO SPSO LADPSO
10 1.559 40.200 79.056 65.886 24.912 16.468
30 1.559 44.841 146.546 82.844 65.209 24.366
50 1.559 51.661 222.774 92.938 109.711 26.465
Table 10 Results of LADPSO for the Lennard-Jones potential prob-lem
N/Method LADPSO SPSO Results of [34] Minimum
5 −9.09772 −9.01407 −9.10385 −9.10385
6 −12.5239 −11.811 −12.7121 −12.7121
7 −16.2533 −14.4687 −16.5054 −16.5054
8 −19.4855 −17.1723 −19.2084 −19.8215
9 −23.2127 −19.5568 −22.9690 −24.1134
10 −27.3535 −22.1338 −26.8424 −28.4225
11 −31.6974 −23.8806 −31.5629 −32.7656
12 −36.4019 −24.399 −36.3692 −37.9676
13 −40.9243 −27.2597 −41.1379 −44.3268
14 −45.3636 −29.5903 −44.3883 −47.8452
15 −49.7374 −32.774 −49.7777 −52.3226
other. The test gives a p-value which indicates whether wecould reject the null hypothesis (two samples have equal me-dians) or not. We have performed MWW for all benchmarkproblems separately on significance level of 0.05. The re-sults of our statistical analysis are shown on Table 12. Wehave reported p-value of the test along with rejection or notrejection of null hypothesis. In Table 12, h = 0 indicates fail-ure of the rejection and h = 1 indicates rejection of hypoth-esis on significance level of 0.05.
Based on Table 12, we could see that on most cases thedifference between results is significant and it gets moretangible when the complexity of problem increased. Thisobservation suggests that the LADPSO seems promising toimprove the base algorithm especially on high dimensionalproblems.
6 Conclusion and future works
In this paper, we proposed a novel extension to PSO opti-mization method, called LADPSO algorithm, through com-bining two operators to the standard PSO. The suggestedapproach utilizes the fuzzy logic to conduct the standardPSO on global optimization problems. We also comparedaccuracy and robustness of our approach with standard PSO,and some other methods by using some well-known bench-mark functions. It has been depicted that our algorithm Ta
ble
11R
esul
tsof
LA
DPS
Ofo
rth
eL
enna
rd-J
ones
pote
ntia
lpro
blem
NA
ppII
App
III.
Stee
pB
FGS
DFP
LG
OB
BL
GO
Rd
Ref
.ESP
SO20
11L
AD
PSO
6−1
.230
3E+
01−1
.271
2E+
01−1
.230
3E+
01−1
.230
3E+
01−1
.230
3E+
01−1
.271
2E+
01−1
.230
3E+
01−1
.271
2E+
01−1
.271
1E+
01−1
.271
2E+
01
7−1
.553
3E+
01−1
.650
5E+
01−1
.549
9E+
01−1
.550
0E+
01−1
.550
0E+
01−1
.553
3E+
01−1
.553
3E+
01−1
.650
5E+
01−1
.650
5E+
01−1
.650
5E+
01
8−1
.982
2E+
01−1
.982
2E+
01−1
.882
9E+
01−1
.882
9E+
01−1
.882
9E+
01−1
.877
8E+
01−1
.982
2E+
01−1
.982
2E+
01−1
.982
1E+
01−1
.982
1E+
01
9−2
.411
3E+
01−2
.411
3E+
01−2
.218
1E+
01−2
.218
1E+
01−2
.218
1E+
01−2
.304
4E+
01−2
.215
6E+
01−2
.411
3E+
01−2
.216
2E+
01−2
.411
3E+
01
10−2
.754
5E+
01−2
.842
3E+
01−2
.694
3E+
01−2
.695
5E+
01−2
.693
9E+
01−2
.727
4E+
01−2
.662
3E+
01−2
.842
3E+
01−2
.509
5E+
01−2
.755
6E+
01
11−3
.276
6E+
01−3
.276
6E+
01−3
.276
6E+
01−3
.276
6E+
01−3
.276
6E+
01−2
.999
7E+
01−3
.095
4E+
01−3
.276
6E+
01−2
.885
6E+
01−3
.191
4E+
01
12−3
.617
8E+
01−3
.796
8E+
01−3
.796
8E+
01−3
.796
8E+
01−3
.796
8E+
01−3
.517
1E+
01−3
.546
2E+
01−3
.796
8E+
01−2
.942
3E+
01−3
.796
8E+
01
13−4
.139
4E+
01−4
.432
7E+
01−4
.432
7E+
01−4
.432
7E+
01−4
.432
7E+
01−3
.723
8E+
01−4
.135
6E+
01−4
.432
7E+
01−3
.147
3E+
01−4
.432
7E+
01
14−4
.784
5E+
01−4
.784
5E+
01−4
.784
5E+
01−4
.784
5E+
01−4
.784
5E+
01−4
.487
4E+
01−4
.693
5E+
01−4
.784
5E+
01−3
.271
3E+
01−4
.784
5E+
01
15−5
.232
3E+
01−5
.232
3E+
01−5
.232
3E+
01−5
.232
3E+
01−5
.232
3E+
01−4
.922
2E+
01−4
.758
3E+
01−5
.232
3E+
01−3
.819
5E+
01−5
.232
3E+
01
M.S. Norouzzadeh et al.
Tabl
e12
Res
ults
ofM
ann–
Whi
tney
–Wilc
oxon
test
(with
sign
ifica
nce
leve
lof
0.05
)on
benc
hmar
kpr
oble
ms
CE
C20
05(d
im=
10)
CE
C20
05(d
im=
30)
Len
nard
-Jon
es
Prob
lem
ph
Prob
lem
ph
Num
ber
ofat
oms
ph
14.
4922
E-0
10
12.
6874
E-0
10
52.
6162
E-0
10
22.
9475
E-0
10
23.
3440
E-0
71
61.
4359
E-0
21
32.
3656
E-0
10
33.
6690
E-0
91
77.
4064
E-0
51
45.
0032
E-0
20
46.
5743
E-0
91
82.
0616
E-0
61
55.
0945
E-0
10
52.
3172
E-0
31
97.
8980
E-0
81
66.
5289
E-0
20
61.
2532
E-0
10
106.
7956
E-0
81
76.
9080
E-0
10
77.
4230
E-0
20
116.
7956
E-0
81
81.
5460
E-0
41
81.
1029
E-0
21
126.
7956
E-0
81
92.
4478
E-0
71
91.
4157
E-0
91
136.
7956
E-0
81
101.
6231
E-0
10
101.
7005
E-0
21
146.
7956
E-0
81
––
––
––
156.
7956
E-0
81 (LADPSO) has better performance in accuracy, stability and
robustness. The future work includes the research on us-ing similar approaches for solving multi-objective and dy-namic optimization problems. Also researching about de-signing more intelligent systems for controlling the usage ofadditional operators within standard algorithm seems to befruitful. In this paper we have used a fuzzy control system toimprove standard PSO algorithm with 2 operators, it is pos-sible to examine this approach for other operators and opti-mization algorithms. Furthermore it seems that some goodresearches could be done about improving the suggested al-gorithm to solve constrained optimization problems.
Acknowledgement The authors are very grateful to the referees fortheir excellent and many helpful comments, suggestions and correc-tions.
References
1. Kennedy J, Eberhart RC (1995) Particle swarm optimization. In:Proceedings of the 1995 IEEE international conference on neuralnetworks, Piscataway, NJ, pp 1942–1948
2. Engelbrecht AP (2007) Computational intelligence, 2nd edn. Wi-ley, New York
3. Clerc M, Kennedy J (2002) The particle swarm-explosion, stabil-ity, and convergence in a multidimensional complex space. IEEETrans Evol Comput 6(1):58–73
4. Wolpert DH, Macready WG (1997) No free lunch theorems foroptimization. IEEE Trans Evol Comput
5. Pant M, Radha T, Singh VP (2007) Particle swarm optimization:experimenting the distributions of random numbers. In: 3rd Indianint conf on artificial intelligence (IICAI 2007), pp 412–420
6. Pant M, Thangaraj R, Abraham A (2008) Improved particle swarmoptimization with low-discrepancy sequences. In: IEEE cong onevolutionary computation (CEC 2008), Hong Kong
7. Norouzzadeh MS, Ahmadzadeh MR, Palhang M (2010) PlowingPSO: anovel approach to effectively initializing particle swarm op-timization. In: Proceeding of 3rd IEEE international conference oncomputer science and information technology, Chengdu, China,vol 1, pp 705–709
8. Mendes R, Kennedy J, Neves J (2005) The fully informed particleswarm: simpler, maybe better. IEEE Trans Evol Comput, 1(1)
9. Suganthan PN (1999) Particle swarm optimiser with neighbor-hood operator. In: Proceedings of the IEEE congress on evolu-tionary computation, pp 1958–1962
10. Kennedy J, Mendes R (2002) Population structure and particleperformance. In: Proceedings of the IEEE congress on evolution-ary computation, pp 1671–1676
11. Kennedy J (2003) Bare bones particle swarms. In: Proceedings ofthe IEEE swarm intelligence symposium, pp 80–87
12. Standard PSO 2007 and 2011, http://particleswarm.info13. Shi Y, Eberhart RC (2001) Fuzzy adaptive particle swarm opti-
mization. In: Proceedings of the IEEE congress on evolutionarycomputation, pp 101–106
14. Venter G, Sobieszczanski-Sobieski J (2003) Particle swarm opti-mization. J Am Inst Aeronaut Astronaut 41(8):1583–1589
15. Clerc M (2001) Think locally, act locally: the way of life of cheap-PSO, an adaptive PSO. http://clerc.maurice.free.fr/pso/. Technicalreport
16. Zheng Y, Ma L, Zhang L, Qian J (2003) On the convergence anal-ysis and parameter selection in particle swarm optimization. In:Proceedings of the international conference on machine learningand cybernetics, pp 1802–1807
LADPSO: using fuzzy logic to conduct PSO algorithm
17. Chen M-R, Lu Y-Z, Luo Q (2010) A novel hybrid algorithm withmarriage of particle swarm optimization and extremal optimiza-tion. Appl Soft Comput J 10(2):367–373
18. Lim A, Lin J, Xiao F (2007) Particle swarm optimization and hillclimbing for the bandwidth minimization problem. Int J Appl In-tell 26:175–182
19. Shuang B, Chen J, Li Z (2011) Study on hybrid PS-ACO algo-rithm. Int J Appl Intell 34:64–73
20. Angeline PJ (1998) Using selection to improve particle swarm op-timization. In: Proceedings of the IEEE congress on evolutionarycomputation, pp 84–89
21. Pant M, Thangaraj R, Abraham A (2007) A new PSO algorithmwith crossover operator for global optimization problems. In: 2ndinternational workshop on hybrid artificial intelligence systems
22. Higashi H, Iba H (2003) Particle swarm optimization with gaus-sian mutation. In: Proceedings of the IEEE swarm intelligencesymposium, pp 72–79
23. Pant M, Radha T, Singh VP A new diversity based particle swarmoptimization using Gaussian mutation. Int J Math Model, SimulAppl
24. Li C, Yang S, Korejo I An adaptive mutation operator for particleswarm optimization
25. Li C, Liu Y, Zhou A, Kang L, Wang H A fast particle swarm op-timization algorithm with Cauchy mutation and natural selectionstrategy
26. van den Bergh F, Engelbrecht AP (2000) Cooperative learning inneural networks using particle swarm optimizers. S Afr Comput J26:84–90
27. Silva A, Neves A, Costa E (2002) An empirical comparison of par-ticle swarm and predator prey optimisation. In: Proceedings of thethirteenth Irish conference on artificial intelligence and cognitivescience, pp 103–110
28. Xie X, Zhang W, Yang Z (2002) Adaptive particle swarm opti-mization on individual level. In: Proceedings of the sixth interna-tional conference on signal processing, pp 1215–1218
29. Xie X, Zhang W, Yang Z (2002) A dissipative particle swarm op-timization. In: Proceedings of the IEEE congress on evolutionarycomputation, pp 1456–1461
30. van den Bergh F (2002) An analysis of particle swarm optimiz-ers. In PhD thesis, Department of Computer Science, Universityof Pretoria, Pretoria, South Africa
31. Blackwell TM, Bentley PJ (2002) Dynamic search with chargedswarms. In: Proceedings of the genetic and evolutionary computa-tion conference, pp 19–26
32. Suganthan PN, Hansen N, Liang JJ, Deb K, Chen Y-P, AugerA, Tiwari S (2005) Problem definitions and evaluation criteriafor the CEC 2005 special session on real-parameter optimiza-tion. Nanyang Technological University, Singapore and KanpurGenetic Algorithms Laboratory, IIT Kanpur, Technical Report
33. Auger A, Hansen N (2005) A restart CMA evolution strategywith increasing population size. In: IEEE congress on evolution-ary computation (CEC2005), vol 2, pp 1785–1791
34. Qin AK, Suganthan PN (2005) Self-adaptive differential evolutionalgorithm for numerical optimization. In: IEEE congress on evo-lutionary computation (CEC2005), vol 2, pp 1785–1791
35. Qin AK, Suganthan PN (2005) Dynamic multi-swarm particleswarm optimizer with local search. In: IEEE congress on evolu-tionary computation (CEC2005), vol 1, pp 522–528
36. Gockenbach MS, Kearsley AJ, Symes WW (1997) An infeasiblepoint method for minimizing the Lennard-Jones potential. ComputOptim Appl 8(3):273–286
37. Fan E (2002) Global optimization of Lennard-Jones atomic clus-ters. MSc thesis, McMaster University
38. Gibbons JD (1985) Nonparametric statistical inference. MarcelDekker, New York
39. Hollander M, Wolfe DA (1999) Nonparametric statistical meth-ods. Wiley, Hoboken
Mohammad Sadegh Norouzzadehreceived his B.S. degree in Soft-ware Engineering from the TarbiatMoallem University, Iran, in 2007and the Master degree in ArtificialIntelligence from Isfahan Univer-sity of Technology, in 2009. Cur-rently he is Ph.D. student of Artifi-cial Intelligence in Amirkabir Uni-versity of Technology. His currentresearch interests include computa-tional intelligence, machine learn-ing and computer vision.
Mohammad Reza Ahmadzadehreceived his B.Sc. degree in Elec-tronic Engineering from the Fer-dowsi University of Mashhad, Iranin 1989 and the M.Sc. degree inElectronic Engineering from theUniversity of Tarbiat Modarres,Tehran in 1992. He received hisPh.D. from University of Surrey,UK in 2001. He was a lecturer atShiraz University from 1993–1997,and an Assistant Professor from2001–2004. He is an Assistant Pro-fessor of Electrical Engineering atIsfahan University of Technology,
Iran from 2004. His research interests include reasoning with uncer-tainty, pattern recognition, image processing, expert systems, informa-tion fusion and neural networks.
Maziar Palhang received his B.Sc.in Computer Hardware from SharifUniversity of Technology, Tehran,Iran. He received his M.Comp.Sc.and Ph.D. from the University ofNew South Wales, Sydney, Aus-tralia. He was a Postdoctoral fellowin UNSW as well. He later joinedElectrical and Computer Engineer-ing Department of Isfahan Univer-sity of Technology, Isfahan, Iran.He has been the Chairman of Hu-manoid league of IranOpen Com-petitions. His research interests areMachine Learning, Computer Vi-sion, and Robotics.