ORIGINAL ARTICLE
A tuned hybrid intelligent fruit fly optimization algorithmfor fuzzy rule generation and classification
Seyed Mohsen Mousavi1 • Madjid Tavana2,3 • Najmeh Alikar1 • Mostafa Zandieh4
Received: 28 December 2016 / Accepted: 15 June 2017 / Published online: 1 July 2017
� The Natural Computing Applications Forum 2017
Abstract Fuzzy rule-based systems (FRBSs) are well-
known soft computing methods commonly used to tackle
classification problems characterized by uncertainties and
imprecisions. We propose a hybrid intelligent fruit fly
optimization algorithm (FOA) to generate and classify
fuzzy rules and select the best rules in a fuzzy if–then rule
system. We combine a FOA and a heuristic algorithm in a
hybrid intelligent algorithm. The FOA is used to create,
evaluate and update triangular fuzzy rule-based and
orthogonal fuzzy rule-based systems. The heuristic algo-
rithm is used to calculate the certainty grade of the rules.
The parameters in the proposed hybrid algorithm are tuned
using the Taguchi method. An experiment with 27
benchmark datasets and a tenfold cross-validation strategy
is designed and carried out to compare the proposed hybrid
algorithm with nine different FRBSs. The results show that
the hybrid algorithm proposed in this study is significantly
more accurate than the nine competing FRBSs.
Keywords Fuzzy rule-based system � Classification
system � Fruit fly optimization � Hybrid algorithm � Taguchi
method
1 Introduction
Fuzzy rule-based systems (FRBSs) are popular computa-
tional intelligence and soft computing methods commonly
used for classification problems. FRBSs utilize linguistic
variables which are accurate and easy to interpret in real-
world problems involving uncertainty and ambiguity.
Accuracy and interpretability are two essential require-
ments of linguistic fuzzy modeling [34]. Accuracy and
interpretability represent the ability to predict and under-
stand the behavior of the system. FRBSs have been suc-
cessful in solving a wide range of classification problems
[5, 6, 7, 8, 11, 40, 43].
Meta-heuristics have played an important role in
employing fuzzy if–then rules for classification problems.
The genetic algorithm (GA) is the most widely used meta-
heuristic in FRBSs. Pourpanah et al. [32] used a hybrid
model for data classification and rule extraction in which
fuzzy ARTMAP with Q-learning was applied for incre-
mental learning of data and a GA was implemented for
feature selection and rule extraction. Tsakiridis et al. [43]
utilized a genetic fuzzy rule-based classification system
with applied differential evolution as its learning algo-
rithm. Derhami and Smith [5] applied a mixed-integer
programming model to extract fuzzy rules from datasets,
& Madjid Tavana
http://tavana.us
Seyed Mohsen Mousavi
Najmeh Alikar
Mostafa Zandieh
1 Department of Mechanical Engineering, Faculty of
Engineering, University of Malaya, Kuala Lumpur, Malaysia
2 Business Systems and Analytics Department, Distinguished
Chair of Business Analytics, La Salle University,
Philadelphia, PA 19141, USA
3 Business Information Systems Department, Faculty of
Business Administration and Economics, University of
Paderborn, 33098 Paderborn, Germany
4 Department of Industrial Management, Management and
Accounting Faculty, Shahid Beheshti University, Tehran,
Iran
123
Neural Comput & Applic (2019) 31:873–885
DOI 10.1007/s00521-017-3115-4
where the model found multiple rules by converting the
obtained optimal solutions into a set of taboo constraints
that prevented the model from re-finding the previously
obtained rules. Rudzinski [34] proposed a multi-objective
genetic algorithm to simultaneously obtain systems with
various levels of compromise between accuracy and
interpretability. Cintra et al. [2] used a genetic fuzzy sys-
tem to improve the interpretability of the fuzzy rule bases
and select the final rule base. Fazzolari et al. [10] analyzed
36 training set selection methods using genetic fuzzy rule-
based classification systems where a wide range of datasets
were utilized to evaluate the proposed method. Sanz et al.
[35] used a post-processing genetic tuning step to enhance
the performance of fuzzy rule-based classification systems
by applying the concept of interval-valued fuzzy sets.
Fazzolari et al. [9] applied a fuzzy procedure which was
integrated within a multi-objective evolutionary algorithm
for performing a tuning and a rule selection process.
Stepnicka et al. [41] used a fuzzy rule-based method to
combine several individual forecasts. In this method, the
weights of the combination were obtained by fuzzy rule
bases according to time series features as well as trend,
seasonality or stationary. Alikar et al. [1] improved the
performance of fuzzy rule-based classification systems on a
dataset by combining particle swarm optimization (PSO)
with a heuristic algorithm.
In order to carry out this, a hybrid algorithm consisting a
FOA and a heuristic proposed by Ishibuchi and Nakaskima
[12] is developed to create fuzzy if–then rules and to obtain
the certainty grade of the rules. In addition, some fuzzy
methods are used to show the features in the dataset. The
fruit fly optimization algorithm (FOA) is a population-
based and global optimization meta-heuristic algorithm
proposed by Pan [30], inspired by the food finding behavior
of the insect of fruit fly. While there is no work in the
literature that uses FOA to improve the performance of
fuzzy rule-based classification systems, FOA has been
becoming very popular in recent years in other fields.
Sheng and Bao [38] optimized a kind of fractional-order
fuzzy PID controller, which was applied to an electronic
throttle. Mousavi et al. [20, 21] solved a multi-item multi-
period inventory-location allocation problem, formulated
into a mixed-integer binary nonlinear programming, using
a modified FOA. Mousavi et al. [22] improved an FOA for
solving an integrated multi-item multi-period two-echelon
supply chain and location allocation problems. Wang et al.
[44] used an improved version of an FOA with swarm
collaboration and random reputation to joint replenishment
problems, which outperformed FOA, PSO and DEA algo-
rithms in terms of accuracy and reliability. Zheng and
Wang [45] optimized an unrelated parallel machine
scheduling problem with additional resource constraints,
formulated into a mixed-integer linear programming
model, in which the aim was to minimize the makespan
using a two-stage adaptive FOA. Lei et al. [18] proposed a
fruit fly clustering optimization to identify dynamic protein
complexes by integrating a fruit fly optimization algorithm
with gene expression profiles. Pan et al. [29] optimized
continuous function optimization problems, where a new
control parameter was applied to adaptively calibrate the
search scope around its swarm location.
In this paper, a design of experiment procedure, i.e.,
Taguchi approach, is used to tune the algorithm parameters
which is applied for the first time in the literature, for rule-
based classification systems. Khanlou et al. [13] used the
Taguchi method to investigate the effects of polymer
concentration and electrospinning parameters on the
diameter of electrospun polymethyl methacrylate (PMMA)
fibers. Several studies have successfully used other design
of experiment methods as well as the response surface
methodology. Khanlou et al. [14] used a regression model
to obtain the dominant factors on the responses where the
molecular weight was appeared to successfully find out the
final objective with a lesser number of experimental runs.
Khanlou et al. [16] applied a central composite design of
RSM to conduct a mathematical model as well as to
characterize the optimum condition where a three-layered
feed-forward artificial neural network was established for
the prediction of the response factor. Khanlou et al. [17]
used an ANFIS model to predict the surface roughness of
titanium biomaterials. Ong et al. [27] applied three algo-
rithms of the Cuckoo search algorithm, flower pollination
algorithm and particle swarm optimization to optimize the
extrusion process parameters in which RSM was used to
tune the algorithms’ parameters. Keshtegar and Heddam
[15] proposed a modified response surface method and a
multilayer perceptron neural network to formulate daily
dissolved oxygen concentration.
2 Fuzzy rule-based classification systems
The old fuzzy rule-based systems are often used for pattern
classification [42]. Pattern classification generally involves
a manual classification of training samples by experts,
where a classifier automatically assigns an unseen data
sample to a set of pre-defined classes [37]. Figure 1 shows
a fuzzy inference system, which includes a fuzzifier for
translating crisp inputs into fuzzy values; a heuristic
algorithm for obtaining a fuzzy output by applying a fuzzy
reasoning mechanism; a defuzzifier for translating this
fuzzy output into a crisp value; and a knowledge base for
storing the fuzzy rules. An if–then rule generally takes the
form of:
If antecedent proposition then consequent proposition
874 Neural Comput & Applic (2019) 31:873–885
123
This work uses fuzzy if–then rules of the following type
for the C = 2 class pattern classification problem:
Rule Rj : If x1 is Lj1 and . . . and
xn is Ljn then ClassCj with CFj for j ¼ 1; 2; . . .;Nð Þð1Þ
where Rj is the label of the j th fuzzy if–then rule, Lj1, …,
Ljn are antecedent of fuzzy sets on the unit interval [0, 1],
Cj is the consequent class (in this study there are 2 classes),
and CFj is the grade of certainty of the fuzzy if–then rule
Rj. This study proposes the antecedents of fuzzy sets, using
two orthogonal and triangular fuzzy sets, as depicted in
Figs. 2 and 3, respectively.
In the next subsection, we apply the proposed hybrid
intelligent algorithm to generate N fuzzy if–then rules, where
each one of the rules follows the form as shown in Eq. (1).
3 The proposed hybrid intelligent algorithm
A hybrid intelligent algorithm comprising a fruit fly opti-
mization algorithm and an extended version of the heuristic
algorithm proposed by Ishibuchi and Nakaskima [12] with
fuzzy features is explained in this section. We generated
and coded several other algorithms including GA, PSO and
SA, in addition to the FOA proposed in this study. In all
instances, the proposed FOA outperformed all other com-
peting algorithms. Furthermore, the proposed FOA is an
enhanced version of the population-based algorithms such
as GA and PSO, and in most cases, the FOA has a superior
performance in comparison with the GA and PSO. Finally,
studies that have compared the FOA to other recent works
show results that are in its favor. Therefore, FOA is a
strong meta-heuristic algorithm for solving the problem in
hand [20, 22]. An introduction of the FOA is presented,
followed by a detailed explanation of the hybrid algorithm,
in which the proposed FOA is integrated with the heuristic
algorithm.
Pan [30] introduced the fruit fly optimization algorithm,
inspired by the behavior of the fruit fly in search of food.
The fruit fly is generally superior to other species because
of its strong sensing and perception. The fruit fly can find
all sorts of scents from far away and use its powerful vision
to find and fly toward the food [30]. The particle foraging
process of the fruit flies is illustrated in Fig. 4.
The hybrid intelligent algorithm is developed to gener-
ate fuzzy if–then rules and classify the dataset explained in
the following stages:
1. Initializing the parameters and population In this
study, the parameters of the FOA algorithm are C1 and C2,
which are two fixed parameters; Pop1 is the number of
articles, Pop2 is the number of fruit flies around each
Fig. 1 Proposed fuzzy
inference system (fuzzy rule-
based system) diagram
Fig. 2 Orthogonal membership functions of two linguistic values
Fig. 3 Membership functions of five linguistic values Fig. 4 Particle foraging process of the fruit flies
Neural Comput & Applic (2019) 31:873–885 875
123
particle, and Gen is the number of generations. Also, the
initial population of fuzzy if–then rules is generated ran-
domly. There are two methods available to represent and
initialize the fuzzy if–then rules. The first method is to
initialize the solutions based on the Pittsburgh approach
applied to orthogonal fuzzy sets. A particle structure for the
orthogonal approach is shown as follows:
where a and b are called the point and the length of
membership function edges, and Lj1, …, Ljn, j = 1, 2,…,
N are antecedent of fuzzy sets, respectively. In the second
method, triangular fuzzy sets, depicted in Fig. 3, represent
five linguistic values, in addition to ‘‘don’t care,’’ and are
denoted by the six symbols listed in Table 1. A particle
structure for the triangular fuzzy sets is displayed as follows:
Aj1 Aj2 Aj3 Aj4 Aj5 Aj6 Aj7 Aj8 Aj9
where vector Aj is the antecedent of triangular fuzzy sets
for rule j (j = 1,2,…, N) which can take one of the six
linguistic values proposed in Table 1.
To clarify how to initialize the rules, a representation of
a particle (fruit fly) for triangular fuzzy if–then rules with
three rules is shown as follows:
A11 A12 A13 A14 A15 A16 A17 A18 A19
A21 A22 A23 A24 A25 A26 A27 A28 A29
A31 A32 A33 A34 A35 A36 A37 A38 A39
2. Generating fruit flies for each particle The number of
Pop2 fruit flies is generated randomly around each one of
the Pop1 particles.
3. Evaluating each fruit fly and particle After the gen-
eration of particles and fruit flies around them, each fuzzy
if–then rule is evaluated by the fitness value. The fitness
value of the fuzzy if–then rule (i.e., Rj) used in this study is
evaluated by the following fitness function [12]:
FitnessðRjÞ ¼ WNCP:NCPðRjÞ �WNMP:NMPðRjÞ ð2Þ
where WNCP is the reward for the correct classification,
NCP(Rj) is the number of correctly classified training
patterns by Rj, WNMP is the penalty for the misclassifica-
tion, and NMP(Rj) is the number of misclassified training
patterns by Rj.
To classify each rule to each class, first calculate the
certainty grade and then obtain the fitness value for that
rule. Calculating the grade of certainty is different for each
one of the orthogonal and triangular fuzzy sets. Assume
that m training patterns (data) xp ¼ ðxp1; xp2; . . .; xpnÞ, p ¼1; 2; . . .;m are given for an n-dimensional and C-class
pattern classification problem. For the orthogonal fuzzy
sets, the grade of certainty for rule Rj is calculated by the
following steps:
Step 1 Obtain the membership functions of each variable
of each training pattern xp p ¼ 1; 2; . . .;m with the fuzzy
if–then rule Rj, as shown in Eq. (3) (fuzzifier phase).
lLowðxpÞ ¼1 xp\a
ðaþ bÞ � xp
ba� xp �ðaþ bÞ
0 xp [ ðaþ bÞ
8><
>:; lHighðxpÞ
¼ 1 � lLowðxpÞð3Þ
where lLowðxpÞ and lHighðxpÞ are the membership
functions of labels Low and High to each training pattern
xp, and a and b are the start point and the length of
membership function edges, respectively.
Step 2 Find the truth value of each rule Rj. The truth
value is computed by using the min operator in fuzzy
logic and combining the antecedent clauses in a fuzzy
fashion, resulting in a continuous value representing the
rule’s degree of activation. Here, the truth value of each
rule Rj is demonstrated by ljðxpÞ.Step 3 For class h, compute bclasshðjÞ as follows:
bclasshðjÞ ¼X
xp2hljðxpÞ ð4Þ
Step 4 Find class hj that has the maximum value of
bclasshðjÞ, as follows:
bclass hðjÞ ¼ max1 � k � C
fbclasskðjÞg ð5Þ
It should be noted that we specify Cj as Cj = [ when
the consequent class Cj of the rule Rj cannot be determined
uniquely, because two or more classes have the maximum
Table 1 Decoding linguistic values
Linguistic value Decoding
Small (S) 1
Medium small (MS) 2
Medium (M) 3
Medium large (ML) 4
Large (L) 5
Don’t care (DC) 6
a1 a2 … an b1 b2 … bn L11 L12 … L1n … LN1 … LNn
876 Neural Comput & Applic (2019) 31:873–885
123
value. In this case, we specify Cj as Class h when a single
class h takes the maximum value. The grade of certainty
CFj is determined as:
CFj ¼bclass hðjÞ � �bP
h bclass hðjÞð6Þ
with
�b ¼P
h 6¼h bclass hðjÞC � 1
ð7Þ
Choose the winning rule using the following formula
[12]:
ljðxpÞ:CFj ¼ max1 � k � C
flkðxpÞ:CFkg ð8Þ
In triangular fuzzy sets, calculation of the grade of
certainty of rule Rj is similar to the steps proposed for
orthogonal fuzzy sets with this difference: In the first step,
the training pattern xp should be normalized in the interval
[0, 1]. Then, one obtains bclass hðjÞ ¼P
xp2h ljðxpÞ using
step 1, where ljðxpÞ is computed as:
ljðxpÞ ¼ minflj1ðxp1Þ; . . .; ljnðxpnÞg ð9Þ
4. Smell-based and vision-based search In the smell-
based search process, all articles and fruit flies around each
particle are updated. First, Pop1 fruit flies are gathered
around each particle and use the position and velocity of
the flies to construct a subpopulation. We obtain the
position of the flies by using:
xrskþ1 ¼ xrsk þ vrskþ1 ð10Þ
where xrsk is the position of the rth fly in dimension s and
iteration k, r ¼ 1 to Pop2, s ¼ 1; 2; . . .; S, vrskþ1 is its
velocity, and Pop2 is the number of flies. Moreover, the
velocities of the flies used in Eq. (11) are calculated using
Eq. (11), as follows:
vrskþ1 ¼ w � vrsk þ C1 � rand � ðpbrsk � xrsk Þ þ C2 � rand � ðgbsk� xskÞ
ð11Þ
In Eq. (11), w is the inertia weight for controlling the
magnitude of the old velocity vrsk when calculating the new
velocity vrskþ1, pbrsk is the position of the best local fly, gbsk is
the position of the best global fly, C1 represents the sig-
nificance of pbk, C2 represents the significance of gbsk, and
rand represents a uniformly distributed real random num-
ber between 0 and 1.
Naka et al. [26] and Shi and Eberhart [39] have shown
that introducing a linearly decreasing inertia weight into
the original PSO significantly improves its performance.
Naka et al. [26] and Shi and Eberhart [39] express the
linear distribution of the inertia weight as follows:
w ¼ wmax �wmax � wmin
Gen� d ð12Þ
where d is the current number of iterations and Gen is the
maximum number of iterations. Equation (12) shows how
the inertia weight is updated, where wmax and wmin are the
minimum and the maximum weights, respectively. The
related results of the two parameters wmax ¼ 0:9 and
wmin ¼ 0:4 are reported in [26, 39].
5. Global vision-based search In this search, the algo-
rithm considers the fitness values of the flies and finds the
best one in a subpopulation.
6. Stopping criterion Several stopping criteria have been
proposed in the literature to stop a meta-heuristic (i.e., a
specific CPU time, a specific fitness function value or a
specific number of generations). We use a specific number
of generations as the stopping rule in our Gen meta-
heuristic algorithm. The flowchart of the proposed hybrid
intelligent algorithm is presented in Fig. 5.
4 Design of experiments
In this section, a Taguchi method, whose parameters are
C1, C2, Pop1, Pop2 and Gen, is applied to tune the FOA
parameters. The Taguchi method utilizes an orthogonal
array to run experiments and to analyze results. We used
the Taguchi method in this study for two reasons: (1)
reducing the time and cost in experimental design. For
example, consider an L12 (211) orthogonal array. In this
example, we can reduce the number of experiments to 12
sets instead of 211 = 2048 sets of experiments and
achieve similar results for a full factorial experimental
design; (2) using a full factorial experimental design does
not allow for adjustment on the level of the factor value,
since the value cannot be adjusted to robustness once it
has been set to be the optimal value. In contrast with the
Taguchi method, once the value has been set to be the
optimal value, it can be adjusted if the factor is recog-
nized to be neutral or an adjustment factor. We used the
Taguchi method in this study because it is widely rec-
ognized as a suitable statistical method for achieving the
closest value to the target in an experimental design
[1, 20, 21, 22, 24, 25, 28].
Obviously, increasing the number of parameters has a
strong influence on the performance of the meta-heuristic
algorithms. This section investigates the behavior of all the
parameters of the proposed FOA. All different combina-
tions of the parameters C1, C2, Pop1, Pop2 and Gen yield
many alternative algorithms. The most frequently applied
and exhaustive approach is a full factorial experiment [19].
This method is not always appropriate, because if the
number of factors becomes significantly high, then it
Neural Comput & Applic (2019) 31:873–885 877
123
Fig. 5 Proposed hybrid
intelligent algorithm flowchart
878 Neural Comput & Applic (2019) 31:873–885
123
becomes increasingly difficult to implement [23]. Frac-
tional factorial experiments (FFEs) are improved by
Cochran and Cox [3] to decrease the number of required
tests. FFEs only allow a portion of the total number of
possible combinations to estimate the main effect of factors
and some of their interactions. Ross [33] used the Taguchi
technique and improved a family of FFE matrices by
decreasing the number of experiments, still producing
adequate information. The orthogonal arrays are used in the
Taguchi method to peruse decision variables with limited
experiments. The factors are divided into controllable and
noise factors, where noise factors cannot be controlled
directly. The Taguchi method determines the optimal level
of the controllable factors according to the concept of
robustness [31].
Taguchi categorizes objective functions into three dis-
tinct groups: the smaller-is-better type, the larger-is-better
type and the nominal-is-best type. Here, a three-level
orthogonal array is used and the objective function is of the
maximizing type. So, the objective function in fuzzy
classification problems is segmented into the larger-the-
better type, where its corresponding S/N ratio is given by
Eq. (13), as follows:
S=N ratio ¼ �10 log1
n1
Xn1
t¼1
1
d2t
!
ð13Þ
In Eq. (13) dt is the objective function value of the
experiment row t, log is the logarithm function, and n1 is
the run times of the objective function. Also, a general
symbol for a three-level standard orthogonal array is rep-
resented by L9, where 9 is the number of rows of the
experiment.
Thus, to determine the best value of the algorithm’s
inputs, a parameter adjustment technique is applied to tune
the control parameters of C1, C2, Pop1, Pop2, and Gen in
FOA. The proper design of the parameters of each algo-
rithm depends on the type of problem. However, many
researchers manually fix parameters and operators
according to the reference values in prior studies. In this
article, the readers are referred to [23] for more information
about the application of the Taguchi method.
The hybrid intelligent algorithm has been coded in
MATLAB R2013a, and the code has been executed on a
computer with 3.8 GHz and 4 GB of RAM. Furthermore,
all the designs of experiments and statistical analyses have
been performed on the Wisconsin dataset using the
MINITAB software version 15.
In this study, the orthogonal array L9 (9 rows) is gen-
erated to analyze using the Taguchi method, in which each
factor includes three levels. Since the goal of the model is
to maximize the classification accuracy, ‘‘the larger than
better type’’ is the most suitable one to use to calculate the
S/N ratio using Eq. (13). Tables 2 and 3 display the values
of the FOA parameters for each level for triangular and
orthogonal fuzzy sets, respectively.
Figures 6 and 7 depict the mean S/N ratio plot for dif-
ferent levels of the parameters for both the FOATFRB and
FOAOFRB approaches, respectively. In this study, the
objective function is run 5 times for each experiment.
Table 4 depicts the Taguchi design of experiments for
the FOA parameters for the orthogonal fuzzy sets accord-
ing to the red lines drawn in Fig. 6. Also, Table 5 proposes
the Taguchi design of experiments for the FOA parameters
for the orthogonal fuzzy sets as shown in Fig. 7.
5 Experimental framework
In this section, we consider 27 real-world classification
datasets for evaluating the proposed fuzzy rule-based sys-
tems experimentally, which are publicly available at the
UCI repository of machine learning datasets and the KEEL
dataset repository. Table 6 presents the characteristics (i.e.,
patterns, features and classes) of the current datasets. The
datasets are chosen for this study to derive a comparison
with the results obtained for the nine various approaches
presented by Cintra et al. [2] and Sanz et al. [36]. The
hybrid intelligent algorithm is used for both triangular and
orthogonal fuzzy classification rule-based approaches,
where their accuracy extracted from each dataset is com-
pared to these nine approaches. We use a tenfold cross-
validation strategy to evaluate the accuracy of the
approaches.
Table 7 shows the accuracy results of all the approaches
extracted from each one of the 27 used datasets. The first
eight approaches described in the second to ninth columns
Table 2 FOA parameter levels
for the orthogonal fuzzy setsParameters 1 2 3
A: C1 1.5 2 2.5
B: C2 1.5 2 2.5
C: Pop1 4 5 7
D: Pop2 50 70 100
E: Gen 300 500 1000
Table 3 FOA parameter levels
for the triangular fuzzy setsParameters 1 2 3
A: C1 1.5 2 2.5
B: C2 1.5 2 2.5
C: Pop1 3 4 7
D: Pop2 60 80 100
E: Gen 200 500 1000
Neural Comput & Applic (2019) 31:873–885 879
123
of Table 7 are explained by Cintra et al. [2] in detail, while
FCA based is explained by Sanz et al. [36].
The results of Table 7 show that on the average,
FOATFRB and FOAOFRB outperform the other approa-
ches in terms of accuracy for the 27 datasets. In Table 7,
the best performance among these datasets is identified in
bold. According to the results in this table, FOAOFRB is
the best in 13 out of the 27 datasets, whereas FOATFRB is
the best in 11 out of the 27 datasets. The last row of the
table depicts the average rankings of each approach.
To compare the accuracy of the clustering approaches
used in this study, several experimental tests are applied to
assess the performance of these approaches. The proposed
FOATFRB and FOAOFRB approaches are compared with
the other methods using Wilcoxon’s signed-rank test to
evaluate whether there are statistically significant
321
-0.23
-0.24
-0.25
-0.26
-0.27
321 321
321
-0.23
-0.24
-0.25
-0.26
-0.27
321
A
Mea
nof
SNra
tios
B C
D E
Main Effects Plot for SN ratiosData Means
Signal-to-noise: Larger is better
Fig. 6 Mean S/N ratio plot of
different parameter levels for
FOATFRB
321
-0.24
-0.25
-0.26
-0.27
-0.28
321 321
321
-0.24
-0.25
-0.26
-0.27
-0.28
321
A
Mea
nof
SNra
tios
B C
D E
Main Effects Plot for SN ratiosData Means
Signal-to-noise: Larger is better
Fig. 7 Mean S/N ratio plot of
different parameter levels for
FOATFRB approaches
880 Neural Comput & Applic (2019) 31:873–885
123
differences among the methods. Tables 8 and 9 show the
results of the statistical comparisons versus FOATFRB and
FOAOFRB, respectively. It is found that Wilcoxon’s test
detects significant differences between FOATFRB and all
the remaining models. The Wilcoxon’s signed-rank test
also detects a significant difference between FOAOFRB
and all the other methods mentioned in Table 7. Consid-
ering the higher predictive accuracy obtained by
FOATFRB and FOAOFRB, it can be concluded that both
algorithms outperform the other methods, while there is no
significant deference between the accuracy results obtained
by FOATFRB and the results using CA based on all the 27
datasets.
The second statistical test performed in this article is
comparing the accuracy results of the methods with each
other using the Bonferroni method. Figure 8 shows an
interval plot of the Bonferroni test for the average of the
results of the approaches on the used datasets with
a = 0.05. The results are in favor of the FOATFRB and
FOAOFRB methods.
Figure 9 compares the average ranking of each classifier
approach using Friedman’s method [4]. Friedman’s and
Iman and Davenport’s tests explored the significant dif-
ferences among the approaches considered. Table 10
shows that both statistical tests reject equivalence of results
between the investigated methods. In the next step, we
applied Holm’s method as the post hoc test to distinguish
the significant difference between the FOATFRB as the
control algorithm and the other approaches.
Table 11 depicts the Holm post hoc test for comparing
all the approaches, where the Holm procedure rejects all
first nine hypotheses, since the corresponding p values are
smaller than the adjusted a. This shows that FOATFRB
performs significantly better than FS, FS ? R, IVFSE,
IVFSWI, IVFSWIE, FARC, FURIA, IVTURS and FCA
based at the significance level a = 0.05. FOATFRB is not
significantly better than FOAOFRB; however, FOATFRB
obtains the best ranks in the Friedman test, according to
Fig. 9.
6 Conclusions and future research directions
In this paper, two linguistic fuzzy rule-based classification
methods (i.e., triangular and orthogonal fuzzy approaches)
are introduced to represent various features of datasets. To
extract the fuzzy rule-based classification and find the best
rules, a hybrid intelligent algorithm is proposed to integrate
a fruit fly optimization algorithm with triangular fuzzy
Table 4 Optimal parameter levels for the FOA algorithm and
orthogonal fuzzy sets
Algorithm Factors Optimal levels
FOA C1 2.5
C2 2.5
Pop1 7
Pop2 70
Gen 1000
Table 5 Optimal parameter levels for the FOA algorithm and tri-
angular fuzzy sets
Algorithm Factors Optimal levels
FOA C1 2
C2 1.5
Pop1 7
Pop2 100
Gen 500
Table 6 Characteristics of the datasets used in the experimental
analysis
Datasets Patterns Features Classes
Australian 690 14 2
Balance 625 4 3
Cleveland 297 13 5
Contraceptive 1473 9 3
CRX 653 15 2
Dermotology 358 34 6
Ecoli 336 7 8
German 1000 20 2
Haberman 306 3 2
Hayes-Roth 160 4 3
Heart 270 13 2
Ionosphere 351 33 10
Iris 150 4 3
Magic 1902 10 2
New-Thyroid 215 5 3
Page-blocks 195 22 2
Penbased 1992 16 10
Pima 768 8 2
Saheart 462 9 2
Spectfheart 267 44 2
Tae 151 5 3
Titanic 2201 3 2
Twonorm 740 20 2
Vehicle 846 18 4
Wine 178 13 3
Winequiality-Red 1599 11 11
Wisconsin 683 9 2
Neural Comput & Applic (2019) 31:873–885 881
123
rule-based (FOATFRB) and orthogonal fuzzy rule-based
(FOAOFRB) approaches. Moreover, to calibrate the
parameters of the FOA, a Taguchi method is derived to get
the best results in the shortest amount of time. In total, 27
popular datasets in the literature are used to evaluate the
efficiency of the proposed hybrid algorithm according to
nine different classifier approaches. The results show the
FOATFRB and FOAOFRB have the best performance,
while the FOATFRB has the highest average in comparison
with the other approaches. Furthermore, some well-known
Table 7 Results obtained by the different approaches
Dataset FS FS ? R IVFSE IVFSWI IVFSWIE FARC FURIA IVTURS FCA based FOATFRB FOAOFRB
Australian 86.38 84.93 85.07 83.48 85.51 85.51 86.09 85.8 88.10 88.24 88.24
Balance 80.96 82.40 82.08 82.40 82.88 87.36 83.68 85.76 79.65 87.36 85.76
Cleveland 58.92 54.54 57.58 57.25 59.59 57.92 56.57 59.60 71.40 69.55 69.88
Contraceptive 53.02 53.90 52.68 52.68 52.55 52.68 54.17 53.36 53.10 53.90 55.63
CRX 85.45 86.83 86.53 82.85 87.75 86.53 86.37 87.14 91.70 91.70 85.48
Dermotology 92.75 90.49 94.69 94.69 94.42 89.94 93.86 94.42 95.10 94.89 96.20
Ecoli 78.60 80.07 76.20 77.69 77.39 80.07 80.06 78.58 71.90 81.20 73.42
German 71.10 73.80 71.90 71.00 70.70 71.60 73.30 73.10 72.10 73.80 73.40
Haberman 73.84 72.19 72.22 73.2 73.51 71.22 72.55 72.85 77.90 76.21 79.14
Hayes-Roth 76.41 79.46 80.23 77.18 80.23 80.20 81.00 80.23 85.40 83.42 80.54
Heart 88.15 86.67 86.3 86.30 85.19 84.44 78.15 88.15 90.10 90.50 90.50
Ionosphere 91.18 90.60 91.17 92.33 90.33 90.32 88.91 89.75 95.90 90.71 96.20
Iris 96.00 94.67 96.67 96.67 96.00 94.00 94.00 96.00 96.30 96.67 96.13
Magic 79.13 80.33 78.81 76.18 78.97 80.49 80.65 79.76 80.10 80.33 83.25
New-Thyroid 92.09 94.42 92.09 93.02 93.49 95.35 95.88 95.35 92.30 95.88 95.30
Page-blocks 94.52 94.89 94.71 93.42 94.52 94.34 95.25 95.07 94.40 94.73 95.40
Penbased 91.82 92.00 91.09 91.16 90.18 92.64 92.45 92.18 81.80 90.09 91.46
Pima 75.71 76.17 74.99 72.00 74.35 74.08 76.17 75.90 78.30 79.14 77.20
Saheart 72.28 69.70 69.28 70.99 68.40 70.77 70.33 70.99 79.30 71.45 76.84
Spectfheart 77.51 77.87 80.52 77.13 80.51 78.64 77.88 80.52 89.20 89.20 91.35
Tae 52.37 54.43 53.68 50.39 54.34 48.41 47.08 50.34 63.50 52.37 65.82
Titanic 77.06 78.87 77.65 77.65 77.65 78.87 78.51 78.87 79.20 80.06 79.23
Twonorm 89.19 90.54 93.24 91.49 92.97 89.32 88.11 92.3 79.70 95.85 92.87
Vehicle 64.66 66.9 65.26 62.89 63.48 68.44 70.21 67.38 56.10 73.20 70.21
Wine 93.24 95.49 96.08 97.76 94.95 96.62 93.78 97.19 98.10 97.19 95.43
Winequiality-Red 59.54 59.66 58.91 56.35 58.97 53.96 51.29 58.28 59.50 61.73 64.20
Wisconsin 96.49 96.78 96.34 95.90 96.78 96.63 96.63 96.49 98.20 96.49 98.42
Average 79.57 79.95 79.85 79.04 79.84 79.64 79.37 80.57 81.42 82.80 83.24
Table 8 Wilcoxon’s test results for FOATFRB
FOATFRB versus W z value p value Hypothesis
FS 353 3.9401 8.1448e-05 Rejected
FS ? R 338 3.5799 3.4379e-04 Rejected
IVFSE 346.5 3.7841 1.5428e-04 Rejected
IVFSWI 351 3.8922 9.9343e-05 Rejected
IVFSWIE 349 3.8440 1.2105e-04 Rejected
FARC 330 3.3875 7.0526e-04 Rejected
FURIA 311 3.4414 5.7866e-04 Rejected
IVTURS 302 3.2128 0.0013 Rejected
CA based 274 2.0423 0.0411 Rejected
Table 9 Wilcoxon’s test results for the triangular fuzzy sets
Triangular versus W z value p value Hypothesis
FS 312 4.0226 5.7563e-05 Rejected
FS ? R 275 3.5714 3.5504e-04 Rejected
IVFSE 334 4.0256 5.6835e-05 Rejected
IVFSWI 335 4.0510 5.1004e-05 Rejected
IVFSWIE 365 4.2284 2.3536e-05 Rejected
FARC 336 4.0765 4.5711e-05 Rejected
FURIA 328 3.8732 1.0742e-04 Rejected
IVTURS 311 3.9957 6.4510e-05 Rejected
882 Neural Comput & Applic (2019) 31:873–885
123
statistical tests including the Wilcoxon’s signed-rank, the
Friedman, Iman–Davenport, Bonferroni and Holm post hoc
tests were conducted for comparison purposes. The results
have been in favor of FOATFRB and FOAOFRB, where
FOATFRB showed slightly better performance over the
FOAOFRB.
For future work, we propose integrating hybrid intelli-
gent algorithms with other meta-heuristic algorithms such
FOAO
RB
FOATFRB
FCA-Based
IVTURS
FURIA
FARC
IVFSWIE
IVFSWI
IVFSE
FS+RFS
90
85
80
75
70
Data
95% Bonferroni CI for the MeanInterval Plot of the proposed approachesFig. 8 Bonferroni interval plots
7.41
6.17
7.228.04
7.19 7.11
6.415.61
4.78
3.06 3.02
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
Aver
age
rank
ings
Fig. 9 Accuracy of the Friedman rankings
Table 10 Friedman’s and
Iman–Davenport’s test results
(a = 0.05)
Test Statistics Critical value p value Null hypothesis
Friedman 72.805 18.307 1.27E-11 Rejected
Iman and Davenport 9.60 1.87 1.338E-13 Rejected
Neural Comput & Applic (2019) 31:873–885 883
123
as Harmony search. In addition, other fuzzy linguistic
approaches can be considered to improve the classification
accuracy of the datasets.
Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of
interest.
References
1. Alikar N, Abdullah S, Mousavi SM, Niaki STA (2013) A hybrid
particle swarm optimization and fuzzy rule-based system for
breast cancer diagnosis. Int J Soft Comput 8(2):126–133
2. Cintra M, Camargo H, Monard M (2016) Genetic generation of
fuzzy systems with rule extraction using formal concept analysis.
Inf Sci 349:199–215
3. Cochran WG, Cox GM (1957) Experimental designs, 2nd edn.
Wiley, New York
4. Demsar J (2006) Statistical comparisons of classifiers over mul-
tiple data sets. J Mach Learn Res 7(Jan):1–30
5. Derhami S, Smith AE (2017) An integer programming approach
for fuzzy rule-based classification systems. Eur J Oper Res
256:924–934
6. Derhami S, Smith AE (2016) A technical note on the paper
‘‘hGA: hybrid genetic algorithm in fuzzy rule-based classification
systems for high-dimensional problems’’. Appl Soft Comput
41:91–93
7. Elkano M, Galar M, Sanz J, Bustince H (2016) Fuzzy rule-based
classification systems for multi-class problems using binary
decomposition strategies: on the influence of n-dimensional
overlap functions in the fuzzy reasoning method. Inf Sci
332:94–114
8. Fallahpour A, Moghassem A (2013) Yarn strength modelling
using adaptive neuro-fuzzy inference system (ANFIS) and gene
expression programming (GEP). J Eng Fabr Fibers (JEFF)
8(4):6–18
9. Fazzolari M, Alcala R, Herrera F (2014) A multi-objective evo-
lutionary method for learning granularities based on fuzzy dis-
cretization to improve the accuracy-complexity trade-off of fuzzy
rule-based classification systems: D-MOFARC algorithm. Appl
Soft Comput 24:470–481
10. Fazzolari M, Giglio B, Alcala R, Marcelloni F, Herrera F (2013)
A study on the application of instance selection techniques in
genetic fuzzy rule-based classification systems: accuracy-com-
plexity trade-off. Knowl Based Syst 54:32–41
11. Gorzałczany MB, Rudzinski F (2016) A multi-objective genetic
optimization for fast, fuzzy rule-based credit classification with
balanced accuracy and interpretability. Appl Soft Comput
40:206–220
12. Ishibuchi H, Nakaskima T (1999) Improving the performance
of fuzzy classifier systems for pattern classification problems
with continuous attributes. IEEE Trans Ind Electron
46(6):1057–1068
13. Khanlou HM, Ang BC, Talebian S, Afifi AM, Andriyana A
(2015) Electrospinning of polymethyl methacrylate nanofibers:
optimization of processing parameters using the Taguchi design
of experiments. Text Res J 85:356–368
14. Khanlou MH, Ang BC, Talebian S, Barzani MM, Silakhori M,
Fauzi H (2015) Multi-response analysis in the processing of poly
(methyl methacrylate) nano-fibres membrane by electrospinning
based on response surface methodology: fibre diameter and bead
formation. Measurement 65:193–206
15. Keshtegar B, Heddam S (2017) Modeling daily dissolved oxygen
concentration using modified response surface method and arti-
ficial neural network: a comparative study. Neural Comput Appl.
doi:10.1007/s00521-017-2917-8
16. Khanlou MH, Ang BC, Kim JH, Talebian S, Ghadimi A (2014)
Prediction and optimization of electrospinning parameters for
polymethyl methacrylate nanofiber fabrication using response
surface methodology and artificial neural networks. Neural
Comput Appl 25:767–777
17. Khanlou MH, Sadollah A, Ang BC, Barzani MM, Silakhori M,
Talebian S (2015) Prediction and characterization of surface
roughness using sandblasting and acid etching process on new
non-toxic titanium biomaterial: adaptive-network-based fuzzy
inference System. Neural Comput Appl 26:1751–1761
18. Lei X, Ding Y, Fujita H, Zhang A (2016) Identification of
dynamic protein complexes based on fruit fly optimization
algorithm. Knowl Based Syst 105:270–277
19. Montgomery DC (2008) Design and analysis of experiments.
Wiley, New York
20. Mousavi SM, Alikar N, Niaki STA (2016) An improved fruit fly
optimization algorithm to solve the homogeneous fuzzy series–
parallel redundancy allocation problem under discount strategies.
Soft Comput 20(6):2281–2307
21. Mousavi SM, Sadeghi J, Niaki STA, Tavana M (2016) A bi-
objective inventory optimization model under inflation and dis-
count using tuned Pareto-based algorithms: NSGA-II, NRGA,
and MOPSO. Appl Soft Comput 43:57–72
22. Mousavi SM, Alikar N, Niaki STA, Bahreininejad A (2015)
Optimizing a location allocation-inventory problem in a two-
echelon supply chain network: a modified fruit fly optimization
algorithm. Comput Ind Eng 87:543–560
23. Mousavi SM, Hajipour V, Niaki STA, Alikar N (2013) Opti-
mizing multi-item multi-period inventory control system with
discounted cash flow and inflation: two calibrated meta-heuristic
algorithms. Appl Math Model 37:2241–2256
24. Mousavi SM, Hajipour V, Niaki STA, Aalikar N (2014). A multi-
product multi-period inventory control problem under inflation
and discount: a parameter-tuned particle swarm optimization
algorithm. Int J Adv Manuf Technol 70(9–12):1739–1756
25. Mousavi SM, Hajipour V, Niaki STA, Alikar N (2013) Opti-
mizing multi-item multi-period inventory control system with
discounted cash flow and inflation: two calibrated meta-heuristic
algorithms. Appl Math Model 37(4):2241–2256
26. Naka S, Genji T, Yura T, Fukuyama Y (2001) Practical distri-
bution state estimation using hybrid particle swarm optimization.
In: Proceeding IEEE Power engineering society winter meeting,
vol 2. USA, p 815–820
Table 11 Holm’s test results
i Approach z p value a/(k - i) Null hypothesis
1 FS 4.862 0.0000 0.0050 Rejected
2 FS ? R 3.488 0.0002 0.0056 Rejected
3 IVFSE 4.657 0.0000 0.0063 Rejected
4 IVFSWI 5.560 0.0000 0.0071 Rejected
5 IVFSWIE 4.616 0.0000 0.0083 Rejected
6 FARC 4.534 0.0000 0.0100 Rejected
7 FURIA 3.754 0.0001 0.0125 Rejected
8 IVTURS 2.872 0.0020 0.0167 Rejected
9 CA based 1.949 0.0257 0.0250 Rejected
10 FOAOFRB 0.041 0.4836 0.0500 Failure to reject
884 Neural Comput & Applic (2019) 31:873–885
123
27. Ong P, Chi DDVS, Ho CS, Ng CH (2016) Modeling and opti-
mization of cold extrusion process by using response surface
methodology and metaheuristic approaches. Neural Comput
Appl. doi:10.1007/s00521-016-2626-8
28. Pasandideh SHR, Niaki STA, Mousavi SM (2013) Two meta-
heuristics to solve a multi-item multiperiod inventory control
problem under storage constraint and discounts. Int J Adv Manuf
Technol 69(5–8):1671–1684
29. Pan Q-K, Sang H-Y, Duan J-H, Gao L (2014) An improved fruit
fly optimization algorithm for continuous function optimization
problems. Knowl-Based Syst 62:69–83
30. Pan W-T (2012) A new fruit fly optimization algorithm: taking
the financial distress model as an example. Knowl Based Syst
26:69–74
31. Phadke MS (1995) Quality engineering using robust design.
Prentice Hall PTR, Englewood Cliffs
32. Pourpanah F, Lim CP, Saleh JM (2016) A hybrid model of fuzzy
ARTMAP and genetic algorithm for data classification and rule
extraction. Expert Syst Appl 49:74–85
33. Ross PJ (1996) Taguchi techniques for quality engineering: loss
function, orthogonal experiments, parameter and tolerance
design. McGraw-Hill Professional, New York
34. Rudzinski F (2016) A multi-objective genetic optimization of
interpretability-oriented fuzzy rule-based classifiers. Appl Soft
Comput 38:118–133
35. Sanz J, Fernandez A, Bustince H, Herrera F (2011) A genetic
tuning to improve the performance of fuzzy rule-based classifi-
cation systems with interval-valued fuzzy sets: degree of igno-
rance and lateral position. Int J Approx Reason 52(6):751–766
36. Sanz JA, Fernandez A, Bustince H, Herrera F (2013) IVTURS: a
linguistic fuzzy rule-based classification system based on a new
interval-valued fuzzy reasoning method with tuning and rule
selection. IEEE Trans Fuzzy Syst 21(3):399–411
37. Schaefer CF, Anthony K, Krupa S, Buchoff J, Day M, Hannay T,
Buetow KH (2009) PID: the pathway interaction database.
Nucleic Acids Res 37(suppl 1):D674–D679
38. Sheng W, Bao Y (2013) Fruit fly optimization algorithm based
fractional order fuzzy-PID controller for electronic throttle.
Nonlinear Dyn 73(1–2):611–619
39. Shi Y, Eberhart RC (1999) Empirical study of particle swarm
optimization. In: Proceedings of the 1999 Congress on Evolu-
tionary Computation, vol 3. p 1945–1950
40. Singh S, Olugu EU, Fallahpour A (2014) Fuzzy-based sustainable
manufacturing assessment model for SMEs. Clean Technol
Environ Policy 16(5):847–860
41. Stepnicka M, Burda M, Stepnickova L (2015) Fuzzy rule base
ensemble generated from data by linguistic associations mining.
Fuzzy Sets Syst 285:140–161
42. Sugeno M (1985) An introductory survey of fuzzy control. Inf Sci
36(1):59–83
43. Tsakiridis NL, Theocharis JB, Zalidis GC (2016) DECO 3 R: a
differential evolution-based algorithm for generating compact
fuzzy rule-based classification systems. Knowl Based Syst
105:160–174
44. Wang L, Shi Y, Liu S (2015) An improved fruit fly optimization
algorithm and its application to joint replenishment problems.
Expert Syst Appl 42(9):4310–4323
45. Zheng X-L, Wang L (2016) A two-stage adaptive fruit fly opti-
mization algorithm for unrelated parallel machine scheduling
problem with additional resource constraints. Expert Syst Appl
65:28–39
Neural Comput & Applic (2019) 31:873–885 885
123