Reliable All-Pairs Evolving Fuzzy Classifiers
Edwin Lughofer and Oliver Buchtala
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 21, NO. 4, AUGUST 2013
Outline
• CLASSIFIER STRUCTURE
• TRAINING PHASE
• CLASSIFICATION PHASE
• Experiment
• is the degree of preference of class k over class l (The degree lies in [0, 1])
• = 1 −
CLASSIFIER STRUCTURE
K(K − 1) binary classifiers
• is a classifier to separate samples that belong to class k from those that belong to class l
• is a training data
• L() being the class label associated with feature vector
CLASSIFIER STRUCTURE
CLASSIFIER STRUCTURE
• In this paper, we are concentrating on two fuzzy classification architectures:
– singleton class labels– regression-based classifiers
Singleton Class Labels
• being the jth membership function (fuzzy set) of the ith rule• is the crisp output class label from the set of two classes (The degree is {0, 1})
TRAINING PHASE
• Input(training data):
(n) = ( (1) , y(1) ) , ( (2) , y(2) ) ,....,( (n) , y(n) )
y(n) containing the class labels as integer values in {0, . . . , K − 1}
TRAINING PHASE
For each input sample s(n) = (x(n), y(n)) DoObtain class label L = y(n)For k = 1, . . . , L − 1, call (upd) For k = L + 1, . . . , K call (upd)
End For
TRAINING PHASEUpdateBinaryClassifier:input: , y = 1 (if belongs to class k, y=0)If= ∅
Set first cluster center to current sample ()Set = with > 0 ( be a very small value)Set the number of rules: C = 1 Set the number of samples: = 1.Set a matrix H: = 1 (there are one input belong to class L) = 0 (there are no input belong to class k) y={1,0}
TRAINING PHASE
input (new center)
TRAINING PHASE
H:Class k Class L
Cluster 1 0 1
Cluster 2
Cluster 3
….
….
…..Cluster n
TRAINING PHASEElse
Find the value of win:
A being a distance metric
If the distance( ) is larger than ρ:Set the number of rules: C = C+1 Set new cluster center: Set the number of samples: = 1Set = with > 0 ( be a very small value) Update the matrix H: = 1 (there are one input belong to class L) = 0 (there are no input belong to class k)
TRAINING PHASE
Cluster 1
inputclass kclass Lcenter
TRAINING PHASEIf the distance( ) is smaller than ρ:
Update old center (): ( )
Update range of influence():
( Δc = (new) - (old) )
Update the matrix H: = + 1 Update the number of samples: = + 1
TRAINING PHASE
Cluster 1
inputclass kclass Lcenter
TRAINING PHASE
Cluster 1
inputclass kclass Lcenter
TRAINING PHASE
Cluster 1
inputclass kclass Lcenter
TRAINING PHASE
• Project updated/new cluster to axes to form Gaussian fuzzy sets and antecedent parts of rules:– a) one cluster corresponds to one rule;– b) each cluster center coordinate of the ith cluster ( , j = 1, . . . , p)
corresponds to a center of a fuzzy set ( , j = 1, . . . , p) appearing in the antecedent part of the rule;
– c) the length of each cluster axis of the ith cluster corresponds to the width of a fuzzy set ( , j = 1, . . . , p)
TRAINING PHASE
j = 1,2,3,……..,p
CLASSIFICATION PHASE
• The classification outputs are produced in two stages.– 1) The first stage produces the output confidence levels
(preferences) for each class pair and stores it in the preference relation matrix
– 2) The second stage uses the whole information of the preference matrix and produces a final class response.
CLASSIFICATION PHASE
belongs to the nearest rule that supports class k belongs to the nearest rule that supports class l is the membership degree of the current sample to the nearest rule that supports class k is the membership degree of the current sample to the nearest rule that supports class k
( T denoting a t-norm )
CLASSIFICATION PHASE
=>
[0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2]=2.0 [0.8 0.0 0.0 0.0 0.0 0.8 0.0 0.0 0.0 0.0]=1.6
……… output
Regression-Based Classifiers
• being the jth membership function (fuzzy set) of the ith rule•
TRAINING PHASE
• Input(training data):
(n) = ( (1) , y(1) ) , ( (2) , y(2) ) ,....,( (n) , y(n) )
y(n) containing the class labels as integer values in {0, . . . , K − 1}
TRAINING PHASE
For each input sample s(n) = (x(n), y(n)) DoObtain class label L = y(n)For k = 1, . . . , L − 1, call (upd) For k = L + 1, . . . , K call (upd)
End For
TRAINING PHASEUpdateBinaryClassifier:input: , y = 1 (if belongs to class k, y=0)If= ∅
Set first cluster center to current sample ()Set = with > 0 ( be a very small value)Set the number of rules: C = 1 Set the number of samples: = 1Set the wight : = Set the weighted inverse Hessian matrix: = αI
TRAINING PHASEElse
Find the value of win:
A being a distance metric
If the distance( ) is larger than ρ:Set the number of rules: C = C+1 Set new cluster center: Set the number of samples: = 1.Set = with > 0 ( be a very small value)Set the wight : = Set the weighted inverse Hessian matrix: = αI
TRAINING PHASEIf the distance( ) is smaller than ρ:
Update the wight():
being the normalized membership function value for the (N + 1)th data sample =
Update weighted inverse Hessian matrix():
Update the number of samples: = + 1
TRAINING PHASE
j = 1,2,3,……..,p
CLASSIFICATION PHASEFor :
For : y(k,l)=
If is lying outside the interval [0,1],we round it toward the nearest integer in {0, 1}
CLASSIFICATION PHASE
……… output
Ignorance
• Ignorance belongs to that part of classifier’s uncertainty that is due to a query point falling into the extrapolation region of the feature space
Ignorance
IF
then
Experiment