31
What fuzziness is?
2
If we separate a group of people assuming that every
person of height above 1.75 m is TALL :
1,75 m
1
0
1,75 m
tall tall
1
0
=0 =1
=0,3
=0,8
3
COLD COOL WARM HOT
0 – 15 C 15-25 C 25 – 38 C 40-100 C
COLD COOL WARM HOT
0 – 20 C 20-27 C 27 – 45 C 45-100 C
WARM HOTCOOL WARM
C C
What fuzziness is?
4
If Z is a space of elements, with main
component Z described by z, such that:
Z = {z}
Thus a fuzzy set A in Z is characterized
by a membership function A(z), which
associate each point in z with real number in
[0,1], along with the elements A(z) for z,
which are called degree of membership of
z in A.
If the value of A(z) is closer to one, then a
greater value of the degree of membership
of z to A is assigned.
A={z, (z)}, z Z, A(z) [0,1]
( ) 1
A
z ab z a
b a
z z b
c zc z b
c b
A(z)
za b c
Fuzzy Set
5
A(z)
B(z)
C(z)
2
1( )
1
B bz
z c
a
2( ) exp
2A
z cz
z
z
z
( ) 1
C
z ab z a
b a
z c z b
d zc x b
d c
c
a b c d
c b a
Fuzzy Sets- type 1
6
(z)
1- (z)0(z) +(z) 1
Lower MF
Upper MF
Type 1 MF
FOU
Type 2 Interval Fuzzy Set
Intuitionistic Fuzzy Set
Footprint Of
Uncertainty
(z)
Other types of Fuzzy Sets
SUBSET А B z Z A(z)<B(z)
UNION А B z Z max{A(z), B(z)}
INTERSECTION А B z Z min{A(z), B(z)}
Fuzzy sets operations
7
B
A
A B
A B
EQUITY А B z Z A(z)=B(z)
COMPLEMENT Ã z Z Ã(z)=1-A(z)
8
A
AB
Fuzzy sets operations
Types of Fuzzy inferences
•Mamdani Fuzzy Inference
• Sugeno Fuzzy Inference
• ANFIS (Adaptive Neuro-Fuzzy Inference System)
• Takagi-Sugeno Neuro-Fuzzy model
• Tsukamoto Fuzzy Inference
9
Generalized structure of a Fuzzy Inference
10
FUZZIFICATIONINFERENCE
MECHANISMDEFUZZIFICATION
Fuzzy RulesBASE
INPUTOUPUT
Sugeno Fuzzy Inference: Fuzzification
11
FUZZIFICATION
INPUT
The main purpose of the “fuzzification” procedure is to
transform the crisp input values into fuzzy couples –
linguistic variable of a set and a corresponding
membership degree!
(z)
z
CRISP VALUE
1,2,... 5... 100
FUZZY COUPLE
{(z), Z}
Z
12
Fuzzy rules Base
( ) ( ) ( )1 1: z ....... ( )i i i
s s iR if is Z and z is Z then F z
HOW many fuzzy rules are
being generated?
FR= NP
N – number of the membership functions
per input
P – number of the input parameters
1 1 2 2( ) .... s sF z a z a z a z
Sugeno Fuzzy Inference: Rules base
Sugeno Fuzzy Inference mechanism
13
(z1)
z1
(z2)
z2
AND
min {(z1), (z2)}} (z1)
z1
(z2)
z2
Fi(z)
Fl(z)
OR
max {(z1), (z2)}}
Sugeno Fuzzy Inference: Defuzzification
14
( )
1
( )
1
( ) ( )
( )
FR ii y
FR iy
F z uu
u
DEFUZZIFICATION
OUPUT
1 2( ) ( ) * ( ) *.....* ( )y j j p ju u u u
Ne0-fuzzy neuron
• The NEO-Fuzzy Neuron concept enables the possibility to model complexdynamics with less computational effort, compared to classical Fuzzy-NeuralNetworks.
• Unfortunately, its application in purpose to process modeling and controlunder uncertainties/ data variations, have not been studied yet.
• In the presented approach, the conventional concept is extended with Type-2Interval Fuzzy Logic in order to be achieved overall robustness of theproposed model
• Thus, introducing Type-2 Fuzzy Logic in purpose of handling uncertainvariations is beneficial for modeling different plant processes with complexdynamics.
• To overcome some deficiencies in the classical gradient learning approach, asimple heuristic approach is introduced.
15
Neo-fuzzy neuron
The NEO-Fuzzy neuron is similar to a 0-th orderSugeno fuzzy system, in which only one input isincluded in each fuzzy rule, and to a radial basisfunction network (RBFN) with scalar argumentsof basis functions
In fact the NFN network is a multi-input single-output system – MISO !
The NEO-Fuzzy neuron has a nonlinearsynaptic transfer characteristic.
The nonlinear synapse is realized by a set offuzzy implication rules.
The output of the NEO-Fuzzy neuron isobtained by the following equation:
m
j
jj wkxxf1
))(()(
16
Type-2 Neo-fuzzy network• The MISO NEO-fuzzy neural network
topology can be represented as:
where x(k) is an input vector of the states interms of different time instants.
• Each Neo-Fuzzy Neuron comprises asimple fuzzy inference which producesreasoning to singleton weightingconsequents:
• Each element of the input vector is beingfuzzified using Type-2 Interval Fuzzy set:
ˆ( ) ( ( ))y k f x k
( ) ( ): ( )i ii i i iR if x is A then f x
2 as
( ) exp as 2
ij ij iji ij
ij i
ij ij ijij
x cx
17
Type-2 Neo-fuzzy network
• The fuzzy inference should match the output of the fuzzifier with fuzzylogic rules performing fuzzy implication and approximation reasoningin the following way:
• The output of the network is produced by implementing consequencematching and linear combination as follows:
which in fact represents a weighted product composition of the i-th
input to j-th synaptic weight.
1
1
**
*
n
ij iji
ij n
ij iji
1 1
1 1ˆ( ) ( * *) ( ) ( * *)
2 2
l l
ij ij i i ij ij ijj iy k f x w
18
• To train the proposed modeling structure an unsupervised learningscheme has been used. Therefore, a defined error cost term is beingminimized at each sampling period in order to update the weights:
• As learning approach of the proposed modeling structure a simpleheuristic backpropagation approach , where the scheduled parametersdepend on the signum of the gradient and defined learning rate, isadopted:
2
ˆ and 2 dE k y k y k
( )( 1) ( ) ( ) ( ) ( )
( )ij ij ij ij ij
ij
E kw k w k w k w k k sign
w k
ˆ ˆ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( )
ˆ( ) ( ) ( ) ( )ij ij ij ij
ij ij ij
E k E k y k y kw k k sign k sign k sign k
w k y k w k w k
Learning Algorithm
19
• The learning rate is local to each synaptic weight and it is adjusted by taking into account the extent of the gradient in the current and the past sample period as:
where the constants are: a=1.2, b=0.5 and ηmin=10-3, ηmax=5.
The main advantage of the proposed approach is that the information aboutthe gradient is neglected, which accelerates significantly the learningprocess!
max
min
min ( 1), ( ) ( -1) 0
( ) max ( 1), ( ) ( -1) 0
( 1) ( ) ( -1) 0
ij ij ij
ij ij ij ij
ij ij ij
a k if E k E k
k b k if E k E k
k if E k E k
Learning Algorithm
20
Numerical examples• To test the modeling capabilities of the proposed NEO-fuzzy neural
network, a numerical experiments in prediction of two common chaotictime series (Mackey-Glass a and Rossler )are investigated.
• The Rossler chaotic time series are described by three coupled first-order differential equations:
a=0.2; b=0.4; c=5.7 and initial conditions x0=0.1; y0=0.1; z0=0.1
• The Mackey-Glass (MG) chaotic time series is described by the following time-delay differential equation:
a=0.2; b=0.1; C=10; initial conditions x0=0.1 and τ= 17s.
- - ( - )dx dy dz
y z x ay b z x cdt dt dt
( ) ( - )( 1)
(1 ( - )) - ( )c
x i ax i sx i
x i s bx i
21
Numerical examples
Modeling of Mackey-Glass and Rossler chaotic time series and
the estimated error in the noiseless case.
22
Numerical examples
Modeling of Mackey-Glass and Rossler chaotic time series and
the estimated error in the case of 5% additive noise and 5% FOU.
23
Numerical examples
Modeling of Mackey-Glass chaotic and Rossler time series and the
estimated error in the case of 5% additive noise and 10% FOU.
24
Numerical examples
Mean Squared Errors
Time step
Without noise10-4
With noise and 5%
FOU,10-4
With noise and
10% FOU,10-4
50 4.70 4.66 4.62100 2.86 2.70 2.64150 3.37 3.90 2.95200 8.07 7.47 6.97250 39.88 22.33 21.82300 81.71 72.81 70.13
Comparison of the proposed heuristic
algorithm to the classical Gradient
Descent.
25
MIMO NEO-fuzzy network
MIMO Neo-Fuzzy Network
m
j
jjw wkxxF1
))(()(
m
j
jjw vkxxG1
))(()(
p
j
wjm xFy1
1 )(
p
j
wjm xGy1
2 )(
The outputs of the MIMO NEO-Fuzzy Network areobtained on the last fifth layer:
On the third layer the obtained membershipdegrees are multiplied by two different groupweight coefficients wji(k) and vji(k). On the fourthlayer are computed two groups of functions:
The MIMO NEO-Fuzzy Network is five-layerstructure.
26
Learning of MIMO NEO-fuzzy network
• In the MIMO Neo-Fuzzy Network is need to be adjusted only one groupof parameters – the consequents.
• The defined error cost terms are being minimized at each samplingperiod in order to update the synaptic weights:
• The updating rules in which w and v are vectors of the trained parameters: the synaptic links in the consequent part of the rules and ηis an adaptive learning rate:
2
1 1 1 1 1ˆ/ 2 ( ) ( )m mE y k y k
2
2 2 2 2 2ˆ/ 2 ( ) ( )m mE y k y k
))(()()(
)(
)()()()1( 1
kxkekw
kw
kEkwwkwkw
iijij
ijijij
))(()()(
)(
)()()()1( 2
kxkekv
kv
kEkvvkvkv
iijij
ijijij
2
))((*1.0)( kxk iij
27
Learning of MIMO NEO-fuzzy network
• To demonstrate the ability of the proposed MIMO NEO-FuzzyNetwork it is chosen to model the following nonlinear system:
where u1(k) and u2(k) are the system inputs, y1(k) and y3(k) are theoutputs. Two benchmark chaotic systems (Mackey-Glass andRossler chaotic time series) are used as inputs:
)1(5.0)1()1()1(
)1()(
)1(3.01)1(
)1()(
)1()1()1()1(
)1()(
)1(5.01)1(
)1()(
224
22
21
23
4
423
23
3
124
23
22
21
2
221
21
1
kukykyky
kyky
kyky
kyky
kukykyky
kyky
kyky
kyky
28
Numerical Examples
0 50 100 150 200 250 300-0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
y1 ymod1 y2 ymod2
Steps RMSE1 MSE1 RMSE2 MSE2
50 2.0e-4 5.5e-8 6.1e-5 3.7e-9
100 1.8e-4 4.8e-8 4.6e-5 3.4e-9
150 5.5e-5 3.7e-8 3.4e-5 2.8e-9
200 4.2e-5 3.1e-8 3.1e-5 2.2e-9
250 3.8e-5 2.4e-8 2.7e-5 1.8e-9
300 3.5e-5 1.9e-8 2.1e-5 1.3e-9
Model validation by using Mackey-Glass
chaotic time series as inputs
29
0 50 100 150 200 250 300-0.4
-0.2
0
0.2
0.4
0.6
0.8
1y1 ymod1 y2 ymod2
Steps RMSE1 MSE1 RMSE2 MSE2
50 1.69e-4 2.85e-8 4.8e-5 2.32e-9
100 1.62e-4 2.6e-8 4.97e-5 3.1e-9
150 1.58e-4 2.5e-8 5.6e-5 6.2e-9
200 9.8e-3 9.7e-7 2.7e-4 7.5e-8
250 6.4e-3 4.1e-5 1.5e-4 2.1e-7
300 3.8e-3 1.5e-6 1.2e-4 4.4e-6
Model validation by using Rossler
chaotic time series as inputs
Numerical Examples
30
31
