Post on 12-Feb-2018
transcript
Emergent Intelligence in Cellular Neural Networks Bipul Luitel and G. Kumar Venayagamoorthy
iambipul@ieee.org, gkumar@ieee.org
Real-Time Power and Intelligent Systems (RTPIS) Lab., Holcombe Dept. of Electrical and Computer Engineering, Clemson University, Clemson, SC 29634
This work is supported by NSF grant EFRI #1238097 and CAREER grant ECCS #1231820. Visit: http://rtpis.org and http://brain2grid.org
Decentralized Asynchronous Learning in Cellular Neural Networks [1]
Communication Unit
Lear
nin
g U
nit
Co
mp
uta
tio
nal
Un
it
Application
Cellular Neural Network (CNN) A Generic Cell CNN in Decentralized Asynchronous Learning (DAL) Framework
Implementation (Hardware/software)
Formation
Structure: architecture and size
Centralized Decentralized
Parallel Sequential
Homogeneous Heterogeneous
Learning or adaptation
Learning Method
Adaptation
Asynchronous Synchronous
Homogeneous Heterogeneous
Subsystem: SSSystem: S
S
..
..
..
..
Z-1
)1( kOiSS
SS2(KS2,KD2)
SS1(KS1,KD1)
SSN(KSN,KDN)
SSi(KSi,KDi)
SSi(KSi,KDi)
(a)
(b)
)(kIiSS
Environment
<
Intelligence in CNN Emerges From A Group of Cells .. Learning of Learning Systems
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
2 3
1 4
Knowledge Propagation in CNN
C7
C3C2
C5
C4
C6
C9
C8 C1
C11
C10
C12
C13
C16
C15
C14
MLP SRN
CNN Implementation of 16-generator 68-bus System
Cell 1Cell 2Cell 3Cell 4Cell 5Cell 6Cell 7Cell 8Cell 9
Cell 10Cell 11Cell 12Cell 13Cell 14Cell 15
Cell 16
10 11 12 13 14 15 16 17 18 19 20
Time (s)
Results
Asynchronous Learning in 16-cells of a Heterogeneous CNN
Prediction of Generator Speed Deviation
Learning of learning systems is a social behavior of swarms where each individual learns at different pace, at different times and in different environment while still interacting with the other individuals of
the society. Learning in an artificial system having spatially distributed interacting individuals is known as learning of learning systems. In a decentralized asynchronous learning framework, learning takes place
locally on spatially distributed cells that learn asynchronously. The cells collaborate to achieve learning of the overall system. Cognitive learning takes place when parameters directly affecting the cell change and the cell has to update itself to reflect the change. This new acquired knowledge is then transferred to the
other members of the network (neighboring cells) through the communication unit. As a result, the neighbors observe a change in the behavior, and update themselves. The new knowledge acquired by
one cell thus propagates through he network which results in social learning.
Wide area monitoring system developed using CNN with DAL framework is able to predict speed deviation of 16 generators in a 16-generator, 68-bus system. The CNN is developed using heterogeneous structure (different computational units in different cells) as well as learning method (some cells trained using backpropagation while others using particle swarm optimization algorithm. Threshold error for learning in each cell is set at 5% .
-202
x 10-3
1
-202
x 10-3
2
-202
x 10-3
3
-20
2x 10
-3
4
-202
x 10-3
5
-2
0
2x 10
-3
6
-2
0
2x 10
-3
7
-2
0
2x 10
-3
8
-2
0
2
x 10-3
9
-2
0
2
x 10-3
10
-2
0
2
x 10-3
11
-2
0
2
x 10-3
12
-2
0
2
x 10-3
13
-2
02
x 10-3
14
-2
0
2
x 10-3
15
10 11 12 13 14 15 16 17 18 19
-202
x 10-3
16
Actual
Predicted
Time (s)
𝑂𝑆𝑆𝑖𝑘 = 𝑓 𝛼𝑖𝑂𝑆𝑆𝑖
𝑘 − 1 , 𝛼𝑛1𝑂𝑆𝑆𝑛
1 𝑘 − 1 , . . . , 𝛼𝑛𝑁𝑂𝑆𝑆𝑛
𝑁 𝑘 − 1 , 𝐾𝑆𝑖 , 𝐾𝐷𝑖
.. that may be either spatially collocated or distributed across a wide geographic area.
.. that may either be identical in architecture and size or different from
each other in their structure.
.. that are either implemented sequentially or in parallel hardware and/or software
platform.
.. that are learning subsystems, all of which either utilize the same or each of which
utilizes a different learning method.
.. that either adapt themselves in synchrony with the other subsystems or independent of the others in their own
pace. Intelligence in CNN emerges over time
through progressive learning and adaptation of these distributed interacting
subsystems represented as cells.
Cell
Computational Unit
Learning Unit
Dynamic
Database
Comm.
Unit
Dynamic
database
buffer
wid = 0,..,ws
Output
BufferI/O
+
-
T
Z-1
)(kIn
)1(
kOn
Computational
element
+
+
Error
Buffer
Target
ARE
If(u1 > u2)
T = 1
Else T = 0 AREt
u1
u2
Structural mirror of
computational
element
Z-1
Input
Target
+
-
[1] Luitel B., Venayagamoorthy G.K., “Decentralized Asynchronous Learning In Cellular Neural Networks,” IEEE Trans. On Neural Networks and Learning Systems, To Appear