Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | allyson-ridgely |
View: | 217 times |
Download: | 2 times |
Hopefully a clearer version of Neural Network
I1
O2
O1H1
H2I2
Layers of Weights
• We Name Sets of Weights between layersAs W1 for weights between input Layer and
First Hidden LayerW2 for weights between next 2 layers and WN-1 for Weights between N-1th and Nth
Layer(i.e. Output Layer)In our example Net we just have 3 layersInput Hidden and Output So we have just
W1 and W2
I1
O2
O1H1
H2I2
W1W2
Weights along Individual Links
• Convention
• Each Weight is named as follows
• WNij
• N refers to the Layer of Weights
• So Between Input and First Hiden Layer i.e. W2ij is the Reference
• Between Hidden and Output W2ij
Individual Weights within a layer
• Reference WNij
• WN refers to the Weight Layer
• ij refers to the indices of the source and destination nodes.
• So for example the weight between hidden node h1 and output node o2
• It belongs to weight layer 2 so W2
• i = 1 and j = 2 so Weight is W212
I1
O2
O1H1
H2I2
W1W2
W212
Full Naming of Weight Set
I1
O2
O1H1
H2I2
W1 W2
W212W112
W221
W211
W222
W121
W111
W122
With Actual Weights
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
Inputs
• 1 and 0
• Target outputs {0.7,0.6}
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
1
0
Hidden Layer Computation
• Xi =iW1 = • 1 * 1 + 0 * -1 = 1, • 1 * -1 + 0 * 1 = -1 = • { 1 - 1} = {Xi1,Xi2} = Xi
xF
1
1
• h = F(X)• h1 = F(Xi1) = F(1)• h2 = F(Xi2) = F(-1)
27.01
1
1
1)2(
73.01
1
1
1)1(
)1(2
)1(1
xi
xi
XiF
XiF
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
1
0
0.73
0.27
Next Outputs
Output Layer Computation
• X = hW2 = • 0.73 * -1 + 0.27 * 0 = -0.73, • 0.73 * 0 + 0.27 * -1 = -0.27 =• { -0.73 - 0.27} = {X1,X2} = X
xF
1
1
• O = F(X)• O1 = F(X1)• O2 = F(X2)
433.01
1
1
1)2(
325.01
1
1
1)1(
)27.0(2
)73.0(1
x
x
XF
XF
I1
O2
O1H1
H2I2
W1 W2
0-1
0
-1
-1
0
1
1
1
0
0.73
0.27
0.325
0.433
Error
• D= Output(1 – Output)(Target – Output)• Target T1 = 0.7 , O1 = 0.325 = 0.33
• d1 = 0.33( 1 -0.33)(0.7 -0.33 ) = 0.33 (0.67)(0.37) = 0.082
• Target T2 = 0.6 , O2 = 0.433 = 0.43
• d2 = 0.43(1 - 0.43)(0.6-0.43) = 0.43(0.57)(0.17) = 0.42
Weight Adjustment
• △W2t = α hd + Θ △W2t-1
• where α = 1• Time t = 1 so no previous time
2212
211121
2
1
dhdh
dhdhdd
h
hhd
)042.0*27.0()082.0*27.0(
)042.0*73.0()082.0*73.0(042.0082.0
27.0
73.0hd
Weight Adjustments
)012.0()022.0(
)031.0()06.0(
Weight Change
)012.02()22.02(
)031.02()06.02(
2221
1211
WW
WW
Equals
)012.01()022.00(
)031.00()06.01(
Equals
)988.0()022.0(
)031.0()94.0(
Putting these new weights in the diagram
• To get
I1
O2
O1H1
H2I2
W1 W2
0.031-1
0.022
-0.94
-0.988
0
1
1
Next
• Calculate Change on W1 layer weights
Error Calculatione = h(1 - h)W2d
21
11
h
h
2
1
d
d
2221
1211
22
22
WW
WW 21 hh
2
1
e
e
Another Way to write the error
outputsk
kikiih dWhhe )1(
What is this
• Outputs are O1 and O2
• So k = {1,2}• So if i = 1
outputsk
kikdW
summation
2121112,1,1
dWdWdWdWki
kikoutputsk
kik
I1
O2
O1H1
H2I2
W1 W2
0.031-1
0.022
-0.94
-0.988
0
1
1
This equals
• e1 = (h1(1-h1)W11 D1 +W12D2• e2 = (h2(1-h2)) W21 D1 +W22D2• d1 = 0.082 d2 = = 0.042e1 = (0.73(1-0.73))( -1* 0.082 +0*0.042)• e2 =( 0.27(1-0.27)) (0 *0.082 +-1*0.042)
• e1 = (0.73(0.27)( -0.082))• e2 =( 0.27(0.73)) (-0.042)• e1 = -0.016• e2 = -0.0083
Weight Adjustment
• △W1t = α Ie + Θ △W2t-1
• where α = 1
2212
211121
2
1
eIeI
eIeIee
I
IIe
)0083.0*0()016.0*0(
)0083.0*1()016.0*1(0083.0016.0
0
1Ie
Weight Adjustment
)0()0(
)0083.0()016.0(
Existing W1
10
11
11
11
2221
1211
WW
WW
Weight Change W1
)01()0(
)0083.01()016.01(
New W1
10
0083.1884.0
• Changing Net
I1
O2
O1H1
H2I2
W1 W2
-0.102-1.0083
-0.04
-1.109
-1.038
0
0.884
1