Date post: | 13-Mar-2016 |
Category: |
Documents |
Upload: | valentina-ovid |
View: | 61 times |
Download: | 6 times |
Artificial Neural Networks- Introduction -
Overview
1. Biological inspiration
2. Artificial neurons and neural networks
3. Why use ANN?
4. ANN Characterization
Biological inspirationAnimals are able to react adaptively to changes in their external and internal environment, and they use their nervous system to perform these behaviours.
An appropriate model/simulation of the nervous system should be able to produce similar responses and behaviours in artificial systems.
The nervous system is build by relatively simple units, the neurons, so copying their behavior and functionality should be the solution.
Action potential
Biological inspiration
Biological inspiration
synapses
axon dendrites
Biological inspiration
Dendrites: Input
Axon: Output
Soma : Cell Body
The information transmission happens at the synapses.
Artificial neuronsNeurons work by processing information.
The McCullogh-Pitts model
Inputs
Outputw2
w1
w3
wn
wn-1
. . .
x1
x2
x3
…
xn-1
xn
y)(;
1
zHyxwzn
iii
Artificial neural network (ANN)is a mathematical model or computational model based on biological neural networks Artificial Neural Network consists of neurons arranged in layers Neurons act as parallel processorNeurons are connected with each other vi connection.there are weights associated with connectionsImplementation:
Learningtesting
Artificial neural networks
Inputs
Output
An artificial neural network is composed of many artificial neurons that are linked together according to a specific network architecture. The objective of the neural network is to transform the inputs into meaningful outputs.
Artificial neural networksDendrites: Input LayerAxon : Output LayerSoma: Net( weighted sum of input y_in) and activation functionSynapse: Weights
Why use ANN?-Adaptive learning: An ability to learn how to do tasks based on
the data given for training or initial experience. -Self-Organization: An ANN can create its own organization or
representation of the information it receives during learning time.
-Real Time Operation: ANN computations may be carried out in parallel, and special hardware devices are being designed and
manufactured which take advantage of this capability. -Fault Tolerance via Redundant Information Coding: Partial
destruction of a network leads to the corresponding degradation of performance. However, some network capabilities may be
retained even with major network damage.
ANN CharacterizationANN can be characterized by:
Activation functionWeights Adjustment (learning algorithm)Architecture
Activation functionFunction to map weighted sum of input into outputDetermine whether neuron fires or notLinear function (Identity)
F(y_in)=y y_in (weighted sum of input) y (output)
Step functiony=f(y_in)=
00_10_
yinyyiny
Activation functionLogistic or sigmoid function
Binary sigmoid• F(y_in)=
Bipolar Sigmoid• F(y_in)=
)_exp(11
inay
)_exp(1)_exp(1
inayinay
Learning AlgorithmLearning in ANN is Weights adjustment to
get desired outputTo minimize the errorTo gain more experience
LearningSupervisedunsupervised
Supervised Learning There is supervisor during learning processInput and output are knownThe job of ANN is to classify any new input according to known classesExample : teaching baby the difference pens and other thingsLVQ (learning vector quantization)
Unsupervised learningInput known but output unknownThe classes are unknown to ANNJob of ANN is to find similarities between input and divide them into categories (cluster)SOM (Self organizing map)
ArchitectureShow the number of layer in Neural NetworkShow the number of neurons in each layerShow how neurons connect to each other
ArchitectureFeed forward
allow signals to travel one way only; from input to output. There is no feedback (loops)
Multi layer
ArchitectureFeedback networks
signals travelling in both directions by introducing loops in the network
linearly separable problemA linearly separable problem is one in which the classes can be separated by a single hyperplane
It is often the case that a problem is not linearly separable. To solve these we use a Multi-Layer Perceptron (MLP) where one layer feeds into the next.
XOR problem
XOR problem
The shape of regions in pattern space that can be separated by a Multi-Layer Perceptron
back-propagationLast time we saw that the delta rule can be used to train a perceptron. When training the MLP, the error (delta) must be propagated back through the layers. This is called error back-propagation. Or just backpropagation.