ANN_Material

8/7/2019 ANN_Material

1/27

Workshop on Neural Network Approach for Image Processing, Feb 4th & 5th, 2011

Dr. D. Devaraj, IEEE-KLU1

CHAPTER 1

FUNDAMENTALS OF ARTIFICIAL NEURAL NETWORK

1.1 Introduction

A majority of information processing today is carried out by digital computers. There are

many tasks that are ideally suited to solution by conventional computers: scientific and

mathematical problem solving, database creation, manipulation and maintenance,electronic communication, word processing, graphics and desktop publication, even the

simple control functions that add intelligence to and simplify our household tools and

appliances are handled quite effectively by todays computers. In contrast cognitive taskslike Speech and Image processing are hard to solve by the conventional algorithmic

approach. Whereas, human beings are typically much better at perceiving and identifying

an object of interest in a natural scene, or interpreting natural language, and many othernatural cognitive tasks, than a digital computer. One reason why we are much better at

recognizing objects in a complex scene, for example, is due to the way that our brain isorganized. Our brain employs a computational architecture that is well suited to solve

complex problems that a digital computer would have a difficult time with. Since ourconventional computers are obviously not suited to this type of problem, we borrow the

features from physiology of the brain as the basis for our new computing models, and this

technology has come to be known as Artificial Neural Network (ANN). This chaptergives an overview of the fundamental principles of artificial neural network.

1.2 Biological Neural Network

The brain is the central element of the nervous system. The informationprocessing cells of the brain are the neurons. Figure 1.1 shows the structure of a

biological neuron. It is composed of a cell body, or soma and two types of out-reaching

tree-like branches: the axon and the dendrites. The cell body has a nucleus that containsinformation about hereditary traits and a plasma that holds the molecular equipment for

producing material needed by the neuron. A neuron receives signals from other neurons

through its dendrites. (receivers) and transmits signals generated by its cell body along

the axon (transmitter), which eventually branches into strands and substrands. At theterminals of these strands are the synapses. A synapse is an elementary structure and a

fundamental unit between two neurons. The synapses effectiveness can be adjusted by

the signals passing through it so that the synapses can learn from the activities in whichthey participate.

Figure 1.1 A biological Neuron


2/27



It is estimated that the brain consists of about 1011

neurons which are interconnected to

form a vast and complex neural network.

1.3 Artificial Neural Network?

An Artificial Neural Network is an information-processing paradigm that is inspired bythe way biological nervous systems, such as the brain, process the information. In

common with biological neural networks, ANNs can accommodate many inputs in

parallel and encode the information in a distributed fashion. Typically the informationthat is stored in a artificial net is shared by many of its processing units that are analogous

to neurons in the brain. Table 1.1 shows how the biological neural net is associated with

the artificial neural net

Table 1.1 Associated Terminologies of Biological and Artificial Neural Net

Biological Neural Network Artificial Neural Network

Cell Body (Soma)Synapse

Dendrites

Axon

NeuronsWeights

Input

Output

Unlike its programmed computing counterpart, neuro computing approach to information

processing first involves a learning process with an artificial neural network architecture

that adaptively responds to inputs according to a learning rule. After the neural network

has learned what it needs to know, the trained network can be used to perform certaintasks depending on the particular application. The ability to learn by example and

generalize are the principal characteristics of artificial neural network. The learning

process is what sets a neural network apart from processing information on a digitalcomputer that must be programmed.

1.4 Computational Model of Artificial Neuron

In 1943, Mc Culloh and Pitts proposed a binary threshold unit as a computational model

for an artificial neuron. Figure 1.2 shows the fundamental representation of an artificialneuron. Each node has a node function, associated with it which along with a set of local

parameters determines the output of the node, given an input. The input signals are

transmitted by means of connection links. The links possesses an associated weight,

which is multiplied along with the incoming signal (net input) for any typical neural net.

Positive weights correspond to excitatory synapses, while negative weights modelinhibitory ones. This mathematical neuron computes a weighted sum of its n input signals

xi, i = 1, 2n given by

=

=+++=n

i

nnnn wxwxwxwxnet1

2211 ...

If this sum exceeds the neurons threshold value then the neuron fires (the output is ON).This is illustrated in Figure 1.3.


3/27



The output of the neuron can be expressed by:

( )

=

=

n

i

ii xwfy1

Where f(.) is a step function defined by

=>=

netnetnetf

,0,1)(

The bias or offset could be introduced as one of the inputs with a permanent weight of

1, leading to a simpler expression:

( )

=

=

n

i

ii xwfy0

The lower limit in the above equation is from 0 rather than 1. The value of input x0 is

always set to 1.

Figure 1.3 Thersholding Function

The other choices for activation function besides thersholding function are given inFigure 1.4.

net

Output

net

a) Thersholding at 0

Outut

b) Thersholding at

yh

wn

w2

xn

x2

x1w1

Figure 1.2 Basic Artificial Neuron


4/27



Figure 1.4 Common non-linear activation functions

Networks consisting of MP neurons with binary (on-off) output signals can be configured

to perform several logical functions. Figure 1.5 shows some examples of logic circuitsrealized using the MP model.

Figure 1.5 Logic Networks using MP Neurons


5/27


6/27



Unsupervised learning or Self-organization in which an (output) unit is trainedto respond to clusters of pattern within the input. In this paradigm the system is

supposed to discover statistically salient features of the input population. Unlikethe supervised learning paradigm, there is no a priori set of categories into which

the patterns are to be classified; rather the system must develop its own

representation of the input stimuli.

Reinforcement learning may be considered as an intermediate form of the abovetwo types of learning. Here the learning machine does some action on the

environment and gets a feedback response from the environment. The learning

system grades its action good (rewarding) or bad (punishable) based on theenvironmental response and accordingly adjusts its parameters. Generally,

parameter adjustment is continued until an equilibrium state occurs, followingwhich there will be no more changes in its parameters. The self organizing neural

learning may be categorized under this type of learning.


7/27



CHAPTER 2

FEED FORWARD NEURAL NETWORK

2.1 Introduction

In chapter 1, the mathematical details of a neuron at the single cell level and as a network

were described. Although single neurons can perform certain simple pattern detection

functions, the power of neural computation comes from the neurons connected in anetwork structure. Larger networks generally offer greater computational capabilities but

the converse is not true. Arranging neurons in layers or stages is supposed to mimic the

layered structure of certain portions of the brain.In this chapter, we describe the most popular Artificial Neural Network (ANN)

architecture, namely Feedforward (FF) network. First we briefly review the perceptron

model.Next a multilayer feedforward network architecture is presented.This type ofnetwork is sometimes called multilayer perceptron because of its similarity to perceptron

networks with more than one layer. We derive the generalized delta (backpropagation)learning rule and see how it is implemented in practice. We will also examine variations

in the learning process to improve the efficiency, and ways to avoid some potentialproblems that can arise during training. We will also discuss optimal parameters settings

and discuss various other training methods. We will also look at the capabilities and

limitations of the ANN.

2.2 Perceptron Model

A two layer feedforward neural network with hard limiting threshold units in the output

layer is called as single layer perceptron model. A perceptron model consisting of twoinput units and three output units is shown in Figure 2.1. The number of units in the

input layer corresponds to the dimensionality of the input pattern vectors. The units in the

input layer are linear as the input layer merely contributes to fan out the input to each ofthe output units. The number of output units depends on the number of distinct classes in

the pattern classification task.

The perceptron network is trained with the help of perceptron algorithm which is

a supervised learning paradigm. The network is given a desired output for each inputpattern. During the learning process, the actual output yi generated by the network may

not equal the desired output di. The perceptron learning rule is based on error-correction

principle. The basic principle of error correction learning rule is to use the error signal(di-yi) to modify the connection weights to gradually reduce this error.


8/27



Figure 2.1 Single layer Perceptron

The algorithm for weight adjustment using perceptron learning law is given below:

Initialize weights and thresholds by assigning random values.Present a training pattern i.e., an input and the desired output.Calculate the actual output: ( ) ( )

=

=

n

i

ii txtwfy0

.

Adjust weights:If correct, then ( ) ( )twtw ii =+ )1

If output 0 should be 1 (class A), then ( ) ( ) ( )txtwtw iii +=+1If output 1 should be 0 (class B), then ( ) ( ) ( )txtwtw iii =+1

Where 10 is a constant that controls the adaptation rate.

Figure 2.2 Supervised Learning

w22

w21

w12

w23

w13

w11

X2

X1

Y1

Y2

Y3

Input layer Output layer


9/27



2.3 Geometrical interpretation of perceptron network output

A pattern classification problem using perceptron model can be viewed as determining

the hyper surfaces separating the multidimensional patterns belonging to different

classes. With linear threshold units on the output, the hyper surfaces reduce to straightlines. The network can produce N distinct lines depending on the number of output units

in the pattern space as shown in Figure 2.2. These lines can be used to separate different

classes, provided the regions formed by the pattern classification problem are linearlyseparable.

Figure 2.2 Linearly separable classes

There are certain restrictions on the class of problems for which perceptron model can be

used. Perceptron network can be used only if the patterns are linearly separable. Because

many classification problems do not possess linearly separable property, this condition

places a severe restriction on the applicability of the perceptron network. A feed forwardnetwork with hidden layer is an obvious choice in this case, the details of which are given

in the next section.

Problem 1: Train the perceptron network using the training vector

=

=

=

1

5.0

1

1

;

1

5.0

5.1

0

;

1

0

2

1

321 xxx

The desired response are d1 = -1, d2 = -1 and d3 = 1 respectively.

Take 1.0=

Take the initial weight vector as

=

5.0

0

1

1

'w

Step 1:

x1

x2


10/27



net = 2.5; w2

= [.8 -0.6 0 0.7]

Step 2:

net = -1.6; No Correction

Step 3:net = -2.1; w

3= [.6 -0.4 0.1 0.5]

2.4 Multilayer Feed Forward Neural Network

Figure 2.3 shows the structure of multilayer feed forward neural network. This type ofarchitecture is part of a large class of feed forward neural networks with the neurons

arranged in cascaded layers. The neural network architecture in this class share a

common feature that all neurons in a layer (or sometimes called a slab) are connected to

all neurons in adjacent layers through unidirectional branches. That is, the branches andlinks can only broadcast information in one direction, that is, the forward direction. The

branches have associated transmittances, that is, synaptic weights that can be adjusted

according to a defined learning rule. Feed forward networks do not allow connectionsbetween neurons within any layer of architecture. At every neuron, the output of the

linear combiner, that is, the neuron activity level is input to a non linear activation

function f (.), whose output is the response of the neuron.The neurons in the network typically have activity levels in the range [-1, 1], and

in some applications the range [0, 1] is used. In Figure 2.3, actually there are three

layers; The first layer in the network does not perform any computations, but only serves

to feed the input signal to the neurons of the second layer (called the hidden layer),

whose outputs are then input to the third layer (or the output layer). The output of theoutput layer is the network response factor. This network can perform the non linear

input/output mapping. In general there can be any number of hidden layers in thearchitecture; however, from a practical perspective, only one or two hidden layers are

typically used. In fact, it can be shown that a Multi Layer Perceptron (MLP) that has only

one hidden layer, with a sufficient number of neurons, act as a universal approximator ofnon-linear mapping.


11/27



Figure 2.3 A typical multi layer feed forward network architecture

2.5 Back Propagation Learning Algorithm

Back Propagation learning is the commonly used algorithm for training the MLP. It is agradient descent method minimizing the mean square error between the actual and target

output of a multi layer perceptron.

Let (xp, dp) be the training data pair

In Iteration , k

( )

( )mj

enetfy

mjwznet

Hhe

netfz

wxnet

kj

kh

net

k

j

k

j

H

h

k

jh

k

h

k

j

net

k

h

k

h

n

i

k

Hhhii

k

h

,....,2,1;1

1)(

,...,2,1;

,....,2,1;1

1)(

0

0

,....,2,1;

=+

==

==

=+

==

=

=

==

Error Function

( )2

12

1

==

m

j

k

jj

kydE

Adjust the weights such that the error is minimized

w42

w41

w32

w31

w21

w22

w12

w11

w33

w32w13

w24

w23

w22

w12

w14

w13

w12

w34

w11

x3

x2

x1

Hidden layer (h) Output layer (j)

h1

Input layer (i)

h2

h3

h4

y1

y2


12/27



kjh

kkjh

kjh

kjh

kjh

w

Ew

www

=

+=+

1

( ) ( )

k

h

k

j

k

h

k

j

k

j

k

jk

jh

k

z

znetfydw

E

=

=

.'.

Here ( ) ( )kjkjkjkj netfyd '.= represents the error scaled by the slope

Input to hidden layer weight

( )k

hi

k

h

k

h

k

h

k

h

k

k

hi

k

k

hi

kk

hi

k

hi

k

hi

k

hi

w

net

net

z

z

E

w

E

w

Ew

www

=

=

+=+

)1(

)('.

)('

1

1

k

h

m

j

k

jh

k

j

k

h

i

k

h

i

k

h

m

j

k

jh

k

jk

h

k

netfwHere

x

xnetfwz

E

=

=

=

=

=

Weight update equations

Hidden to output layer

kh

kj

kjh

kjh

kjh

kjh

zw

www

+=

+=+1

Input to hidden layer

ikh

khi

khi

khi

khi

xw

www

+=

+=+1


13/27



The back propagation algorithm consists of the following steps:

Step 1. Initialize weights and offsets

Initialize all weights and node offsets to small random values.

Step 2. Present Input and Desired Output vector

Present continuous input vectorx and specify the desired output d. Output vector

elements are set to zero values except for that corresponding to the class of thecurrent input.

Step 3. Calculate actual outputs

Calculate the actual output vectory using the sigmoidal nonlinearity

( )neti

enetf

=

1

1

Step 4. Adapt weights

Adjust weights by ( ) ( ) '1 ijijij xtwtw +=+

where 'ix is the output of the node i, is the learning rate constant and jis the

sensitivity of the node j. If node j is an output node, then

( )( )jjjj ydnetf = ' where dj is the desired output of the node j, y j is the actual output and ( )jnetf' isthe derivation of the activation function calculated at netj. If the node j is aninternal node, then the sensitivity is defined as

( )=k

jkkjj wnetf '

where k sums over all nodes in the layer above the node j. Update equations are

derived using the chain derivation rule applied to the LMS training criterion

function.

Step 5. Repeat by going to step 2

Training the network with back propagation algorithm results in a non-linear

mapping between the input and output variables. Thus, given the input/output pairs, the

network can have its weights adjusted by the back propagation algorithm to capture thenon-linear relationship.

After training, the networks with fixed weights can provide the output for the

given input. Once trained, the network can be used as a classifier model for anyengineering applications.

2.6 Design Issues

This section discusses the various design issues that concern the inner workings of the

back propagation algorithm.

2.6.1 Pattern or Batch Mode Training


14/27



The Back Propagation algorithm operates by sequentially presenting the data drawn from

a training set to predefined network architecture. There are two choices in implementingthis algorithm. Once a data is presented to the network, its gradients are calculated and

proceed to change the network weights based on these instantaneous (local) gradient

values. This is called pattern mode training. Alternatively, one could collect the error

gradients over an entire epoch and then change the weights of the initial neural networkin one shot. This is called batch mode training.

2.6.2 Selection of Network Structure

Both the generalization and approximation ability of a feed forward neural network are

closely related to the architecture of the network and the size of the training set. Choosingappropriate network architecture means the number of layers and the number of hidden

neurons per layer. Although the back propagation algorithm can be applied to any

number of hidden layers, a three-layered network can approximate any continuousfunction. The problem of selecting the number of neurons in the hidden layers of

multilayer networks is an important issue. The number of nodes must be large enough toform a decision region as complex as that required by the problem. And yet the number

must not be excessively large so that the weights cannot be reliably estimated byavailable training data.

In general cross validation approach is used to select appropriate networkarchitecture. The operational procedure of cross validation approach is given below:

Divide the data set into a training set Ttraining and a test set Ttest.Subdivide Ttraining into two subsets: one to train the networkTlearning, and

one to validate the networkTvalidation.

Train different network architectures on Tlearning and evaluate theirperformance on Tvalidation.Select the best network.Finally, retrain this network architecture on Ttraining.Test for generalization ability using Ttest.

2.6.3 Weight Initialization

It is important to correctly choose a set of initial weights for the network. Sometimes it

can decide whether or not the network is able to learn the training set function. It iscommon practice to initialize weights to small random values within some

interval ],[ . Initialization of weights of the entire network to the same value can leadto network paralysis where the network learns nothing-weight changes are uniformly

zero. Further, very small ranges of weight randomization should be avoided in general

since this may lead to very slow learning in the initial stages. Alternatively, an incorrect

choice of weights might lead to network saturation where weight changes are almostnegligible over consecutive epochs.


15/27



To get the best result the initial weights (and biases) are set to random numbers

between -0.5 and 0.5 or between -1 and 1. In general, the initialization of weights (bias)can be done randomly.

2.6.4 Termination criteria for Training

The motivation for applying back propagation net is to achieve a balance between

memorization and generalization; it is not necessarily advantages to continue training

until the error reaches a minimum value. The weight adjustments are based on thetraining patterns. As along as the error for validation decreases training continues.

Whenever the error begins to increase, the net is starting to memorize the training

patterns. At this point training is terminated.

2.7 Applications of Feed Forward Neural Networks

Multi layered feed forward neural networks trained using back propagation algorithm

account for a majority of applications of real world problems. This is because backpropagation algorithm is easy to implement, fast and efficient to operate. Some of the

applications of ANN are mentioned below.

Speech recognitionData MiningRobot arm controlBio-InformaticsPower system security assessmentLoad forecasting

Image processing

References

1.B.Yegnanarayana, Artificial Neural Networks, Prentice-hall of India Pvt Ltd,1999.

2.Fredric M.Ham Ivica Kostanic, Principles of Neurocomputing for Science andEngineering, McGraw-Hill Higher Education, 2001.

3.James A.Freeman / David M.Skapura, Neural Networks Algorithms, Applicationsand Programming Techniques, Pearson Education, 1991.


16/27



CHAPTER 3

WORKING WITH MATLAB

3.1 Introduction to MATLAB

MATLAB stands for MATrix LABortary. It is a software package for high performance

numerical computation and visualization. It also provides an interactive environment withhundreds of reliable and accurate built-in mathematical functions. MATLABs built-in

functions provide excellent tools for linear algebraic computations, data analysis, signal

processing, optimization, numerical solution of ordinary differential equation and manyother types of scientific computations. The basic building block of MATLAB is the

matrix. The fundamental data type is the array. Vectors, Scalars, real matrices and

complex matrices are all handled as special cases of the basic data type.Dimensioning of a matrix is automatic in MATLAB. MATLAB is case-sensitive:

Most of the MATLAB commands and built-in functions are in lower case letters. Theoutput of every command is displayed on the screen unless MATLAB is directed

otherwise. A semi-colon at the end of a command suppresses the screen output, exceptfor graphics and on-line help commands.

3.2 MATLAB Windows

MATLAB works through the following three basic windows:1.Command Window2.Graphics Window3.Edit Window

3.2.1 Command Window

This is the main window, which is characterized by the MATLAB command prompt,

>>. All commands, including those for running user-written programs, are typed in this

window at the MATLAB command prompt. It consists of four other sub-windows.

Launch Pad:This sub-window lists all MATLAB related applications and toolboxes.

Work Space:This sub-window lists all variables that have been generated so far and shows

their type and size. Various operations can be performed on these variables such

as plotting.Command History:All commands typed on the MATLAB command prompt get recorded, even

across multiple sessions in this window.

Current Directory:This is the sub-window, in which all files from the current directory are listed.


17/27



3.2.2 Graphics Window

The output of all graphics commands typed in the command window are flushed to the

graphics or figure window, a separate gray window with white background color. The

user can create as many figure windows as the system memory will allow.

3.2.3 Edit Window

In this sub-window, programs can be written, edited, created and saved in files calledM-files. MATLAB provides its own built-in editor.

3.3 Working with Script Files

A script file is a user-created file with a sequence of MATLAB commands in it. It may be

created by selecting a new M-file from the file menu in edit window. The file must besaved with a .m extension to its name, thereby, making it an M-file. This file is

executed by typing its name, without extension at the command prompt in commandwindow. If we use % symbol before a line in the MATLAB program then it will be

treated as command line.Eg. % MATLAB is user friendly program

The character limit for a line in MATLAB program is 4096.

3.4 Working with Directories

The following are some important commands for working with directories:

pwd (print working directory)This command opens a separate sub-window, to the left of the command window,

and also displays the current directory.Eg: >>pwd

C:\matlabR12\work

Displays the present working directory.

cd (change directory)This command is used to change the current working directory to a new directory.

Eg: >>cd new directory

dir (directory)On execution of this command, the contents present in the current directory can

be viewed.

addpath

This command is used to add the specified directories to the existing path.Eg: >>addpath D:\matlabR12\work

>>addpath C:\mywork

(or)>>addpath D:\matlabR12\work C:\mywork


18/27



3.5 Variables

Expressions typed without a variable name are evaluated by MATLAB, and the result is

stored and displayed by a variable, ans. The result can also be stored to a variable name.

The variable name should begin with an alphabet and it can have a maximum wordlength of 31 characters. After the first letter any symbols, numbers (or) special characters

may be used. Variables used in script files are global. Declaration of a set of variables to

be globally accessible to all or some functions without passing the variables in the inputlist.

Eg: >>global x y;

An M-file can prompt for input from the keyboard by using input command.Eg: >>V=input(enter radius)

displays the string - enter radius - and waits for a number to be entered. That number

will be assigned to the variable, V.

3.6 Matrix and vectors

A matrix is entered row-wise with consecutive elements of a row separated by a space orby comma, and the rows are separated by semi-colons, The entire matrix must be

enclosed within square brackets. Elements of the matrix may be real numbers, complex

numbers or valid MATLAB expressions.Eg: >>A=[1 2 3 ; 5 6 7; 10 11 12];

Displays, A =

093

521

A Vector is a special case of a matrix, with just one row or one column. It is entered the

same way as a matrix.

Eg: >> V = [1 2 3 ] produces a row vector.>> U = [1;2;3] produces a column vector.

To create a vector of numbers over a given range with a specified increment.The general command to do this in MATLAB is,

1.V=Initial value : Increment : Final valueThe three values in the above assignment can also be valid MATLABexpressions. If no increment is specified, MATLAB uses the default

increment of 1.

Eg: >>a = 0:10:100 produces a = [0 10 20 .100]

2.linspace(a,b,n)This generates a linearly spaced vector of length n from a to b.

Eg: >>u = linspace(0,100,200) generates u = [0 20 40 100]3.logspace(a,b,n)

This generates a logarithmically spaced vector of length n from 10a

to 10b

Eg: >>v = logspace(0,3,4) produces v =[1 10 100 1000]

Special vectors, such as vectors of zeros or ones of a specific length, can be created withthe utility matrix functions zeros, ones, etc.

Eg: >>p = zeros(1,2) initializes a two element long zero row vector.

>>m = ones(10,1) creates a 10 element long column vector of 1s.


19/27



The following are some of the commands used in matrix and vector manipulation:

Square brackets with no elements between them creates a null matrix.Eg: >>X = [ ] produces a null vector

Any row(s) or column(s) of a matrix can be deleted by setting the row orcolumn to a null vector.

Eg: >>A(2,:) = [ ] deletes the second row of the matrix A.It provides a much higher level of index specifications. It allows a range of

rows and columns to be specified at the same time.

Eg: >>B=A(2:4,1:4) creates a matrix B consists of the elements in 2 to4 rows and 1 to 4 columns from A matrix.

When a row (or column) to be specified range over all rows (or columns) ofthe matrix, a colon alone can be used.

Eg: >>B=A(2:3,:) creates a matrix B consists of all elements in 2rd

and3

rdrow of A matrix.

No Dimension declaration required normally, but if the matrix is large then itis advisable to initialize it

Eg: >>A=zeros(m,n); creates or initializes a matrix A of zeros aselements with a dimension of m x n.

The size of the matrix can be determined by the command size(matrix name)in the command window.

Eg: >>a=size(A);

If we want to know about the size of the row and column in a matrix then typeEg: >>[m,n] = size(matrix name)

Transpose of a matrixEg: >>B = A

Appending a row or a column in a matrix>>A=[A ; u] appends a column

>>A=[v ; A] appends a rowwhere, u and v represents the column or row to be added.

Deleting a row or a column in a matrix any row or column in a matrix canbe

deleted by setting it to a null vector.

>>U (: ; a) = [ ] will delete the ath

column from the U matrix.>>V (b ; :) = [ ] will delete the b

throw from the V matrix.

Performing arithmetic operation in a matrix and in an arrayEg: >> A=[ 1 2: 3 4]

>>A=A*A;

For Array Operation:>>A=A.*A


20/27



3.7 Plotting a graph

The following are the commands used in plotting graphs or figure:

>>plot(x,y) /* For plotting a graph

>>xlabel(x) To label the axis

>>ylabel(y) To label the axis>>Title(Title name) To set the title for the graph

>>axis(equal) To get a circle, we have to set the length scales of the axis to

be the same.

3.8 Loops, Branches and Control flow

3.8.1 For loop

It is used to repeat a statement or a group of statements for a fixed no of times

Format:

1. for i = m:n

..

endWhere,

m initial value

n final value2. for i = initial value: increment: final value

.

end

3.8.2 While Loop

It is used to execute a statement or a group of statements for an indefinite no of timesuntil the condition specified by while is no longer satisfied

Format:

While n>i=6;

>j=21;

>>if i>5k=i;


21/27



elseif (i>1) & (j==20)

k=5*i+j;else

k=1;

end

3.8.4 Switch case otherwise

This construction provides a logical branching for computations. A flag is used as switchand the values of the flag make up the different cases for execution.

Eg: color = input(color =,s);

switch colorcase red

c=[1 0 0]

case bluec=[1 1 1]

otherwiseerror(Invalid color)

end

3.8.5 Error

The command error inside a function or a script aborts the execution, displays the error

message and returns the control to the keyboard.

3.8.6BreakThe command break insider the for loop or while loop terminates the execution of the

loop, even if the condition for execution of the loop is true

Eg: for i=1:10if u(i)


22/27


23/27



CHAPTER 4

MATLAB NEURAL NETWORK TOOLBOX

4.1 Introduction

The neural network Toolbox is a collection of predefined functions built on the

MATLAB numerical computing environment. These predefined functions can be calledby the user to simulate various types of neural network models. Artificial neural network

technology is being used to solve a wide variety of complex scientific, engineering and

business problems. Neural networks are ideally suited for such problems because liketheir biological counterparts, and artificial neural network can learn, and therefore can be

trained to find solutions, recognize patterns, classify data, and forecast future events.


24/27



Because neural networks require intensive matrix computations, MATLAB provides a

natural framework for rapidly implementing them and for their behavior and application.

4.2 Basic Commands in Neural Network Toolbox

HARDLIM - Hard limit transfer function.A = hardlim(N)

info = hardlim(code)HARDLIM is a transfer function. Transfer functions calculate a layer's output from its net

input.HARDLIM(N) takes one input,N - SxQ matrix of net input (column) vectors and returns 1

where N is positive, 0 elsewhere.HARDLIM(CODE) returns useful information for each CODE string:

'deriv' - Name of derivative function.

'name' - Full name.'output' - Output range.

'active' - Active input range.

HARDLIMS - Symmetric hard limit transfer function.

A = hardlims(N)

info = hardlims(code)HARDLIMS(N) takes one input,N - SxQ matrix of net input (column) vectors and returns1 where N is positive, -1 elsewhere.

PURELIN - Linear transfer function.A = purelin(N)

info = purelin(code)PURELIN calculate a layer's output from its net input.PURELIN(N) takes one input,N - SxQ matrix of net input (column) vectors and returns N.PURELIN(CODE) returns useful information for each CODE string:


25/27



'deriv' - Returns name of derivative function.'name' - Returns full name.

'output' - Returns output range.

'active' - Returns active input range.

LOGSIG - Logarithmic sigmoid transfer function.

A = logsig(N)info = logsig(code)LOGSIG calculate a layer's output from its net input.LOGSIG(N) takes one input,N - SxQ matrix of net input (column) vectors and returns

each element of N squashed between 0 and 1.LOGSIG(CODE) returns useful information for each CODE string:

'deriv' - Returns name of derivative function.'name' - Returns full name.

'output' - Returns output range.'active' - Returns active input range.

TANSIG Hyperbolic tangent sigmoid transfer function.

A = tansig(N)info = tansig(code)

TANSIG calculate a layer's output from its net input.

TANSIG(N) takes one input,

N - SxQ matrix of net input (column) vectors and returns each element of N squashedbetween -1 and 1

TANSIG(CODE) returns useful information for each CODE string:'deriv' - Returns name of derivative function.

'name' - Returns full name.


SATLIN Saturating linear transfer function.A = satlin(N)

info = satlin(code)

SATLIN calculate a layer's output from its net input.SATLIN(N) takes one input,

N - SxQ Matrix of net input (column) vectors and returns values of N truncated into the

interval [-1, 1].

SATLIN(CODE) return useful information for each CODE string:'deriv' - Returns name of derivative function.

'name' - Returns full name.


.

.


26/27



Design and Simulation Using MATLAB Neural Network Toolbox

Generate training and test data setLoad the data using the command load and store it in a variable

(eg.) >> load PR.dat>> XY=PR

Separate the input and output data from the variable XY

where

X be the variable to store the input data & Y be the variable

to store the output data and store them separately

Separate the input and output data into training and test casesX1,Y1 for trainingX2,Y2 for testing

Normalize the input / output data if they are in different ranges using the formula

Normalized value of the ith

pattern in jth

variable is given byX1n(i,j)=(X1(i,j)-minX(j))/(maxX(j)-minX(j));

Similarly normalized value of the ith

pattern in the jth

output variableY1n(i,j)=(Y1(i,j)-minY(j))/(maxY(j)-minY(j));

Define the input structure using the command

net=newff(PR,[nhid nout],{transig purelin}, trainscg)

Where,

newff creates a feed forward network to create the network objecttansig implies tan sigmoid transfer function

purelin implies linear transfer function

trainscg uses the conjugate gradient algorithm to train thenetwork

Initialize the weights and biases using the command

rand (seed,0);net.layers{1}.initFcn='initwb';

net.inputWeights{1,1}.initFcn='rands';

net .biases{1,1}.initFcn=rands;

net .biases{2,1}.initFcn=rands;net=init(net);

Specify the number of hidden layer and number of output layer using the

assignment

nhid =---nout= ---


27/27


Specify the number of epochs and the error goal using the assignmentnet.trainParam.epochs=----;

net.trainParam.goal=-----;

Train the network using the command[net TR]=train(net Xn,Yn)

Compare the network output and actual outputRe normalize the output values

Date post:	08-Apr-2018
Category:	Documents
Upload:	benita0208
View:	255 times
Download:	0 times

ANN_Material

Documents