Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | darcy-palmer |
View: | 218 times |
Download: | 1 times |
Artificial Neural Network Theory and ApplicationAshish Venugopal
Sriram Gollapalli
Ulas Bardak
Presentation Overview
Artificial Neural Networks NASA profiles as a Pattern
Classification Task Implementing Pattern Classification
on the Silicon Recognition Neuron Board
ANN General Overview
Inspired by biological neuron modelsDecision are distributed throughout
systemComponents interacts at several stagesComponents work in unison to solve
one problem ANN well suited to problems that don’t
have closed form algorithmic solution Black box model – hard to interpret
ANNs Applied
Driving a car. Task involves determining the direction the road is curving based on an imageImage pixels are inputCurve angle is outputLearned non-linear relationship
between pixels and roads• hard to understand the model
Theory
Input layer, Hidden layer, Output LayerInput – one “neuron” for each pixelHidden – set of neurons to store what
was learned.Output layer – one neuron for each of 5
turn directions – ie straight, slight left, hard right..
But what is a neuron?
Neurons
Math terms: Binary decision makers English terms: take in some input, decide
whether is “high” or “low”, and pass this value on.
Similar to neurons in the brain propagating signals.
How is the model learned?
Once you have training examplesPair (feature set, classification)Run the back propagation algorithms
Learns connection weights between neurons, as well as within neuron transfer weights.
Network topology changes affects learning behavior!
Unsupervised Networks
So far we have used labeled training examples to learn relationship between features and response variable.
What if we are just exploring patterns that exist in the features?
Unsupervised learning / clustering…
Unsupervised Goal
Find prototypical points within data set that can be used to approximate distribution of the whole data set.
Compression / Visualization Typical methods
Define a measure of similarity between data points
Select initial prototypes Move prototypes to best fit the data –
minimize error
Kohonen Net
Unsupervised technique
Each output neuron represents a prototype point, input layer is presented with data
Self Organizing Map
Each prototype is pulled towards that data, and brings its closest prototypes a little closer as well…
SOM Demo
How is it relevant to our problem?
We will go through:Definition of what we are facingHow we can use neural netsHow we can improve on this method
LIDAR in Space
Data from LITE projectSend a laser beam down from a
shuttle and get reflection readings.Data gathered for 53 hours in 1994.40GB of data
Input
Problem
There is too much data to send and processEach pixel is a number to be
transmitted.Each reading (one column) contains
3000 pixels. Need a way to transmit the
information in a more compact fashion.
Applying Neural Nets
Some readings are very similar to each other.
Define classes that will contain multiple readings.
Define a representative for the class which is close enough to all the class members.
Applying Neural Nets (cont.)
In order to get class Train Kohonen Nets on data for
specific number of classes.Once trained, just pass in a new
reading (3000 data points) and get the class id. Just transfer the class id.
Go from 3000x3000 numbers to be transferred to 3000 numbers
Example
Result – 8 classes
Result – 16 Classes
Result – 64 Classes
Result – 128 Classes
Result – 256 Classes
Problems
Takes a while to train 256 classes took 44 hours on a dual
P3 1GHz computer with 1.5GB of RAM.
Not optimalClassifying by the complete reading is
wasteful.• Need one number for each reading
Improving NN
sky
Improving NN (cont.)
Easier version:Look at blocks of 50x50 pixels instead
of just 1 pixel.Classify as sky, etc.Can preprocess using kNN, basic
filters to get rid of noise in data.Classes can be formed from
sequence of 50 pixel blocks.
Improving NN (cont.)
Harder version:Use variable sized mesh
• In parts with just the sky use a big block.• In detailed parts use smaller blocks.
Something to think about…
Muren Board
Silicon Recognition, Inc. 2 ZISC078 Chips (156 neurons in
parallel) 1 Mb of memory Up to 1 million recognitions/second
ZISC
ZISC, the cornerstone of the Muren board, completely overcomes the three major limitations of software-based pattern recognition applications: ZISC fully handles non-linearity and
fuzziness ZISC is a fully massively parallel processing
system. Recognition speed is consistent, regardless of the number of stored patterns
ZISC does not use any algorithms for programming the solution, ZISC is taught
RBF Space mapping
Example
Source picture
(circuit board)
Result (after processing)
(shows good and bad points)
Process
Learning Classifying Training Testing with new data
NASA Data
Further investigation
Can we write directly to the board? Ability to preprocess data?
Questions?