Artificial Neural Network Theory and ApplicationAshish Venugopal

Sriram Gollapalli

Ulas Bardak

Presentation Overview

Artificial Neural Networks NASA profiles as a Pattern

Classification Task Implementing Pattern Classification

on the Silicon Recognition Neuron Board

ANN General Overview

Inspired by biological neuron modelsDecision are distributed throughout

systemComponents interacts at several stagesComponents work in unison to solve

one problem ANN well suited to problems that don’t

have closed form algorithmic solution Black box model – hard to interpret

ANNs Applied

Driving a car. Task involves determining the direction the road is curving based on an imageImage pixels are inputCurve angle is outputLearned non-linear relationship

between pixels and roads• hard to understand the model

Theory

Input layer, Hidden layer, Output LayerInput – one “neuron” for each pixelHidden – set of neurons to store what

was learned.Output layer – one neuron for each of 5

turn directions – ie straight, slight left, hard right..

But what is a neuron?

Neurons

Math terms: Binary decision makers English terms: take in some input, decide

whether is “high” or “low”, and pass this value on.

Similar to neurons in the brain propagating signals.

How is the model learned?

Once you have training examplesPair (feature set, classification)Run the back propagation algorithms

Learns connection weights between neurons, as well as within neuron transfer weights.

Network topology changes affects learning behavior!

Unsupervised Networks

So far we have used labeled training examples to learn relationship between features and response variable.

What if we are just exploring patterns that exist in the features?

Unsupervised learning / clustering…

Unsupervised Goal

Find prototypical points within data set that can be used to approximate distribution of the whole data set.

Compression / Visualization Typical methods

Define a measure of similarity between data points

Select initial prototypes Move prototypes to best fit the data –

minimize error

Kohonen Net

Unsupervised technique

Each output neuron represents a prototype point, input layer is presented with data

Self Organizing Map

Each prototype is pulled towards that data, and brings its closest prototypes a little closer as well…

SOM Demo

How is it relevant to our problem?

We will go through:Definition of what we are facingHow we can use neural netsHow we can improve on this method

LIDAR in Space

Data from LITE projectSend a laser beam down from a

shuttle and get reflection readings.Data gathered for 53 hours in 1994.40GB of data

Input

Problem

There is too much data to send and processEach pixel is a number to be

transmitted.Each reading (one column) contains

3000 pixels. Need a way to transmit the

information in a more compact fashion.

Applying Neural Nets

Some readings are very similar to each other.

Define classes that will contain multiple readings.

Define a representative for the class which is close enough to all the class members.

Applying Neural Nets (cont.)

In order to get class Train Kohonen Nets on data for

specific number of classes.Once trained, just pass in a new

reading (3000 data points) and get the class id. Just transfer the class id.

Go from 3000x3000 numbers to be transferred to 3000 numbers

Example

Result – 8 classes

Result – 16 Classes

Result – 64 Classes

Result – 128 Classes

Result – 256 Classes

Problems

Takes a while to train 256 classes took 44 hours on a dual

P3 1GHz computer with 1.5GB of RAM.

Not optimalClassifying by the complete reading is

wasteful.• Need one number for each reading

Improving NN

sky

Improving NN (cont.)

Easier version:Look at blocks of 50x50 pixels instead

of just 1 pixel.Classify as sky, etc.Can preprocess using kNN, basic

filters to get rid of noise in data.Classes can be formed from

sequence of 50 pixel blocks.

Improving NN (cont.)

Harder version:Use variable sized mesh

• In parts with just the sky use a big block.• In detailed parts use smaller blocks.

Something to think about…

Muren Board

Silicon Recognition, Inc. 2 ZISC078 Chips (156 neurons in

parallel) 1 Mb of memory Up to 1 million recognitions/second

ZISC

ZISC, the cornerstone of the Muren board, completely overcomes the three major limitations of software-based pattern recognition applications: ZISC fully handles non-linearity and

fuzziness ZISC is a fully massively parallel processing

system. Recognition speed is consistent, regardless of the number of stored patterns

ZISC does not use any algorithms for programming the solution, ZISC is taught

RBF Space mapping

Example

Source picture

(circuit board)

Result (after processing)

(shows good and bad points)

Process

Learning Classifying Training Testing with new data

NASA Data

Further investigation

Can we write directly to the board? Ability to preprocess data?

Questions?