NEURAL NETWORKS

& Machine Learning

Justin ChowLevon Mkrtchyan

Eric Su Senior Project

5/16/07

What are Neural Nets?

– Refers to a computing paradigm that is modeled after the structure of the brain.

– Inspired by examination of the central nervous system and the neurons

• In Neuroscience, refers to physically collected neurons in our brains.

– Is a network because the function f(x) executed by a node is a composition of other functions, which are in turn defined as compositions of other function.

What is Machine Learning?

– Learning – Given a task to solve and a class of functions F, learning means using a set of observations to find an optimal solution that is an element of F

– Requires a cost function to determine how close we are to the optimal solution.

• Learning Paradigms– Supervised learning – Unsupervised learning – Reinforcement learning

– Training employs many cutting edge mathematical theories

– The NN has a learning algorithm, which you train with thousands of examples.

Relations to A.I.

– Marvin Minsky, one of the founding fathers of A.I., built first neural network learning machine and wrote Perceptrons, foundational work of artificial neural networks.

- Neural network is one of the main methods for developing computational intelligence. They often have very strong pattern

recognition capabilities.

What is currently being done?

• IBM is funding a four-year program called “systems neurocomputing” – Developing neural networks to recognize patterns and

avoid the “superposition catastrophe”. Is now using this research to recreate a person’s ability to perceive a broken line.

• Aston Martin, Daimler Chrysler, and other car companies are developing ANN models to detect cylinder misfires in engines.

• Georgia Tech introduced a neural network that combines living and robotic elements. – uses neural networks of cultured rodent brain cells and

robotic body• Recent advances in VLSI circuits, optical computing,

fuzzy logic, and protein-based computing have moved the field closer to realizing massively parallel hardware.

Learning

• Associative mapping – network learns to produce a particular pattern on the set of input units whenever another particular pattern is applied on the set of input units.

• Regularity detection - units learn to respond to particular properties of the input patterns.

How do they work?

• The neuron– Model biological

neurons– many inputs, one output– have weights, a bias,

and a threshold (activation) function

• The network architecture– Three interconnected

layers– Input layer partitions– processing (hidden)

layer analyzes– output layer … outputs– programmer uses

previous knowledge to ease training

How do they work?

• Tissues– networks like neural tissue– output may be input to another network– hidden layer may consist of a number of

such tissues

• Training– weight adjustments– recognizing key part of input– hard to see what the network “learned”

How do they work?

• Firing rule – determines how one calculates whether a neuron should fire for an input – Ex: take a collection of data, some

which causes firing and some which don’t. If new data is inputted, elements most in common with firing data will then cause firing.

How do they work?

• Feed forward architecture– Signals traveling one

way, from input to output (associates input with output)

• Feed backward architecture- Signals can travel both ways with loops. The state continually changes until equilibrium is reached

Successes

Strengths of neural networks:

• Pattern recognition

• Unclear algorithm

• No existing algorithm

• Large amounts of test data

Successes

• 20q– Based on a word game– Learns from users– Correct 80% of the time

• Image recognition– Recognizing objects– Categorizing images– Rendering images searchable

Successes

• Signature analysis– First large-scale use in US– Compares with stored signatures– 97% accuracy over old 83%– old four-way classification more difficult

• Face recognition– seeking to distinguish people– takes 100-200 of training pictures per person– average recognition rate of over 95%– more training does not guarantee better recognition

Current Implementation

• Instant physician– Developed in 1980s, trained a NN to

store a large amount of medical records. After training, could be presented with symptoms, and could then present the best diagnosis

Current implementations

• Business– Marketing control of seat allocation on an

airplane using feed-forward mechanism– Credit models and mortgage screening

boosted profitability of HNC by 27%

• Medicine– NN used to model cardiovascular system.

Build a NN of a patient and compare to actual patient. Can detect medical conditions before they happen.

Current implementations

• User interfaces– Handwriting analysis tools, text-to-

speech conversion (IBM, Babel)

• Industrial processes– control machinery, adjust

temperature settings, and diagnose malfunctions in robotic factories (Alyuda Research Factory)

Problems Encountered

• Applications using neural networks have little or no data available for training on fault conditions, so fuzzy logic is used, based on an expert’s definition of certain rules.

• “Neural network programs sometimes become unstable when applied to larger problems.” – the larger the problem, the more neural

networks must draw information from to obtain a solution, making neural networks very problem specific.

Problems Encountered

• Larger datasets require more extensive training time to reach a predictive solution, and there is the possibility of overtraining, in which there was low training error but high actual testing error. Unknown data necessary for the solution will also cause a high error rate, sometimes by affecting the weighted values used in determining a solution.

Future

• Simple systems which have learned to recognize simple entities (e.g. walls looming, or simple commands like Go, or Stop) may have neural network chips implanted to help in decision-making. Japanese are already using fuzzy logic for this purpose.

• Use of neural networks to put labels on what is determined to be in the pictures, for use in medical searches

• User-specific systems for education and entertainment based on readings taken of the user.

Future

• Development of integration of man and machine, such as with retinal and cochlear implants

• Generally the development of use of neural networks in more everyday and diverse applications, such as in retail and manufacturing, to help make accurate decisions.

Questions?