ICS 586: Neural NetworksICS 586: Neural Networks

Dr. Lahouari GhoutiDr. Lahouari Ghouti

Information & Computer Science DepartmentInformation & Computer Science Department

ICS 586: Neural NetworksICS 586: Neural Networks

First Semester 2008/2009 (081)First Semester 2008/2009 (081) Instructor:Instructor:Dr. Lahouari GhoutiDr. Lahouari Ghouti

Information and Computer Science DepartmentInformation and Computer Science Department

Email: lahouari@kfupm.edu.saEmail: lahouari@kfupm.edu.sa

Office: Building 22 – Room 128Office: Building 22 – Room 128

Tel: 1922Tel: 1922

Grading PolicyGrading Policy

Task Weight

Four Quizzes 10%

Homeworks 10%

Research Paper (Lecture) Presentation 10%

One Major Exam 15%

Final Exam 25%

Research Project [Proposal 5% - Final Report + Prototype 25% - Class Presentation 10%]

40%

Tentative ScheduleTentative Schedule• Introductory Meeting [Introduction to Neural Networks]• Single Layer Perceptron• Multilayer Perceptron• ADALINE• The LMS Algorithm• Backpropagation Learning• Overfitting, Cross-Validation, and Early Stopping• Simple Recurrent Networks• Pattern Classification (Guest Speaker?)• Radial Basis Functions• Support Vector Machines• Competitive Learning and Kohonen Nets• Hebbian Learning• Principal Components Analysis (PCA)• Adaptive Principal Components Extraction (A Student)• Non-Negative Matrix factorization (A Student)• Hopfield Networks and Boltzmann Machines• Bayesian Networks (A Student)• Hidden Markov Models (A Student)•Extreme Learning Machines (A Student)

Programming EnvironmentProgramming Environment

We will be using MATLAB in this course.If you know, that’s fineIf you do not, you will need to learn it.

Getting Started With ANNsGetting Started With ANNs

Foundations of Neural Computation:

Understand the operation of single neurons or small neural circuits.

Detailed biophysical models of nerve cells (receptors, ion channels, membrane voltage), and collections of cells.

Varieties of “Neural Networks” ResearchVarieties of “Neural Networks” Research

1- Neuronal Modeling

2- Computational Neuroscience

3- Connectionist / Parallel Distributed Processing (PDP) Models

4- Artificial Neural Networks (ANNs)

Connectionist (PDP) ModelingConnectionist (PDP) Modeling

Model human cognition in a brain-like way:

• Massively parallel constraint satisfaction.

• Distributed activity patterns instead of symbols.

• Models are fairly abstract.

ANN LandscapeANN Landscape

Artificial Neural NetworksArtificial Neural Networks

Models:Simple, abstract, .neuron-like. computing elements; local computation.

Applications: Pattern recognition, adaptive control, time series prediction.

This is where the money gets made!

Reference: Pomerleau 1993: ALVINN

Artificial Neural Networks: The BeginningsArtificial Neural Networks: The Beginnings

W. S. McCulloch and W. Pitts (1943) Logical calculus of the ideas immanent in nervous activity. Philosophy of Science 10(1), 18-24.

Warren McCulloch Walter Pitts

Revolutionary Idea: think of neural tissue as circuitry performing mathematical computations!

The McCulloch-Pitts NeuronThe McCulloch-Pitts Neuron

Linear weighted sum of inputs:

Learning rule:

Nonlinear, possibly stochastic transfer function:

Transfer function g(x)

i

iixwnetact

netactgy

iw

What to do now?What to do now?

• Check WebCT for course syllabus + soft copy of textbook

• Start learning MATLAB if you do not know it!

• Select a topic you want to present in the class from the tentative schedule (first-come first-serve basis!)

• Select your time slot to discuss with me your project. The earlier you start, the better off you will be.