Win

ter

200

7Bi

oE33

2A

© Kwabena Boahen

LargeLarge--scale neural modelingscale neural modeling

KwabenaKwabena BoahenBoahenStanford BioengineeringStanford Bioengineeringboahen@stanford.eduboahen@stanford.edu

Goal: Link structure to function by Goal: Link structure to function by developing multideveloping multi--level level computational models of neural computational models of neural systems.systems.

Win

ter

2007

BioE

332A

© Kwabena Boahen

WeWe’’re acquiring brain data re acquiring brain data at an unprecedented rate at an unprecedented rate

Reid et al 2005

CaCa++++ imagingimaging

Computational Computational primitivesprimitives

Functional Functional behaviorbehavior

MicrocircuitryMicrocircuitry

Hausser et al 1997

Dendritic recordingDendritic recording Serial Scanning EMSerial Scanning EM

Denk et al 2005

Now all we have to is connect the dotsNow all we have to is connect the dots……

+ =

Win

ter

2007

BioE

332A

© Kwabena Boahen

MultiMulti--level simulations can link level simulations can link structure to functionstructure to functionThe problem is one of scaleThe problem is one of scale

7 levels of investigation7 levels of investigation

10 orders of magnitude10 orders of magnitude

Option 1: Dissect experimentallyOption 1: Dissect experimentally

System properties are lostSystem properties are lost

Option 2: Analyze theoreticallyOption 2: Analyze theoretically

Stochastic, heterogeneous, Stochastic, heterogeneous, recurrent, nonlinearrecurrent, nonlinear

Option 3: Simulate directly Option 3: Simulate directly

Include all the detailsInclude all the detailsComplements theoryComplements theory

Control all the parametersControl all the parametersComplements experimentComplements experiment

Churchland & Sejnowski 1992

Win

ter

2007

BioE

332A

© Kwabena Boahen

The fastest supercomputers The fastest supercomputers available are not up to the taskavailable are not up to the task

CellCell

8M neurons connected by 4B synapses8M neurons connected by 4B synapses99°° visual field in V1visual field in V1

1sec of activity took 1hr 20mins to simulate1sec of activity took 1hr 20mins to simulate47504750×× slower then realslower then real--timetime

Had to perform 38 trillion evaluationsHad to perform 38 trillion evaluations8M neurons 8M neurons ×× 6 comp. 6 comp. ×× 8 8 eqeq. . ×× 101055 steps/sec steps/sec

CompartmentCompartment

( )

( )1

V

Vu u

α

β−

( )( )

u V ududt Vτ

∞ −=

( ) ( )( ) ( )

Vu V

V Vα

α β∞ =+

( ) ( ) ( )1V

V Vτ

α β=

+

IonIon--channelchannel Blue Gene supercomputerBlue Gene supercomputer

LansnerLansner et al. used one et al. used one 20482048--processor rack processor rack (3Tflops, $2M).(3Tflops, $2M).

Shenoy et al. 2006

Win

ter

2007

BioE

332A

© Kwabena Boahen

DonDon’’t evaluate equationst evaluate equations——emulate physicsemulate physics

Emulate ionic currents with electronic currentsEmulate ionic currents with electronic currents

Exploit physical Exploit physical analogyanalogyIncluding stochastic behaviorIncluding stochastic behavior

Analog VLSIAnalog VLSIVery Large Scale Integration Very Large Scale Integration

Runs in realRuns in real--timetimeTakes 1sec instead of 1hr and 20minsTakes 1sec instead of 1hr and 20mins

( )

( )1

V

Vu u

α

β−

( )( )

u V ududt Vτ

∞ −=

( ) ( )( ) ( )

Vu V

V Vα

α β∞ =+

( ) ( ) ( )1V

V Vτ

α β=

+

DrainSource

Gate

Bulk

e-⇔ ⇔

Ion channelIon channelTransistorTransistor

Mead 1989

Win

ter

2007

BioE

332A

© Kwabena Boahen

Physicists revolutionized astrophysics Physicists revolutionized astrophysics by building their own supercomputerby building their own supercomputer

Hubble Telescope 1999

Two spiral galaxiesTwo spiral galaxies

Hardwired to calculate gravitational forceHardwired to calculate gravitational force

A third as fast as Blue Gene rack (1Tflop)A third as fast as Blue Gene rack (1Tflop)

Sixteen times more costSixteen times more cost--effective ($42K)effective ($42K)First to show First to show gravothermalgravothermal oscillationsoscillations

Resulted in 40 papers in 2000 aloneResulted in 40 papers in 2000 alone

Univ. of Tokyo Univ. of Tokyo astrophysicist astrophysicist Jun MakinoJun Makino

2001

Point mass approx.Point mass approx. Law of gravityLaw of gravity

2i

j ji ij

mF Gmr

= ∑

GRAPE6 supercomputerGRAPE6 supercomputer

Win

ter

2007

BioE

332A

© Kwabena Boahen

NeurogridNeurogrid——an affordable supercomputer an affordable supercomputer for neuroscientistsfor neuroscientists

2 years2 years 5 years5 yearsNeurogrid Neurogrid (chips)(chips) 44××44 88××88

Neurocore Neurocore (neurons)(neurons) 256256××256256 1K1K××1K1K

Total Total (neurons)(neurons) 1M1M 64M64MSpeedup Speedup ((××GRAPE)GRAPE) 280280 18,20018,200

NeurogridNeurogrid: Board with grid of chips: Board with grid of chipsProgrammable connectionsProgrammable connections

One chip per cortical cellOne chip per cortical cell--layer or typelayer or type

NeurocoreNeurocore:: Chip with array of neuronsChip with array of neuronsProgrammable ionProgrammable ion--channel propertieschannel properties

Multiple compartments per neuronMultiple compartments per neuron

Chip 1

Chip 2

0 1 2 3

0 1 2 3

0 1 2 31 3 2 0

prepostRAM

Win

ter

2007

BioE

332A

© Kwabena Boahen

MultiMulti--area cortical models area cortical models

Society for Neuroscience 2005Essen & Fellerman 1991

isual areasisual areas

Sillito ‘06

MTMT

V1V1

Bac

kwar

dB

ackw

ard

Forward

Forward

Feedforward view of motionFeedforward view of motion

V1: PartsV1: Parts

MT: ObjectMT: Object

Anatomy has feedbackAnatomy has feedback

MT projects to V1MT projects to V1

Hypotheses about feedback:Hypotheses about feedback:

Aggregates parts into Aggregates parts into coherent objectcoherent object

Composes cues into Composes cues into unambiguous perceptunambiguous percept

Win

ter

2007

BioE

332A

© Kwabena Boahen

BioE332BioE332’’s thousands thousand--neuron babyneuron baby

RAM

Computer

STDP Chip

USB

CPLD

Win

ter

2007

BioE

332A

© Kwabena Boahen

The chip: SpikeThe chip: Spike--timing dependent plasticitytiming dependent plasticity

INTER-NEURON

PRINCIPLE CELLS

PRINCIPLE CELLS

PLASTIC SYNAPSES

PLASTIC SYNAPSES

PLASTIC SYNAPSES

PLASTIC SYNAPSES

1024 excitatory principle cells1024 excitatory principle cells21 plastic synapses each21 plastic synapses each

256 inhibitory interneurons256 inhibitory interneurons

~750,000 transistors~750,000 transistors

10.2mm10.2mm22 in 0.25in 0.25μμm CMOSm CMOS

95µm

60µm

Win

ter

2007

BioE

332A

© Kwabena Boahen

The GUI: Memorizing patternsThe GUI: Memorizing patterns

Before learning After learning

Synaptic strengths

LTP

LTD

Synaptic strengths

Neuron array Spike trains Neuron array Spike trains

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 1: Synapse ModelLab 1: Synapse Model

Axon

TQ

Dendrite

G

Sum

Temporal IntegrationTemporal Integration

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 2: Neuron ModelLab 2: Neuron Model

NaEKE

Nag

Kg

mV

lkgCurrent ClampCurrent Clamp

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 3: Adaptation and BurstingLab 3: Adaptation and Bursting

NaEKE

Nag

KgmV

lkg

CaECag

KEKCag

Current ClampCurrent Clamp

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 4: Phase ResponseLab 4: Phase Response

Effect of InhibitionEffect of Inhibition

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 5: SynchronyLab 5: Synchrony

Inhibitory NetworkInhibitory Network

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 6: BindingLab 6: Binding

ExcitatoryExcitatory--Inhibitory NetworkInhibitory Network

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 7: Lab 7: PotentiationPotentiation & & DepresssionDepresssion

SpikeSpike--timing dependent plasticitytiming dependent plasticity

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 8: Plasticity and SynchronyLab 8: Plasticity and SynchronyPlasticity enhanced phasePlasticity enhanced phase--codingcoding

Win

ter

2007

BioE

332A

© Kwabena Boahen

Lab 9: Associative memoryLab 9: Associative memory

Memory recall Memory recall

Before learning After learning