Optimizing with synapses

Sebastian Seung

Howard Hughes Medical Institute

and Brain & Cog. Sci. Dept., MIT

Practice makes perfect

• Birdsong learned from male tutor

• Stored template• Zebra finch: up to

100,000 iterations• Known anatomy and

physiology

Hahnloser, Kozhevnikov, Fee (2002)

Supervisory signals in the brain

• Global broadcast of reward signal• E.g. dopaminergic system

Neural basis of learning

Global signal(Reward, motor error, etc.)

Local signals (Voltage, calcium, etc.)

Synaptic plasticity

The interaction between global and local signals is largely uncharacterized.

Noise injection hypothesis

• HVC vs. LMAN– lesion– neural activity

RA

HVC

LMAN

motor neurons

Doya and Sejnowski

Trial and error learning

• Generation of variability• Reinforcement of favorable variations

Synaptic learning rule

1 2reward

3

regular synapses

noise synapses

regular

noise

reward

0W 0W Fiete and Seung

Optimization in biology

• Evolution– Search in genotype space– Random genetic variation

• Learning– Search in synapse space?– Unreliable synapses– Noise injection

Stochastic gradient learning

• Systems-level models of learning– Birdsong– Oculomotor system

• Synaptic plasticity in vitro– Microisland cultures

• Training neural circuits in vitro– Silicon stimulation– Pattern culture– Intrinsic imaging by interferometry

Reward-driven plasticity in vitro

Jen WangNaveen Agnihotri

Patterned culture by inkjet printing

Sawyer Fuller and Neville Sanjana

In vitro models of learning

• Goal: study how synaptic plasticity affects the dynamics of neural circuits

• Technical challenge: electrical and chemical control of neurons

• Conceptual challenge: training neural circuits