Michael RoukesCa l tech /Phys i c s
Thanos S i apa sCa l t e ch /Neu r o s c i e n ce
Andreas To l i a sBay l o r CM/Neu r o s c i e n ce
Gi l l e s Lauren tMPI f o r B ra i n Re sea r c h
Co-Proposers –BRAIN In i t i a t ive :
Pau l A l i v i s a tos Be r ke l e y / LBL
Miyoung Chun Kav l i F ounda t i o n
George Church Har va r d
Ra lph Greenspan UCSD/Kav l i
Ra f ae l Yus te Co l umb i a /Ka v l i
FUNCTIONAL CONNECTOMICS: TOWARD AN ACTIVITY MAP OF THE BRAIN
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 1 of 30
WE NEED NEW METHODS
MIchael Roukes. Caltech – © 2013 25 June 2013
overarching goal: to understand how the brain worksthe problem: “emergent” properties of brain function (that involve vastly different length scales)
What apparatus, in general terms, enables the brain to implement its remarkable performance?
The number of components (neurons) in the brain is probably about 1011 (a hundred bi l l ion) . The number of synapses, or contacts , between them is perhaps 1014 (a hundred tr i l l ion) . On the average ever y neuron receives some thousands of dist inct inputs and itsel f connects to many other neurons.
The physical layout of most of the components is not par t icular ly neat .
How might one make some headway through this impossible jungle?
Francis H Crick (1979) Think ing about the bra in . Scient ific Amer ican 241: 219–232
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 2 of 30
Evolving technology that enables complex functional imaging of the brain…
The Human Connectome Project (e.g. Van Essen, et al .)
Optical Reporters for Observing Brain Circuit Activity
The Brain Activity Map Project*
5 June 2013 MIchael Roukes. Caltech – © 2013 3
APPROACHES / PARADIGMS / ANALOGIES
*Alivisatos, A.P., Chun, M., Church, G.M., Greenspan, R.J., Roukes, M.L., and Yuste, R. (2012) The Brain Activity Map Project and the Challenge of Functional Connectomics.
Neuron 74: 970-974.
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 3 of 30
f MRI-basedimaging of wholebrain activity
The Human Connectome Project (conceptual ized representation)
5 June 2013 MIchael Roukes. Caltech – © 2013 4
APPROACHES / PARADIGMS / ANALOGIES
analogy:computer network
“regional processors” in the brain
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 4 of 30
f MRI-basedimaging of wholebrain activity
The Human Connectome Project (conceptual ized representation)
5 June 2013 MIchael Roukes. Caltech – © 2013 5
APPROACHES / PARADIGMS / ANALOGIES
• imaging of local computational activity via energy consumption
analogy:computer network
“regional processors” in the brain
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 5 of 30
f MRI-basedimaging of wholebrain activity
The Human Connectome Project (conceptual ized representation)
5 June 2013 MIchael Roukes. Caltech – © 2013 6
APPROACHES / PARADIGMS / ANALOGIES
• imaging of local computational activity via energy consumption
• observation in correlations of computational activity across regions
analogy:computer network
“regional processors” in the brain
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 6 of 30
5 June 2013 MIchael Roukes. Caltech – © 2013 7
APPROACHES / PARADIGMS / ANALOGIES
• imaging of local computational activity via energy consumption
• observation in correlations of computational activity across regions
• Elucidation of trans-regional connections via such correlations
f MRI-basedimaging of wholebrain activity
The Human Connectome Project (conceptual ized representation)
“regional processors” in the brain
thenetwork
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 7 of 30
The Human Connectome Project
Brain Circuit Activity via Optical Reporters
5 June 2013 MIchael Roukes. Caltech – © 2013 8
APPROACHES / PARADIGMS / ANALOGIES
What’s going on within each of the brain’s local circuits?(it “regional processors”)
Time evolution of logic states at multiple gates
f MRI-basedimaging of wholebrain activity
analogies
logic gate = neuronbits = neuronal “spiking”
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 8 of 30
5 June 2013 MIchael Roukes. Caltech – © 2013 9
OPTICS: FREE-SPACE FUNCTIONAL IMAGINGTHE STATE-OF-THE-ART
3-D Random Access Two-Photon ImagingWith Acousto-Optical Deflectors (AOD)
Two-Photon Functional Imaging of Visual Cortex
i.e. watching brain processing in real time!
Tolias and Saggau Groups, Baylor CM
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 9 of 30
5 June 2013 MIchael Roukes. Caltech – © 2013 10
OPTICS: FREE-SPACE FUNCTIONAL IMAGINGTHE STATE-OF-THE-ART
MAJOR PROBLEM: Calcium recording is, at present, not fast enough to faithfully record the “bits” (neuronal spiking in real time).
Two-Photon Functional Imaging of Visual Cortex
Tolias and Saggau Groups, Baylor CM
Optical recording (via calcium influx)
Simultaneous electrical recording
L. Moreaux & G. Laurent, Estimating firing rates from calcium signals in locust projection neurons in vivo, Frontiers in Neural Circuits 1, 2 (2007)
But, opticalrecordingis too slowto followfull details of neuronalspiking
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 10 of 30
Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration 11
MechanicalForces
ChemicalConcentrations
Electric Fields
…at each point within, there is a multiplicity of information to be gleaned
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 11 of 30
Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration 12
But sub-cortical brain activity cannot be accessed by free-space optics
We posit: The only near-term, general, and massively-multiplexed solution, producible en masse, is a neural-probe-based architecture
inaccessible via free-space optics
can watch here(cortex)
but not in here
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 12 of 30
Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration 13
nanotechnologies can enable real-time, high resolution sensing of these fields
MechanicalForces
ChemicalConcentrations
Electric Fields
integratedneuroelectronics
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 13 of 30
MIchael Roukes. Caltech – © 2013
Toward real-time observation of highly correlated “bit trafficking” (neural coding at the individual gate level)
The Human Connectome Project (e .g. Van Essen, et al .)
Brain Circuit Activity via Optical Repor ter s The Brain Activity Map Project
5 June 2013
APPROACHES / PARADIGMS / ANALOGIES
What’s going on in the brain’s local circuits?(its “regional processors”)
Time evolution of logic states at multiple gates
analogies
logic gate = neuronbits = neural “spiking”
f MRI-basedimaging of wholebrain activity
transmitting neuron
receivingneuron
14
individual neuronal
spikingMichael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 14 of 30
15Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration
Roadmap: Electrophysiology
15
Questions:(a) Can we realize >50,000 channels?(b) How can we access other physical “brain fields”?(c) And, can we address (a) & (b) to permit large-
scale integration and production en masse?
>50,000 channelsmass-produced
and fully integrated100 channels1000 channels
custom
generation 1 generation 210,000 channelsmass-produced
currentstate of the art
goal
Integrated Neuro/Nanoelectronics Roadmap:
We are here.
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 15 of 30
16Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration
Next-Gen Brain-Machine Interfacing
16
integrated nanophotonics
nanoparticle-basedfunctional optical reporters+
integrated neurophotonics
in the optical domain
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 16 of 30
17Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration
Neuro/Nanoparticle Reporters Under Investigation
E
- - + + + +
λem= ~573 nm λem= ~568 nm
CdSe CdS CdSe CdS
Alexandra Courtis
Mechanism of Electric Field Sensor:Quantum Confined Stark Effect
CdSe CdS
λem= >650 nm λem= ~650 nm
F
Tetrapods are Sensitive Local Force Detectors
Alivisatos Group, UCB
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 17 of 30
Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration 18
nanotechnologies can enable real-time, high resolution sensing of these fields:
MechanicalForces
ChemicalConcentrations
Electric Fields
integratednanoelectronics
integratednanophotonics
functionalnanoparticles+
INTEGRATED NEUROPHOTONICS
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 18 of 30
19
>50,000 channelsmass-produced
and fully integrated100 channels1000 channels
custom
generation 1 generation 210,000 channelsmass-produced
currentstate of the art
goal
Proprietary © Caltech 2011-2013 Center for Neuro/Nano Large-Scale Integration
Roadmaps: 10Y Horizon Integrated Neuro/Nanoelectronics
19
Probe Based, LocalizedTwo-Photon Excitation
Massively-Multiplexed Optogenetic Stimulation
Next-Gen NanoparticleOptical Reporters
Two-PhotonFunctional ImagingAnd Stimulation
Probe-IntegratedMicrofluidic Deliveryand Chemisensing
Integrated Neurophotonics
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 19 of 30
f MRI-based imagingof brain “processor” connectivity
resolution: >100K neurons/voxel
5 June 2013 MIchael Roukes. Caltech – © 201320
AN EVOLVING EXPERIMENTAL TOOLKIT
Two-photon functional imagingof brain activity
resolution: single neurontime response: seconds
Highly-multiplexed,nanoprobe-based functional imagingof brain activity
resolution: single neurontime response: ~10’s μs
nanowire arrays
recording sites
10 µm
proposed development via brain activity map project
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 20 of 30
Goal 1: Measure every action potential for every neuron in complete brain circuits
Goal 2: Manipulate the activity of every neuron in these circuits
Goal 3: Computationally analyze/model these circuits
Goal 4: (concurrent!) Develop next-gen technological platforms to permit scale-up to ever-larger hierarchical brain structures
BRAIN ACTIVITY MAP …GOALS WE ORIGINALLY ENVISAGED
MIchael Roukes. Caltech – © 2013 215 June 2013worm fly fish mouse
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 21 of 30
TECHNOLOGICAL PARALLEL:THE HUMAN GENOME PROJECT
technologyevolution
MIchael Roukes. Caltech – © 2013 225 June 2013
Single InvestigatorLab
Academic Center
Large-Scale Commercialization
& Production
Genome Centers
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 22 of 30
…lessons learned: HASTENING TECHNOLOGICAL EVOLUTION
phases of technological
evolution
12
43
University research: Single-investigator development
& prototype demonstrations of “mom & pop” technologies
One-of-a-kind prototypes, Not amenable to near-term
automation or scale-up
University CENTERS research:
Standardization of technologies
Protocols become readily adaptable to automation and production en masse
Corporate spin-offs
Production of automated systems based on “core technology” that is:
Robust Scalable Mass-producible
Data Centers (“Brain Observatories”)
Creation of massively-parallel data acquisition “centers”, which amass and synchronize the operation of large automated systems arrays; enables collection of unprecedented data libraries.
SCALE-UP “REAL” UTILITY
IP transfer
5 June 2013 MIchael Roukes. Caltech – © 2013 23
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 23 of 30
A DRAFT ROADMAP
MIchael Roukes. Caltech – © 2013 245 June 2013
15 years: Entire brains behaving
10 years:1 million neurons
5 years: 50,000 neurons
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 24 of 30
Nanoscientists ; and engage their key exper t faci l i t ies and staf f
Chemists/biochemists developing nanopar t ic le and molecular repor ter s
(for st imulat ion and recording)
State-of-the-ar t microchip research foundries to translate “one-ofs” into
prototypes capable of scale-up and production en masse
Industrial partners to subsequently enable mass production and system
integrat ion: enabl ing real izat ion and deployment of robust , integrated
measurement instr uments
WHO MUST WE ENGAGE TO MAKE THIS HAPPEN?
MIchael Roukes. Caltech – © 2013 255 June 2013
and then…
TOO
L D
EVEL
OPM
ENT
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 25 of 30
Nanoscientists ; and engage their key exper t faci l i t ies and staf f
Chemists/biochemists developing nanopar t ic le and molecular repor ter s
(for st imulat ion and recording)
State-of-the-ar t microchip research foundries to translate “one-ofs” into
prototypes capable of scale-up and production en masse
Industrial partners to subsequently enable mass production and system
integrat ion: enabl ing real izat ion and deployment of robust , integrated
measurement instr uments
WHO MUST WE ENGAGE TO MAKE THIS HAPPEN?
MIchael Roukes. Caltech – © 2013 265 June 2013
Experimental neuroscientists explor ing worm, fish, mouse , rat , tur t le , …and, when wel l-val idated, pr imates and humans Computer scientists at the forefront of massive data mining technologies Computational neuroscientists bui lding models/analyses with next-gen complexity
and then…
TOO
L D
EVEL
OPM
ENT
NEW
SC
IEN
CE
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 26 of 30
Devices and techniques for diagnosing brain disorder s ear l ier and more accurately
Strategies for high-resolution patterned brain stimulation to
rebalance diseased circuits, i .e . next-gen brain-machine interfaces
Sensit ive , miniature , and intel l igent nanosystems for engineer ing
and environmental applications
Development of novel, biological ly-inspired, next-gen
computational devices and architectures.
AMONG THE LARGER BENEFITS…
MIchael Roukes. Caltech – © 2013 275 June 2013
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 27 of 30
The BRAIN Initiative needs your support!
Large scale: it’s too large for single-investigator grants or foundations alone. Truly big science… think satellites , telescopes, Human Genome Project.
Long term: Horizon is too far for ear ly stage corporate or VC investment. We are talking 5, 10, 15 year horizons. Must star t now though!
A fundamentally cross-disciplinary effort: no obvious single agency home.
MISSION CRITICAL: The new nanotechnologies must be assembled first!
THEN: Integrated nanosystems can provide platforms enabling entirely new ways of doing things – hastening a next generation of neurophysiological discover y
Human genome-like project in scope – but unlike HGP we know at the outset what technologies to invest in to make the BRAIN Initiative happen!
HOW DO WE GO FORWARD FROM HERE?
MIchael Roukes. Caltech – © 2013 285 June 2013
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 28 of 30
NEW TOOLSNew directions in science are launched by
new tools much more often than by new concepts.
The effect of a concept-dr iven revolution is to explain old things in new ways.
The effect of a tool-driven revolution is to discover new things that have to be explained. ”
Freeman Dyson (1997) Imagined Wor lds
Har vard Univer s i ty Press , Cambr idge , MA
MIchael Roukes. Caltech – © 2013 295 June 2013
Neuroscience and nanoscience have separately evolved to the present, auspicious juncture – the synergistic opportunities at this moment are without precedent.
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 29 of 30
THANK YOU
5 June 2013 MIchael Roukes. Caltech – © 2013 30
Michael Roukes, Caltech / 5 June 2013 © 2013 Caltech, Baylor College of Medicine, UC Berkeley Page 30 of 30