Neuromorphic Computing in the
European Human Brain Project
Karlheinz Meier @brainscales
Ruprecht-Karls-Universität Heidelberg
NICE 2016, Berkeley
1. Neuroinforma-csPla3ormAggregateneurosciencedata,deliverbrainatlases
2. MedicalInforma-csPla3ormAggregateclinicalrecords,classifybraindiseases
3. BrainSimula-onPla3ormDevelopsoDwaretools,runclosedloopbrainsimula-ons
4. HighPerformanceCompu-ngPla3orm
DevelopandoperateHPCsystemsop-mizedforbrainsimula-ons
5. NeuromorphicCompu-ngPla3ormDevelopandoperatenovelbrainderivedcompu-nghardware
6. Neurorobo-csPla3ormDevelopvirtualrobo-csystemsforclosedloopcogni-veexperiments
Publiclauncheventatendoframp-upphase–March,30th
The6ICTPla+ormsinHBP
Temporal Scales and Strong Scaling
Computa7onalComplexity
Memory
Requirement
1MB
10GB
1TB
100TB
100PB
CellularNeocor-calColumn
CellularMesocircuit
CellularRodentBrain
CellularHumanBrain
1Gigaflops 1Teraflops 1Petaflops 1Exaflops
SingleCellularModel
Subcellulardetailandplas-cityrequireadvancesinstrongscaling!
Glia-Cell/VasculatureO(1-10x)
Reac-on-DiffusionO(100-1,000x)
MolecularDynamicsO(>1,000,000,000x)
Plas-cityO(1-10x)
LearningO(10-100x)
DevelopmentO(100-1000x)
TheONLYwaytoevermakeuseof
ar-ficialneuralcircuitsderivedfrom
biologyistomakethemadap$ve
Connec-vity–Synapses–Neurons
byclosed-loopinterac-onwithdata
Click to edit Master title style
• Click to edit Master text styles
– Secondlevel• Third level
– Fourth level
Previous work is essential
BrainScaleS
SpiNNaker
FACETS/BrainScaleS 2005-2015 SpiNNaker 2005-2015
8-10 years from chip design to system !
Requires roadmap and sustained funding
- Not developed
in HBP -
• 18ARM968Coresperchip• IntegerArithme-c
• 200MHzProcessorClock
• SharedsystemRAMondie
• 128MbyteDRAMstackedondie
• EachChip6bi-direc-onallinks• 6millionspikes/s/link
• RealTimeSimulator
SpiNNaker
Group(+HBP)
HBP SpiNNaker Machine Generations (Manchester Site)
103 104 105
Ra-onalesfortheBrainScaleSPhysicalModelSystem
Ø Mixed-Signal(Localanalogcomputa-on,binaryspikecommunica-on)
Ø Drivenbyarchitecture,notdevices(180nmCMOS)
Ø HighNeuronInputCount(>10.000)
Ø Configurability(cellparameters,connec-ons)->Universality
Ø Scalability:ChipScale(105)->WaferScale(108)->Systems(>109)
Ø Accelera-onx10.000,consistent-meconstants(1daycompressedto10seconds)
Ø Short-termundlong-termPlas-city
Ø Upgradabilitywithunchangedsystemarchitecture
Ø HybridOpera-on,closedloopexperiments
Ø Non-ExpertUserAccess
Objec-ve:Exploitconfigurabilityandaccelera-on
-rapidexplora-onoflargeparameterspaces
-covershortandlong-mescalecircuitdynamics
-performcompu-nginthepresenceofspa-alandtemporalnoise
HiCANNHigh
InputCount
Analogue
NeuralNetwork
Chip
Millner,S.,Grübl,A.,Meier,K.,Schemmel,J.andSchwartz,M.-O.,AVLSIImplementa-onoftheAdap-veExponen-alIntegrate-and-FireNeuronModel
AdvancesinNeuralInforma-onProcessingSystems(NIPS)(2010)
Physical Model, local analogue computing,
binary continuous time communication
Wafer-Scale Integration of 200.000 neurons and 50.000.000 synapses on
a single 20 cm wafer
Short term and long term plasticity, 10.000 faster
than real-time
Wafer-scaleintegra$onofanalogneuralnetworks,J.Schemmel,J,FieresandK.Meier
In:ProceedingsofIJCNN(2008),IEEEPress,431
Click to edit Master title style
• Click to edit Master text styles
– Secondlevel• Third level
– Fourth level
HBP : Neuromorphic Computing Platform
THEPHYSICALMODELSYSTEM
Localanaloguecompu-ngwith4Millionneurons
and50Millionsynapses–binary,asynchronous
communica-on–runningatx10000real--me
Loca-on:Heidelberg(Germany)
Offering : Access to a unique set of 2 complementary, highly configurable neuromorphic machines for modelling neural microcircuits and applying brain-like principles in machine learning and cognitive computing
THEMANY-COREDIGITALPROCESSORSYSTEM
0.5–1MillionARMprocessors–address-based,smallpacket,
asynchronouscommunica-on–runningatreal--me
Loca-on:Manchester(UK)
1
5
67
23
4
8
8
500.000core
machine
Loca-on:
Manchester(UK)SeetalkbySteveFurber
20Wafermodule
machine
Loca-on:
Heidelberg(GE)
SeetalkbyJohannesSchemmel
Click to edit Master title style
• Click to edit Master text styles
– Secondlevel• Third level
– Fourth level
Remote Access ready for users
Slide16
Heidelberg
PyNN
Job queue server
Model description
Experiment description
Data storage
Mapping
Calibration data HAL
Authentication Validation Notification
Manchester
PyNN
PACMAN
SpinnMan
See demo by Eric Mueller
HBPNeuromorphicCompu-ngGuidebook–Con-nuousUpdates
Comprehensiveopenaccessdocumenta-on:
Hardware,systems,firmware,low/highlevelsoDware
Benchmarks(neuroscience,machinelearning)
Tutorials,smallsystemsdescrip-on
hnp://electronicvisions.github.io/hbp-sp9-guidebook/
NeuromorphicLaptopAdd-on
PlugsintoUSB,fullsoDware
supportanddocumenta-on
498neurons
100.000plas-csynapses
100.000fasterthanreal--me
SeedemobyEricMueller
Increasingnumberofusecasesandapplica-ons
coveringawidespectrumofnetworktypes
Exploi-ngSubstrateUNIVERSALITY–selec-onofpublishedwork:
- Canonicalcircuits(synfirechains,WTA,aOractorcircuits)- Balancedrandomnetworks- Liquidcompu$ng,temporalpaOerniden$fica$on- MinicolumnLayer2/3circuits- Closed-loophybridcontrolsystems- Mul$variatedataclassifica$on- Phasedetec$on,applyingSTDP- Decorrela$onthroughinhibitoryfeedback- Stochas$cinferencethroughneuralsampling- BayesiannetworksasBoltzmannmachinesofLIFneurons- Impleme$ngdeeplearningwithspikingneurons- Implemen$ngHTMwithspikingneurons
SeetalkbyMihaiPetroviciandposterbyLuziweiLeng
2023 Roadmap details in FPA document
Click to edit Master title style
• Click to edit Master text styles
– Secondlevel• Third level
– Fourth level
From SpiNNaker to SpiNNaker2
Feature SpiNNaker SpiNNaker2
technology 130nm 28nm
cores 18 68
core frequency 200MHz >400MHz
external memory 128MByte (1 Gbyte/s) 2GByte (>10 Gbyte/s)
power 1W 1W
power management no yes
floating point support no yes
vector processing no yes
true random numbers no yes
biological realtime operation yes yes
no. of neurons / chip 16k 128k
no. of synapses / chip 16M 128M
energy/synaptic event 10-8J 10-9J
≈10ximprovementatconstantpower
PhysicalModel:TargetsinHBPfor2023
PrototypesintheLab
StructuredNeurons
Ac-vedendriteswithspa-al
structure:Neuronsascomplex
panerndetectors(e.g.
hierarchicaltemporalmemory
Plas-cityProcessor
400PowerPCprocessorsper
wafer:Re-wiringonthefly,data
drivenaccelerateddevelopment,
slowandfastcircuitsdynamics
SeedemobyEricMueller
Observables Controls Synapseevalua-on
Popula-onrates
Arbitraryinternal
parameters
Weights
Connec-vity
Rewiring
Neuronparameters
Homeostasis
S-mulusgenerators
Externalrewards
andcontrols
Essen-al:Any-mescale>100µs(bio)isaccessible
Wafer-PCBLamina-onforlargescale
highdensitymanufacturing
neuromorphic.eu