FROM BRAIN RESEARCH TO FUTURE TECHNOLOGIES

Dirk Pleiter | Post-H2020 Vision for HPC Workshop, Frankfurt | 24.06.2018

Science Challenge and Benefits

Understanding the human brain• Understand the organisation of the brain• Understand brain functioning and dysfunction

Social and economic impact• Enable treatment of brain diseases• Enable new approaches to artificial intelligence

Complexity of the brain• About 90 billion nerve cells, each with 10,000 connections• Multi-scale challenge

Need for interdisciplinary approaches• Neuroscience, medicine, physics etc.• Computer science and engineering, mathematics etc.

224.06.2018

Whole braincm scale

Neuronμm to mm scale

Synapsenm to μm scale

The Human Brain Project

Flagship project funded by the EC• Science-driven, seeded from Future and Emerging Technologies, extending beyond ICT• Started in 2013 as a 10 years efforts• More than 110 partners in 19 countries

Objectives of the HBP• Realise an European scientific Research Infrastructure for brain research/brain-inspired sciences• Gather, organise and disseminate data describing the brain and its diseases• Simulate the brain• Build multi-scale scaffold theory and models for the brain• Develop brain-inspired computing, data analytics and robotics• Ensure that the HBP's work is undertaken responsibly and that it benefits society

324.06.2018

A Case for Co-Design

Need for of a cutting-edge ICT-based scientific research infrastructure• Simulate the brain• Gather, organise and disseminate data

Brain research seed future and emerging technology• Develop brain-inspired computing, data analytics and robotics

424.06.2018

ENABLING BRAIN RESEARCH THROUGH NEW ARCHITECTURES AND TECHNOLOGIES

High-Resolution Brain Models from Histological Data

Research goal• Accurate, highly detailed computer model of the

human brain based on histological input

Approach• Create high-resolution 2-dimensional brain

section images (1μm resolution)• Re-construct 3-dimensional models from these

images• Reconstruct nerve fibers in histological brain

sections• Automatised analysis of high-resolution images

624.06.2018

[K. Amunts et al., Science 2013]

[M. Axer et al.]

Data Analytics on Extreme-Scale Data Volumes

Data challenge• Data acquisition using 8-9 microscopes at ~15 MByte/s → ~10 TByte/day• Archival data increase 1-2 PByte per year

Data analytics challenge• Deep learning methods for automatised cell segmentation• Massively parallel feature extraction (e.g. cell count)• Image registration

Data access speed challengee• Example of multi-GPU training requirements

(using P100 GPUs)

724.06.2018

DataAcquisition

Site local storage

PermanentHPC storage

Fast HPC storage

HPC/HPDA systemΔ t IO < ϵ Δ t comp → BIO >

NGPUϵ 0.3GByte / s

Need for improved data transport capabilities and support for high performance data analytics workflows

[K. Amunts, T. Dickscheid et al.]

Dynamical Numerical Brain Models

Research goal• Create biologically realistic models of the brain

– Realistic size of models– Realistic level of detail

Multi-scale challenge• Multi-scale simulations based on co-simulation of different models

Supercomputer requirements• Maximize memory footprint

– Application today is memory capacity limited• Optimize processing performance → memory performance

– Keep ratio simulation versus simulated time small

Allow for interactive steering of the applications• Requires interactive visualisation as well as low-latency control loop

824.06.2018

Need for scalable compute resources (efficiently exploitable for applications)

Brain Simulation Workflows

924.06.2018

HPC system

PermanentHPC storage

Fast HPC storage

Interactive viz

Jupyter notebook

Knowledge Graph

Need for interactive compute services integrated in HPC centres

Coupling HPC Systems and External Facilities

Closed-loop brain and robot model based experiments• Brain simulation coupled to robot simulation engines

1024.06.2018

Need for bridges to external services

Integration and Creation of Knowledge

Enable ICT platforms integrating different discovery steps• Experiment• Model building and simulation• Data integration and strategic data collection

Creation of knowledge spaces• Creation of data sets curated following agreed procedures• Improve on tapping all possible data sources (including

unstructured data sources• Creation of knowledge spaces

Federation of services• No centralisation → be open and allow for extensions• Allow for integration of very heterogeneous compute

services or data sources

1124.06.2018

Need for integration of various compute and data services

Need for infrastructure federation

Knowledge and Theory

Experiment

Data

Simulation

New experiments New models

Other data sources

Need for technologies tapping all relevant data sources

Medical Informatics: Coping with Sensitive Data

Objective• Enable access to distributed harmonised pan-omics patient data for clinical and fundamental brain research

– E.g., electronic health records (EHR), picture archiving and communication systems (PACS)

Challenge• Use of personal data

Strategies• Securing data repositories for personal data• Introduce de-identification mechanisms and services

– Pseudonomisation– Anonymisation

1224.06.2018

Need for technical means supporting- Securing data storage and access- Secure provisioning of de-identification services

Service-Oriented Provisioning of HPC/HPDA Resources

Provisioning of broad range of integrated infrastructure services• Different types of compute services including

– Scalable compute services– Interactive compute services– VM services

• Different types of data stores

Develop and deploy services that facilitate federation• Federation of AAI, monitoring etc.

Encourage communities to build community specific platforms• Push for further separation of concerns• Close engagement with communities

1324.06.2018

Need for support of new HPC use models including- Flexible provisioning of hardware resources- Supporting federation of infrastructure- Support for new compute devices (NC, QC)

FROM BRAIN RESEARCH TO NEW ARCHITECTURES AND TECHNOLOGIES

Learning from the Brain: Extremely High Connectivity

Large Rent’s exponent• Rent’s rule: Scaling of connections as function of

logic element

• Typical value for micro-processors: p ≈ 0.3 … 0.8• Smaller values of p outside of micro-procesor• Estimated value for mammalian brains: p ≈ 0.8

1524.06.2018

Brain as inspiration for improving on new approaches to improve data transport capabilities and space filling computers

C=k N p

[P. Ruch et al., 2011]

[Landman and Russo, 1971;M.Y. Lanzerotti et al., 2005; D.S. Bassett et al., 2010 ]

Learning from the Brain: Extremely Small Power Density

Smaller frequency and power density• Brain based on massively-parallel distributed

architecture• Micro-processors based on von-Neumann

architecture are much sequential and centralised

Analogue logics• Analogue logics can be much faster for certain

operations, e.g. time integration for neural cell membrane

1624.06.2018

Brain as inspiration for neuromorphic processing architectures and massively-parallel digital-analogue circuitries for compute

[Merolla et al., Science, 2014]

HBP’s Neuromorphic Systems

SpiNNaker @ Manchester

1724.06.2018

BrainScales @ Heidelberg

[K. Meier et al.][S. Furber et al.]

New Approaches to Artificial Intelligence

Today: Deep learning based on algorithms and implementations that are different from those used by our brains• Currently used ANN+algorithms quite successful but very compute intensive

Principles of learning in the brain seem to be different• Neural circuits are highly recurrent networks consisting of different types of neurons and synapses with diverse

dynamic properties• Networks of neurons in the brain provide stable computational function in spite of ongoing rewiring and network

perturbations

Research starts leading to new learning algorithms• Algorithms particular suitable for neuromorphic hardware

1824.06.2018

[Nature Editorial, 2/2018]

Brain as inspiration for new approaches to artificial intelligence and possibly new semiconductor innovations

[W. Maass, 2017, 10.1016/j.cobeha.2016.06.003]

SUMMARY AND CONCLUSIONS

Summary and Conclusions

Science drivers are key for defining future HPC system/infrastructure architecture and technology roadmaps• Brain research is only one research area• Others include, e.g., earth system modelling (including weather)

Brain research as driver for architectures and technologies• Exascale computing and beyond• Creation of federated, ICT-based research infrastructures integrating various compute and storage

resources connected to various data sources and coupled to external experiments• Convergence of HPC and “big data” and cloud

Brain research as enabler for new technologies• Neuromorphic computing• New approaches to artificial intelligence

2024.06.2018