Intro Design Hardware Software Future

SCILENS INFRASTRUCTUREHTTP://SCILENS.ORG

Niels Nes

Martin KerstenArjen de Rijke

15-04-2016

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

DATABASE ARCHITECTURES

Data Science:Query Data via DBMSData VaultsSQL extended with Python/R.

MonetDBDA developedMain MemoryColumn StoreCache ConsciousAnalytic Work loads (full transaction support)http://monetdb.orgstartup: http://monetdb.com

Collaboration with other research areas, Astronomy(Lofar/Skyserver), Life Science (HBP), Data Mining (UU),Information Retrieval (KUN)

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

HISTORY

1997-2003 Medusa:SGI Origin64GB32 R12000/R10000 cpusccNuma

2003-2009 FACS:64 bitAMD opteron clusterIB (10G)

2010-2015 Scilens I2016-2018 Scilens II

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

WHY

Big Data, 4th Paradigm, DBMS research challengesmulti paradigm data models: array vs tablestream processing: sensor data vs transactionsmulti-scale query processing: (sci-lens)Summarization: query formulation problemGreen Computationworkflows: meta (derived) data explosion

Platform for emerging technologies:multicoreGPUSSD/NVMSoftware Defined ’anything’

Hands on, ie local maintenance

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

DESIGN GOALS

Amdahl lawsparallelism law: If a computation has a serial part S and aparallel component P, then the maximum speedup isS/(S+P).balanced system law: A system needs a bit of IO persecond per instruction per second: about 10 instructionsper second implies a need for 1 byte of IO per second.memory law: α = 1: that is the MB/MIPS ratio (called alpha(α)), in a balanced system is 1.IO law: Programs do one IO per 50,000 instructions

Different systems sizes:query at different scalescompare efficiency, single system vs multiple

flexibility

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

WISH

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

TEST PHASE: 2011

Test Systems:2 Intel atom level systems2 2011 desktops2 2011 heavy desktops (i7)different cases, power suppliesstorage extenders.disks, SDDs

ResultsPower Efficiencypower suppliesARM cpus hardly availableSSD’s to expensive and first SDDs relatively slow.

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

SCILENS I: OVERVIEW

Pebbles 2011Rocks 2011Bricks 2012Stones 2013Gems 2013Diamonds 2014

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

PEBBLES

144 machines8 GB RAM1 AMD Bobcat, 2 core, 1600 Mhz, 80 radeon cores(opencl)5 2T disks (Total > 1PB)1x1G ethernetEOL May 2015

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

PEBBLES2

1232 GB RAM1 Intel Xeon, 8 core, ht, 2 Ghz12 6T disks (Total 860 TB)2x10G ethernet (sfp+)expected May 2015

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

DISTRIBUTED STORAGE

Ceph, BeeGFS, HDFS, ...replication, erasure codes (raid levels)

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

ROCKS

144 machines16 GB RAM1 Intel i7 K2600, 4 core + ht, 3.4Ghz,1x2T disks1x1G ethernet1xIB (QDR) (40GB)EOL Dec 2015

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

BRICKS/STONES

16/16 machines256 GB RAM2 Intel Xeon E5 2650/2650 v2, 8 core + ht, 2/2.6 GHz,4x2T disks3x1G ethernet2xIB (QDR) (40GB)

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

SPECIAL

200 TB enclosures, 8GB/sSSD RAIDs

10 x Nvidia/AMD GPU

13 x Intel Phi 5100 (60 cores, 4 threads)

Using GPU/Phi is (still) Hard for a DBMS

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

GEMS

4 machines64 GB RAM2 Intel Xeon E5 2650 v2, 8 core + ht, 2.6Ghz24x.256 SSD3x1G ethernet2xIB (QDR) (40GB)

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

DIAMONDS

16 machines1 TB RAM4 Intel Xeon E5 4657, 12 core + ht, 2.4Ghz4x2T disks1x1G ethernet2x10G ethernet2xIB (QDR) (40GB)

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

SCILENS I, SOFTWARE

Fedora 20/22, Centos 7mpss 3.7Cuda 7.5openclicc, gcc, openmp, mpiceph fs

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

SCILENS II

Expected Hardware64 bit arm servers, low energylarge distributed memories, using numa connectneuromorphic co-processorsof the self intel/arm cpu + fpga boardsmicro servers, 10GB back planeNVMe, pci based fast, large flash storage

ProjectsNumaScale, back to ccNuma.HBPExaNeST

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

EXANEST

The ExaNeSt HPC project: Storage, Interconnect, CoolingStorage: fast, distributed in-node non-volatile memoryInterconnect: low-latency, unified compute & storage trafficPackaging: advanced, liquid-cooledApps: real, scientific and datacenterPrototype: 1000+ ARM coresfrom EuroServer: ARM nodes with UNIMEM addressspace & shared I/Ofrom ExaNoDe: Chiplets, Si Interposerwith ECOSCALE: Heterog. ARM+FPGAs

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

DBMS IMPLEMENTATION CHALLENGES

Elastic DBDBMS scheduler aware of distributed storage and cachingheterogeneous Hardware, multi cores, GPU, Phy, FPGA

Niels CWI

Scilens Infrastructure

Intro Design Hardware Software Future

IMPRESSION

Niels CWI

Scilens Infrastructure