SOFTWARE-DEFINED SUPERCOMPUTER

OPENSTACK AND THE

Stig Telfer, Research Computing Services, Cambridge University

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

ABOUT ME

▸ HPC

▸ Distributed systems

▸ HPC

▸ Software-defined networking

▸ HPC

▸ OpenStack

▸ HPC on OpenStack

“…AT THE SC14 SUPERCOMPUTING CONFERENCE IN NOVEMBER, CRAY ANNOUNCED THAT IT IS WORKING WITH CAVIUM TO DELIVER CLUSTERS BASED ON THE 48-CORE VARIANT OF THE THUNDERX CHIPS TO INVESTIGATE THE FEASIBILITY OF USING THESE CHIPS TO RUN HPC WORKLOADS…”

The Next Platform February 27, 2015

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

HPC VS CLOUD

▸ Performance is king

▸ OS and software stack

▸ Network

▸ Storage

▸ Telemetry and analytics

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

CLOUD VS HPC

▸ Flexibility

▸ Self-service

▸ Consistency

▸ User-friendly

THE PROJECTCLOUD FOR HPC

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

High-SpeedNetwork

Outside World

Compute

ManagementNetwork

Compute

Compute

Compute

Compute

Compute

Management

Monitoring

Storage

…

FIRST ITERATION

▸ Started with OpenStack Havana

▸ Economy hardware

▸ Diskless compute nodes

▸ Neutron networking

▸ Ironic bare metal

▸ Platform-as-a-Service use case

MAKE LEMONADE

WHEN LIFE GIVES YOU LEMONS

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

HACKED INTO SHAPE

▸ Ironic power driver for smart PDUs

▸ Neutron driver for SNMP-enabled switches

▸ Fixes for multi-homed bare metal hosts

▸ HPC-like diskless compute node instances

High-SpeedNetwork

Outside World

Compute

ManagementNetwork

Compute

Compute

Compute

Compute

Compute

Management

Monitoring

Storage

…

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

Compute

Management

StorageNFS export

Instancemetadatacloud-init

Ephemeraltmpfs

Overlayrootfs

Read-only NFS

DISKLESS COMPUTE NODES

▸ NFS image exported read-only, common to all compute nodes

▸ Writeable ephemeral overlay mounted by initrd

▸ Instance metadata applied

▸ Largely implemented sideband of OpenStack

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

Compute

Management

Storage

Cinder attachiSCSI/iSER

Instancemetadatacloud-init

Ephemeraltmpfs

Overlayrootfs

Read-only block device

Image filesystem

BETTER DISKLESS COMPUTE NODES

▸ Root image is Cinder volume

▸ Ironic + Cinder support

▸ Use iSER for Cinder protocol

▸ Read-only attach of Cinder volume

▸ Ramdisk applies ephemeral overlay

▸ Multi-attach of Cinder volume

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

Storage

Compute

Workload

ManagementNetwork

Compute

Workload

High-SpeedNetwork

ManagementStorageWorkload ExternalWorkload Internal

OutsideWorld

NETWORKING FOR HPC

▸ Layer-2 protocols, eg RoCE

▸ RDMA is highly desirable

▸ Maximum latency dominates

WORKLOAD MANAGEMENTITS NOT HPC WITHOUT

OPENSTACK AND THE SOFTWARE-DEFINED SUPERCOMPUTER

ACHIEVING THE BEST OF BOTH WORLDS

▸ The flexibility of cloud without the performance overhead?

▸ The programming environment of HPC?

▸ A snowflake?

▸ What is next?

THANK YOUWATCH THIS SPACE…