Solutions Showcase: Machine Learning Workloads on Cray Systems€¦ · Solutions Showcase: Machine...

9/11/2016

1

Solutions Showcase: Machine Learning Workloads on Cray Systems

Mark Staveley

Machine Learning Research @ Cray

9/11/2016

2

Machine Learning

Analytics

Artificial Intelligence

eResearch 2016 - (c) Cray Inc

NoSQL

Streaming

GraphMachine

Learning

Some Deep

Learning

MapD

SQreamSome Machine

Learning

Deep Learning

CPU focus

GPU focus

SQL


9/11/2016

3

Create code from data

Insight Into Data

Emulate human mind



9/11/2016

4


Machine Learning Workflow


9/11/2016

5

DL Example: Radiograph Classifier

Training Images

Image Results:

Bounding Boxes

Classification Scores For:

Tumors

Broken bones

Lesions

Etc.

3) Trained model

scores images1) Designs Neural

Net Architecture

2) Training defines

linking weights

Developer /

Researcher

Important Points:

• Parallelism and local memory: GPUs

• Single / half-precision floats speed up

learning without loss of accuracy

• All-to-all communication is required

for scale and works well on Aries

• Scoring is separate from training


DL Example: Radiograph Classifier

Training Images

Image Results:

Bounding Boxes

Classification Scores For:

Tumors

Broken bones

Lesions

Etc.

3) Trained model

scores images1) Designs Neural

Net Architecture

2) Training defines

linking weights

Developer /

Researcher

Important Points:

• Parallelism and local memory: GPUs

• Single / half-precision floats speed up

learning without loss of accuracy

• All-to-all communication is required

for scale and works well on Aries

• Scoring is separate from training


9/11/2016

6

Artificial Intelligence Pipeline




9/11/2016

7

Why Cray ?


NoSQL

Streaming

GraphMachine

Learning

Some Deep

Learning

MapD

SQreamSome Machine

Learning

Deep Learning

CPU focus

GPU focus

SQL


9/11/2016

8

HPC-type Scale

and Problems

NoSQL

Streaming

GraphMachine

Learning

Some Deep

Learning

MapD

SQreamSome Machine

LearningDeep

Learning

CPU focus

GPU focus

SQL

Copyright 2016 Cray Inc.


HPC-type Scale

and Problems

NoSQL

Streaming

GraphMachine

Learning

Some Deep

Learning

MapD

SQreamSome Machine

LearningDeep

Learning

CPU focus

GPU focus

100+ Layer

Neural

Networks

Specific Scope

and Use Cases

Convergence Data chains & Analytics workflows

Teams & Tools

Platforms and Technologies

SQL



9/11/2016

9

HPC-type Scale

and Problems

NoSQL

Streaming

GraphMachine

Learning

Some Deep

Learning

MapD

SQreamSome Machine

LearningDeep

Learning

CPU focus

GPU focus

100+ Layer

Neural

Networks

Specific Scope

and Use Cases

Convergence Data chains & Analytics workflows

Teams & Tools

Platforms and Technologies

SQL



Map Reduce

N-body methods

Graph traversal

Graphical models

Dense and sparse linear algebra

Spectral methods

Structured and unstructured grids

Combinational logic

Dynamic programming

Backtrack and branch-and-bound

Finite-state machines

Basic statistics – simple Map Reduce implementation

Generalized n-body problems

Graph-theoretic computations

Linear algebraic computations

Optimizations – e.g., linear programming

Integration/machine learning

Alignment problems – e.g., BLAST


Landscape of Parallel Computing Research (Berkeley – 2006/2008)

State of Big Data: Use Cases and Ogre Patterns (NIST 2014)

9/11/2016

10

Components



Hardware

Data

OS Software

Management

Application Software

9/11/2016

11


Single Machine

Cloud

Cluster Cloud

Linux (CoreOS, CentOS, Ubuntu, RedHat)

Docker, Mesos, Kubernetes, Marathon, Fleet,

CNTK

NAS HDFS

TensorFlow MXNet Caffe Torch Warp DSSTNE


Single Machine

Cloud

Cluster Cloud


Docker, Mesos, Kubernetes, Marathon, Fleet

CNTK

NAS HDFS

TensorFlow MXNet Caffe Torch Warp

HW

Storage

OS

Mgmt

DSSTNEToolkits

9/11/2016

12

Where is Cray headed ?



Single Machine

Cloud

Cluster Cloud


Docker, Mesos, Kubernetes, Marathon, Fleet

CNTK

NAS HDFS

TensorFlow MXNet Caffe Torch Warp

HW

Storage

OS

Mgmt

DSSTNEToolkits

9/11/2016

13

CS-Storm

Three Focus Areas

• Computation

• Storage

• Analytics

● NVIDIA M40 24GB + CS-Storm● Variant of CS-Storm designed for Machine Learning (ML)

● 8 x M40 24GB / Machine ● (3072 CUDA cores + 24 GB GPU memory)

● 512 GB – 1 TB of RAM

● Up to 6 SSDs

● Dual Rail IB

● Key Features / Data Points● Workloads have seen a 1.2 – 1.8x improvement

● Optimizations in CUDA not available with K40 or K80

● Building Block for Deep Learning Compute Solution

● Power and Cooling Integrity



9/11/2016

14



9/11/2016

15




9/11/2016

16

Deep Learning @ Cray

Many supported frameworks, common challenges

Need for Data

Support for 3rd Party Libraries (e.g. MKL-DNN & cuDNN)

Highly scalable SGD codes are on the way and

some are already here:

● CNTK-1 Bit SGD and BlockMomentum

● TensorFlow distributed

● MXNET MPI+OpenMP parallelism


Cray Deep Learning Research


9/11/2016

17

Three Focus Areas

• Computation

• Storage

• Analytics

https://blogs.technet.microsoft.com/inside_microsoft_research/2015/12/07/microsoft-computational-network-toolkit-offers-most-efficient-distributed-deep-learning-computational-performance/


CNTK Scaling on XC-40s and Clusters

0

5

10

15

20

25

30

1 2 4 8 16 32

AV

G E

PO

CH

TIM

E

NUMBER GPUS

CNTK-1BIT SGD FFN BenchmarkXC(Aeries) vs Storm(IB)

Storm-K40 + Default OpenMPI Storm-K40 + Tuned OpenMPI XC-K40 Cray MPICH Defaults XC-K40 Cray MPICH Tuned


https://blogs.technet.microsoft.com/inside_microsoft_research/2015/12/07/microsoft-computational-network-toolkit-offers-most-efficient-distributed-deep-learning-computational-performance/

9/11/2016

18

Wrap Up


Summary

● Close Relationship between HPC workloads and Machine Learning

● Machine Learning is changing how we think about HPC (Data Movement, Workload Resiliency, etc.)

● Desire to make ML easy on Cray Systems (choice of HW and Toolkits)

● Fake Data / Small Data – negative influence on performance optimization targets

● Real Data Sets / Large Scale Workloads – challenges with libraries, implementations and HW

● Engineering and Research Development across Cray HW platforms & components

● Understanding and Learning (Different ML Toolkits + Data Movement + Network Performance Optimizations)


9/11/2016

19

Thank You

Mark_Staveley

[email protected]

Thursday – 10:30-10:50 – Industry State 1/2 –

Scaling Out Deep Learning Workloads on Cray

Systems

mailto:[email protected]

Date post:	20-May-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

Solutions Showcase: Machine Learning Workloads on Cray Systems€¦ · Solutions Showcase: Machine...

Documents