GPU Virtualization: Doing Much More with GPUs

transcript

GPU Virtualization: Doing Much More with GPUsMazhar Memon, CTO BitfusionSC16 Salt Lake City, UT

Quick Poll: Any GPU users?

GPU Users: Much variety

• Learners• Application developers• HPC Scientist• CAD/CAE• Artist + Designers• Data Analyst

One size doesn’t fit all

Manufacturing

Retail & Finance

Media & Entertainment

Pharma & Healthcare

Oil & Gas

Deep Learning

Variety of GPU Sizes

• TX1• GTX• Tesla• Quadro

Problem: How to do more with your (static) GPUs?

Virtualization 101

Server

Hypervisor

Server

Hypervisor

Virtualization: Support More Users or Applications on a Single Server

Many users, small problems

GPUGPU

One user, one big problemGPUGPUGPUGPU

GPUGPUGPUGPU

Large data, small device memory

GPUGPU

App requiremen

tAvailable memory

App demand >> GPU memory

GPU Virtualization on Steroids

Use your favorite GPUapplications as-is

Bitfusion Boost Layer

Your existing GPU infrastructure

Solve Small Problems Cheaply

GPUGPU

Slice GPUs into arbitrary fractionsMemory and process isolation

Available on Nimbix Today: $0.49 GPU instances

Solve Large Problems Dynamically

CPU-only Node

48 Cores3 TB Memory

72 TB SSD Storage

BoostMassive Virtual NodeGPUGPUGPUGPU

GPUGPUGPUGPU

Racks with GPUs

GPU GPUGPU GPU

Creating the largest virtual GPU machines on demand

GPUGPU

Host Memory

Solve Large Data Problems Efficiently

Available memory

Dynamic paging of GPU memory backed by host memoryWorks for non-Pascal GPUs as well

Monitoring and Managing GPUs Easily

• Use your favorite tools:All common tools e.g. nvidia-smi work across

virtual clusters

Handling Faults Automatically

GPUGPU

Failover to any other available GPU server uponCatastrophic, memory, intermittent faults

Bitfusion Boost: Software Stack

application

remote servers

local server

System view

Hardware

VM Hypervisor

Drivers

Operating system

User Space

Intercepts applications and applies a variety of rules including automatic scale-out, resource pooling, high availability, etc.

Hardware

VM Hypervisor

Drivers

Operating system

Hardware

VM Hypervisor

Drivers

Operating system

Open APIs

Custom APIs

Libraries

Application

Core Functions

Hardware

VM Hypervisor

Drivers

Operating system

Deploy on bare metal, containers, VMs. Secure, Portable, Frictionless

App Specific Instance Configurations as Machine

Images

Resource Pooling:• Consolidate use of compute resources• Increase utilization• Lower capital costs

Resource Provisioning:• Enforce CPU, memory, utilization quotas• Effect QoS policy and guarantees• Maximize utilization and reduce costs

High availability:• Detect failures at app level• Rollback, failover, error detection• Events for higher level reporting

Heterogeneous Offload:• Leverage HPC hardware• Interpose vendor libraries• Retarget hot functions to efficient specialized devices

Scale-out:• Distribute and load balance load across systems• Scale performance on demand• Take advantage of runtime optimizations

Advanced Profiling:• Understand application

demands of the datacenter• Fine-grained data provides

unique insight• Precise recommendations for

capacity planning

Deep Learning Caffe Deep Learning Torch

Deep Learning TensorflowMedia Transcoding

Rendering Scientific Computing

Boost: Add broad set of features to your application

http://www.bitfusion.io/boost-machine-images

Boost Available on Nimbix todayDeveloper-optimized machine configurations:

Learn more about Bitfusion Boost at boost.bitfusion.io