MARK RYLAND - DARPA · (Re)invent and simplify; can we do better? Data center CPUs are powerful and...

MARK

DIRECTOR, OFFICE OF THE CISO

RYLAND

AMAZON WEB SERVICES

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Hardware Innovation in (and around) the AWS CloudMark RylandDirector, Office of the [email protected]


AgendaIn the cloud:

Amazon EC2 Nitro architecture

A1 instances

AWS Inferentia system

Around the cloud: Outposts

Snowball family

AWS Robomaker, Greengrass, and FreeRTOS


Intel mainboardAmazon EC2 CR1: January 2013Traditional software-based virtualization

Virtual machine monitor is relatively simple

Device models and privileged OS are not

Amazon Linux(dom0)

cr1.8xlarge(domU/guest)

Amazon EBS volumes

DM

Instance storage

VPC networking

Oth

er s

oftw

are

DM

DM

DM

Xen


Beginning the Nitro journey(Re)invent and simplify; can we do better?

Data center CPUs are powerful and expensive; it is wasteful to use them as acceleratorsDevice models compete for CPU and system resources; jitter is hard to avoidDom0 OS is a big, complicated piece of software and a convenient landing zone

Apply microservices and building block concepts to simplify development, enhance quality, and speed up innovation?Use specialized hardware for acceleration and increased security?


Intel mainboardStep by stepMulti-year process to decompose the system

Device model by device model

Learnings along the way

Amazon EBS volumes

DM

Instance storage

VPC networking

DM

Oth

er s

oftw

are

DM

DM

Amazon Linux(dom0)

cr1.8xlarge(domU/guest)

Xen


PCIe bus

AmazonRDS

m5d.24xlarge

Nitro hypervisor (KVM-based)

Amazon EBS volumes

Instance storage

Intel (or AMD or Graviton) mainboardNitro architecture

ENA

Private network

Nitro controller & other Nitro computers


PCIe bus

AmazonRDS

Nitro hypervisor (KVM-based)

Instance storage

MainboardNitro architecture

ENA

Private network

m5d

.4xl

arge

m5d

.4xl

arge

m5d

.4xl

arge

m5d

.4xl

arge

m5d

.4xl

arge

m5d

.4xl

arge


Amazon EBS volumes


PCIe bus

AmazonRDS

M5d.metal instance type

Instance storage

MainboardNitro architecture

ENA

Private network


Amazon EBS volumes


PCIe bus

AmazonRDS

Instance storage

Mainboard

Amazon EBS attach volume APIUser calls Amazon EC2 API endpointInternal microservices send command to control planeControl plane sends command to Nitro controller

Nitro controller sends command to EBS controllerEBS controller sends hot-plug event for PCIe deviceNVMe device (emulated) shows up on the bus

ENA


Amazon EBS volumes


Nitro benefitsSecurity benefits

Nitro controller provides root of trust, scans mainboard to validate all firmware is in a known good stateHardware acceleration allows for full line-rate AES-256 encryption of all storage (EBS and instance) and networking (“N” instance types – 100Gbp/s!)Encryption keys stored in Nitro hardware

PerformanceFar more consistent performance everywhere100Gps/s networking, Clos topologies, and Elastic Fabric Adaptor (EFA) allows massive clusters with very low latency and non-over-subscribed throughput using commodity networking technologies

Node-level horizontal scaling of networking!


AgendaIn the cloud:


A1 instances



Snowball family



64-bit Arm Neoverse

AWS Graviton Processor


First instance powered by AWS Graviton Processor

Up to 45% cost savings AWS Graviton Processor with 64-bit Arm Neoverse cores and custom AWS silicon

Amazon EC2 A1

Lower cost

Run scale-out and Arm-based workloads in the cloud

Maximize resource efficiency with AWS Nitro

SystemFlexibility and choice for

your workloads


Software ecosystem

Containers

Most Docker official images support arm64

Tools

ECSAvailable

today

EKSPublic preview

available today

OSVs and ISVsAmazon Linux 2

16.04 and newer

Red Hat Enterprise Linux

7.6, 8.0

Linux Enterprise Server 15

+ Fedora, Debian 9.8, NGINX Plus


AgendaIn the cloud:


A1 instances



Snowball family



AWS Inferentia chip and system: motivation

Deep Neural Network (DNN) is achieving state-of-the-art results in many application domainsDNN demands orders of magnitude more computation than traditional techniquesCustomers require low-cost, high-throughput, low-latency deep learning inference


Purpose-built for cloud scale acceleration

Ease of application integrationSupport Elastic Inference, SageMaker, ECS, EKS; TensorFlow, MxNet, PyTorch, etc.

Leverage ahead-of-time compiler to achieve high performanceLow-overhead runtime to interface with deviceStandard profiler / debugger to provide actionable user feedbackAutomatically offload complex operator to host

© 2019 Annapurna Labs. C fid ti l


AWS-Inferentia deliverables and status

Built from scratch - chip, hardware, compiler, runtimeOperates at 1/4 the power, 1/10 the costPrivate beta: Oct 2019User guide, tutorial, model zoo, forum to aid migration to AWS-Inferentia


AgendaIn the cloud:


A1 instances



Snowball family



AWS Outposts: delivering AWS on-premises

Traditional Enterprise Applications

Low Latency Applications

Local Data Processing Applications


Get started in 3 steps

OrderOrder via the AWS

Management Console or API

ConnectConnect to power

and network

LaunchUse standard AWS APIs or

Management Console to launch EC2 instances, RDS instances,

SM instances, and so on


Launch local AWS services, reach back to regions

• Supports multiple accounts; multiple VPCs

• Amazon EC2, EBS, VPC (new subnet(s))• Amazon Relational Database Service

(RDS)• Amazon ECS and EKS• Amazon EMR, SageMaker, ElastiCache,

etc.• Full network access to regional services

via PrivateLink (modulo latency)• Full access to local network

• ElasticIP-style 1:1 NATing• Or DX-style full Layer 3 routing w/


Bringing AWS On-premises

• Same Programming Interface – Standard AWS CLI and SDK

• Same Functionality – Fully-featured, fully-managed AWS services

• Same Deployment Path – Same deployment pipeline as in the cloud

• Same Monitoring & Automation – Same metrics, reporting, operational tools

• Same Infrastructure – Same secure Nitro system with AWS-designed hardware and virtualization, no more ‘software lifecycle’ management

• Same Performance – Same EC2 instances and EBS volumes


AgendaIn the cloud:


A1 instances



Snowball family



Moving large volumes of data over the internet can take years – we ship secure physical devices to you to transfer your data at the source before shipping it back for bulk import to the cloud.

The cloud is not always accessible from remote locations where connectivity is limited or intermittent –deploy ruggedized devices at the edge with local storage and compute capacity to process data without network dependencies.

Traditional shipping of conventional hard drives is laborious and error prone – our E-Ink shipping label and chain of custody tracking simplifies logistics at scale.

Snow family introductionWhat is the customer problem?

AWS Snowball

AWS Snowball Edge


AWS Snowball AWS Snowball Edge AWS Snowmobile

• 50 or 80TB storage capacity

• 10GE networking

• Data encryption end-to-end

• Chain of Custody, Tamper Detection

• Rugged 8.5 G impact case

• Rain and dust resistant

• S3 Data Import

• Small file Batching

• Snowball command line

• S3 Adapter runs on workstation

• 42/100TB storage capacity (s3)

• 10/25/40GE networking


• Rugged 8.5 G impact case

• Chain of Custody, Tamper Detection

• Rain and dust resistant

• S3 Data import

• NFSv4 Server

• Clustering

• AWS Greengrass

• EC2/AMI support for edge computing

• GPU options

• Exabyte-scale storage in a 45ft container (90PB s3/Glacier/EBS)

• 10/25/40GE networking


• S3/Glacier Data import

• Dedicated security personnel

• GPS tracking, alarm monitoring, 24/7 surveillance, and optional additional security

Data Transfer Only Data Transfer & Edge Compute 20+ PB Data Transfer

AWS Snow Family portfolio


AgendaIn the cloud:


A1 instances



Snowball family



Software[for_now]@the_edgeAWS Robomaker, Greengrass, and FreeRTOSRobomaker

Commercial-grade version of Robot Operating System (ROS)Rich set of supporting clould services: IDE, hosted simulation environment, etc.

GreengrassOpen source implementation of Lambda run-time for more powerful IoT devicesIntegrated with local capabilities, including GPU/inference integrationIntegrated with AWS IoT Service

FreeRTOSAWS has taken on commercial stewardship of FreeRTOSRich set of security enhancementsIntegrated with AWS IoT Service

Thank you!


Mark [email protected]

Date post:	09-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

MARK RYLAND - DARPA · (Re)invent and simplify; can we do better? Data center CPUs are powerful and...

Documents