Webscale ComputingMike Culver

Amazon Web Services

Amazon Has Three Parts

1

2

3

Ideas Meet Pagers

What if Gravity Sets In?

How Do You Survive This?

Don’t Do the Muck

What is Cloud Computing?

o Remote computing capacityo On-demando Infinite scaleo XML Web services

Fallacies of Distributed Computing

1. The network is reliable.

2. Latency is zero.

3. Bandwidth is infinite.

4. The network is secure.

5. Topology doesn't change.

6. There is one administrator.

7. Transport cost is zero.

8. The network is homogeneous.

Wikipedia.com

2001 2002 2003 2004 2005 2006 2007

Bandwidth Consumed byAmazon Web Services

Bandwidth Consumed byAmazon’s Global Websites

2008

Amazon Web Services Are...

Building block services that allow developers to innovate and make money

Infrastructure As a ServiceAmazon Simple Storage Service

Amazon Elastic Compute Cloud

Amazon Simple Queue Service

Amazon SimpleDB

Commerce As a ServiceAmazon Flexible Payments Service

Fulfillment Web Service

People As a ServiceAmazon Mechanical Turk

Alexa Web ServicesAlexa Web Information Service

Alexa Top Sites

Alexa Site Thumbnail

Alexa Web Search Platform

Amazon Simple Storage Service (Amazon S3)

Highly scalable data storage in-the-cloud

Programmatic access via web services API

Simple to get going, simple to use

Highly available and durable

Pay-as-you-go:

Storage: $0.15 / GB / month

Data Transfer: starts at $0.17 / GB

Requests: nominal charges

Billions of Objects Stored

.8 Billion

5 Billion

10 Billion

Q3 06 Q1 07 Q2 07

14 Billion

Q4 07

18 Billion

Q1 08

22 Billion

Q2 08

Amazon S3 Namespace

Amazon S3

bucket bucket

object object objectobject

bucket

object object

Amazon S3 Namespace

Amazon S3

mculver-images media.mydomain.com

Beach.jpg img1.jpg img2.jpg2005/party/hat.jpg

public.blueorigin.com

index.html img/pic1.jpg

Internet Backup on a StickOr Stick Backed Up on the Net?

Open Source Backup

1 TB: Basecamp and Campfire Tools

Amazon Elastic Compute Cloud(Amazon EC2)

Resizable compute capacity in the cloud

Obtain and boot new server instances in minutes

Quickly scale capacity, up or down, as your computing requirements change

Full root access to a blank Linux machine

Simple Web service management interface

Changes the economics of computing

You just lost

customers

You just lost

customers

Predictions Cost Money

InfrastructureCost $

time

LargeCapital

Expenditure

LargeCapital

Expenditure

PredictedDemand

TraditionalHardware

ActualDemand

AutomatedVirtualization

Resilient Infrastructure Requires Physical Isolation

Availability Zone A Availability Zone B

Availability Zone C

Three Flavors of Amazon Machine Images

Public AMIs: Use pre-configured, template AMIs to get up and running immediately. Choose from Fedora, Movable Type, Ubuntu configurations, and more

Private AMIs: Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings

Paid AMIs: Set a price for your AMI and let others purchase and use it (Single payment and/or per hour)

Virtual Machine Choices

Standard High CPU

Small Large Extra Large

Medium Extra Large

Bits 32 64 64 32 64

RAM 1.7 GB 7.5 GB 15 GB 1.7 GB 7 GB

Disk 160 GB 850 GB 1690 GB 350 GB 1690 GB

EC2 Compute Units

1 4 8 5 20

I/O Performance

Medium High High High High

Firewall Yes Yes Yes Yes Yes

Per Hour $0.10 $0.40 $0.80 $0.20 $0.80

$5 or $30,000 For a Prototype?

$30,000 worth of in-house servers $30,000 worth of in-house servers

DISA paid a total of $5DISA paid a total of $5

Build Your Own Virtual ISP

Operating Systems as a Service

4 TB Data 100 Nodes 11 Million PDFs

100 instances x 24 hours x $0.10 / Hr = $240

From 50 to 3400 Instances in 3 Days

Amazon Simple Queue Service(Amazon SQS)

A distributed queue in the cloudUsed for storing messages traveling between computersReliable

Runs within Amazon's high-availability data centersMessages are stored redundantly across multiple servers and locations

Scalable to millions of messages a daySimple: Only 6 methodsPlatform agnosticProvides access control and message locking

Amazon SimpleDBLimited Beta

Databases Should Just WorkBut They’d Better Scale…

Easy: no administration

Web Service API

Flexible (no schemas)

Scalable: create new domains as

Your data grows

Request throughput increases

Durable

Multiple nodes

Replicated data centers

Amazon SimpleDB

item description color material

123 Sweater Blue, Red

456 Dress shirt White, Blue

789 Shoes Black Leather

PUT (item, 123), (description, Sweater), (color, Blue), (color, Red)

PUT (item, 456), (description, Dress shirt), (color, White), (color, Blue)

PUT (item, 789), (description, Shoes), (color, Black), (material, Leather)

QueryDomain = MyStore

[‘description’ = ‘Sweater’]

Learn More About AWS

Explore http://aws.amazon.com

Read our blog at http://aws.typepad.com

Email me at mculver@amazon.com

Demo