(SOV202) Choosing Among AWS Managed Database Services | AWS re:Invent 2014

transcript

November 12, 2014 | Las Vegas, NV

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

OS installation

App optimization

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

OS installation

App optimization

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

OS installation

App optimization

OS patches

DB s/w patches

Database backups

Scaling

High availability

DB s/w installs

App optimization

Power, HVAC, net

Rack & stack

Server maintenance

OS installation

Power, HVAC, net

Rack & stack

Server maintenance

OS patches

DB s/w patches

Database backups

App optimization

High availability

DB s/w installs

OS installation

Scaling

Self-managed database AWS-managed database

You have full responsibility for

upgrades and backup

AWS provides upgrades, backup, and

failover as a service

You have full responsibility for security AWS provides high infrastructure

security, certifications; gives you tools

to ensure DB security

Full control over parameters of server,

OS, and database

Database is a managed appliance, so

you can easily automate

Replication is expensive, complex,

and requires a lot of engineering

AWS provides failover as a packaged

service

Amazon DynamoDB

Document and key-

value store

Amazon RDS

SQL database engines

Amazon ElastiCache

In-memory cache

Amazon Redshift

Data warehouse

Amazon DynamoDB

“By using DynamoDB, we

reduced delivery time for

video events to less than 50

milliseconds.”— Greg Nelson, VP of

Software Engineering

table items

Attributes

(name/value

pairs)

Hash key

(DynamoDB maintains an

unordered index)

Hash key

Range key

(DynamoDB maintains a

sorted index)

Hash key

Range key

LSI key

Choose which

attributes

to project (if any)

capacity units

Pay to bearer

on demand

1 write per sec

of up to 1KB

Pay to bearer

on demand

1 read per sec

of up to 4KB

Eventually consistent reads at 50% off!

One write capacity unit

One read capacity unit

CreateTable

UpdateTable

DeleteTable

DescribeTable

ListTables

PutItem

GetItem

UpdateItem

DeleteItem

BatchGetItem

BatchWriteItem

Manage tables

Query specific

items OR

scan full table

“Select”, “insert”,

“update” items

Bulk select or

update (max 1MB)

Elastic Load

Balancing Amazon EC2

App Instances

Clients

DynamoDB

Business logic

How DynamoDB billing works

Monthly

bill = GB +

Assumes DB instance accessed only from AWS region

Further details at http://aws.amazon.com/dynamodb/pricing/

≈ 5 GB * $0.25 +

21 * 720 hrs * $0.0065/10 +

35 * 720 hrs * $0.0065/50

≈ $14.36

Storage consumed(plus 100 bytes per item)

Charge for

write capacity units

per hour

+Charge for

read capacity units

per hour

How DynamoDB billing works

Monthly

bill = GB +

Assumes DB instance accessed only from AWS region

Further details at http://aws.amazon.com/dynamodb/pricing/

≈ 5 GB * $0.25 +

21 * 720 hrs * $0.0065/10 +

35 * 720 hrs * $0.0065/50

≈ $14.36

Storage consumed(plus 100 bytes per item)

Charge for

write capacity units

per hour

+Charge for

read capacity units

per hour

Before

AWS Free

Usage Tier

Amazon RDS

"We were able to go from

concept to delivered product

in about six months with just

a handful of engineers." — Greg Scallan, Chief Architect

How Amazon RDS delivers high performance

How Amazon RDS backups work

Choose Multi-AZ

Choose Read Replicas for greater scalability

Native

Choose cross-region snapshot copy for even greater durability,

ease of migration

Choose cross-region read replicas for enhanced data locality, even

more ease of migration

http://aws.amazon.com/rds/aurora/preview/

Amazon DynamoDB Amazon RDS

How Amazon RDS billing works

Monthly

bill = GB+

Assumes DB instance accessed only from Amazon EC2

Further details at http://aws.amazon.com/rds/pricing/

= 720 hrs * $0.37 + 100 GB * $0.115

= $279.05

db.m3.xlarge; MySQL;

Oregon; Single-AZ;

On-Demand

100 GB

General Purpose

4 vCPUs;

15 GiB

ElastiCache

2U, Inc.“ElastiCache helps us

specifically a lot around

our social and

collaborative tools…It just

works. We don’t even

know its there.”— James Kenigsberg

Chief Technology Officer

Elastic Load

Balancing

Amazon EC2

App Instances

Clients

Amazon RDSAmazon

ElastiCache

How ElastiCache billing works

Monthly

bill = N ×

Further details at http://aws.amazon.com/elasticache/pricing/

= 4 nodes * 720 hrs * $0.182

= $524.16

Standard large; Oregon;

On-Demand

2 vCPUs;

7.1 GB

Amazon Redshift

“By migrating to

Amazon Redshift,

we can run our queries

98% faster and 80% cheaper

than we could before.

— John O’ Donovan, CTO

Traditional Enterprise DW Companies with Big Data SaaS Companies

Leader

Compute nodes

Existing business

intelligence tools

PostgreSQL

JDBC/ODBC

Amazon S3

Amazon DynamoDB

AWS Data Pipeline

Amazon EMR

• With row storage you do

unnecessary I/O

• To get total amount, you have

to read everything

ID Age State Amount

123 20 CA 500

345 25 WA 250

678 40 FL 125

957 37 WA 375

• With column storage, you

only read the data you need

ID Age State Amount

123 20 CA 500

345 25 WA 250

678 40 FL 125

957 37 WA 375

analyze compression listing;

Table | Column | Encoding

---------+----------------+----------

listing | listid | delta

listing | sellerid | delta32k

listing | eventid | delta32k

listing | dateid | bytedict

listing | numtickets | bytedict

listing | priceperticket | delta32k

listing | totalprice | mostly32

listing | listtime | raw

• COPY compresses

automatically

• You can analyze and override

• More performance, less cost

• Column storage

• Data compression

• Zone maps

• Direct-attached storage

10 | 13 | 14 | 26 |…

… | 100 | 245 | 324

375 | 393 | 417…

… 512 | 549 | 623

637 | 712 | 809 …

… | 834 | 921 | 959

• Track the minimum and

maximum value for each block

• Skip over blocks that don’t

contain relevant data

• Column storage

• Data compression

• Zone maps

• Direct-attached storage

DW.HS1.8XL:

• > 2 GB/s scan rate

• Optimized for data processing

• High disk density

DW.HS1.XL:

Dense Storage Node (dw1.xlarge)

2 TB, 16 GB RAM, 2 cores

Dense Compute Node (dw2.large)

0.16 TB, 16 GB RAM, 2 cores

Single Node (2 TB)

Cluster 2-32 Nodes (up to 64 TB)

8XL Dense Storage Node (dw1.8xlarge)

16 TB, 128 GB RAM, 16 cores, 10 GigE

8XL Dense Compute Node (dw2.8xlarge)

2.56 TB, 128 GB RAM, 16 cores, 10 GigE

Cluster 2-100 Nodes (up to 1.6 PB)

Note: Nodes not to scale

How Amazon Redshift billing works

Monthly

bill = N ×

Further details at http://aws.amazon.com/rds/pricing/

= 4 nodes * 720 hrs * $0.25

= $720

dw2.large; Oregon;

On-Demand

2 vCPUs;

15 GiB RAM;

0.16 TB SSD

http://aws.amazon.com/redshift/free-trial

http://aws.amazon.com/redshift/partners/

Amazon DynamoDB

Document and key-

value store

Amazon RDS

SQL database engines

Amazon ElastiCache

In-memory cache

Amazon Redshift

Data warehouse

Pay only for what you use

No up-front cost

Fully managed services

AWS handles installs,

patching, restarts

Easy to scale

Grow as you need

Designed for use with

other AWS services

Amazon

CloudWatchAmazon

Amazon

Data Pipeline

http://aws.amazon.com/marketplace

Service Free every month Term

Amazon

DynamoDB

25 GB of storage

25 units of write capacity

25 units of read capacity

Indefinite

Amazon

ElastiCache

750 Micro Cache Node

instance hours

First 12 months of

AWS account

Amazon RDS 750 Micro DB instance hours

20 GB of DB Storage

20 GB for Backups

10 million I/O operations

First 12 months of

AWS account

Please give us your feedback on this session.

Complete session evaluations and earn re:Invent swag.

http://bit.ly/awsevals