MYSQL IN THE CLOUD – ENDLESS POSSIBILITIESGrant McAlister, Senior Principal Engineer

• Cloud computing is Internet-based computing, whereby shared resources, software and information are provided to computers and other devices on-demand, like a public utility. - Wikipedia

WHAT IS THE CLOUD?

• Build your own– AWS – EC2, EBS & S3– Rackspace – CloudServers & CloudFiles

• Relational Database Services– AWS RDS– FathomDB

WHAT IS MYSQL IN THE CLOUD?

BREAKING APART SCALING

CPU MEM

Storage IOPS

CPU

StorageIOPS

• Drive capacity is doubling every 18-24 months for the same price

• IOPS are not CHEAP and are not getting much cheaper

• You can now split IOPS from storage to reduce costs

STORAGE IS CHEAP!!!!

• Tape Sucks!– Slow and error prone– Humans are involved

• Cloud Storage– Disk based backups– Incremental backups – save time and money

• Recoveries that work!!!!

USABLE BACKUPS

• Multiple locations across the world– Reduce latency to customer

• Locations can have multiple datacenters– Increase durability and availability

LOCATION, LOCATION, LOCATION

• OLD– Typically measured in years– Almost never kill off a database

• NEW– Measured in hours or minutes– Application based creation and deletion of db

DATABASE LIFETIMES

• Standard DB Instance creation– Get a new host – weeks to months– Get the host setup with OS and software – hours to days– Setup db instance, monitoring and backups – hours to days

• Cloud DB Instance creation– From API calls to running DB instance – 5 to 10 minutes

• Demo deleting and creating 10 DB Instances

RAPID CREATION AND DELETION

• Automated Backups• Patching• Monitoring & Recovery of DB Instance and Hosts• Replication / HA

CLOUD DB SERVICE CAPABILITIES

• New methods of using databases• Allow us to build applications that would have been

difficult or impossible to do.– Scaling up or down based on workload– Large number of partitions or shards– A hierarchical storage database system– Testing

WHAT ARE ENDLESS POSSIBILITIES

ON DEMAND DAILY SCALING

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240

2

4

6

8

10

12

14Daily CPU Load

Hour

Load 25% Savings

ON DEMAND YEARLY SCALING

1 5 9 13 17 21 25 29 33 37 41 45 490

2

4

6

8

10

12

Yearly CPU Load

Week

Load

50% Savings

VERTICAL PARTITIONING

Very Large CPU

Customers

Orders

Addresses

Order Items

Catalog

Very Large CPU Very Large CPU Large CPU

Customers

Addresses

Orders

Order Items

Catalog

Before After

SHARDING HORIZONTALLY

Very Large CPU

Customers

Orders

Addresses

Order Items

Before After• Hashed by CustomerID

Very Large CPU

Customers

Orders

Addresses

Order Items

Very Large CPU

Customers

Orders

Addresses

Order Items

Very Large CPU

Customers

Orders

Addresses

Order Items

Very Large CPU

Customers

Orders

Addresses

Order Items

• Cloud Service Provides– Provisioning and management of large numbers of

instances– Backup/Restore to facilitate making new partitions– Allows you to both scale up and scale out – Should only have to scale for CPU/MEM

INITIAL PARTITIONING & RE-SHARDING

DATABASE OVER TIME

Storage

Small Server

Storage

Large Server

Storage

Very Large Server

Storage

Extra Large Server

Small

HSM DB SYSTEM

Application Layer

Large CPU

Week 1

Large CPU

Week 1

Week 2

Small

Week 1

Week 2

Week 3

Small

Week 1

Week 2

Week 3

Week 4

Small

Week 1

Week 2

Week 3

Week 4

Week 5

Week 1

Week 2

Week 3

Week4

Week 5

Week 6

Give me Week1 data

Writes

Reads

TYPICAL TESTING SETUP

PROD

Very Large Server

Storage

TEST

Large Server

Storage

WEEKLY REFRESH

CLOUD TESTING SETUP

PROD

Very Large Server

Storage

Bob Test

Very Small Server

Storage

Ted Test

Very Small Server

Storage

Mary Test

Very Small Server

Storage

7 AM6 PM

DEMO TESTING 10 DIFFERENT CONFIGURATIONS

Test Driver

Instance TPCC-0

innodb_thread_concurrency=0

Test Driver

Instance TPCC-1

innodb_thread_concurrency=1

Test Driver

Instance TPCC-2

innodb_thread_concurrency=2

Test Driver

Instance TPCC-3

innodb_thread_concurrency=3

Test Driver

Instance TPCC-4

innodb_thread_concurrency=4

Test Driver

Instance TPCC-5

innodb_thread_concurrency=5

Test Driver

Instance TPCC-6

innodb_thread_concurrency=6

Test Driver

Instance TPCC-7

innodb_thread_concurrency=7

Test Driver

Instance TPCC-8

innodb_thread_concurrency=8

Test Driver

Instance TPCC-9

innodb_thread_concurrency=9

• The Cloud provides very powerful features • The new features can enable new use cases

• Still very early in cloud computing for databases• Just scratching the surface

CONCLUSIONS

THANK YOUgrant@amazon.com