23
MYSQL IN THE CLOUD – ENDLESS POSSIBILITIES Grant McAlister, Senior Principal Engineer
MYSQL IN THE CLOUD – ENDLESS POSSIBILITIESGrant McAlister, Senior Principal Engineer
• Cloud computing is Internet-based computing, whereby shared resources, software and information are provided to computers and other devices on-demand, like a public utility. - Wikipedia
WHAT IS THE CLOUD?
• Build your own– AWS – EC2, EBS & S3– Rackspace – CloudServers & CloudFiles
• Relational Database Services– AWS RDS– FathomDB
WHAT IS MYSQL IN THE CLOUD?
BREAKING APART SCALING
CPU MEM
Storage IOPS
CPU
StorageIOPS
• Drive capacity is doubling every 18-24 months for the same price
• IOPS are not CHEAP and are not getting much cheaper
• You can now split IOPS from storage to reduce costs
STORAGE IS CHEAP!!!!
• Tape Sucks!– Slow and error prone– Humans are involved
• Cloud Storage– Disk based backups– Incremental backups – save time and money
• Recoveries that work!!!!
USABLE BACKUPS
• Multiple locations across the world– Reduce latency to customer
• Locations can have multiple datacenters– Increase durability and availability
LOCATION, LOCATION, LOCATION
• OLD– Typically measured in years– Almost never kill off a database
• NEW– Measured in hours or minutes– Application based creation and deletion of db
DATABASE LIFETIMES
• Standard DB Instance creation– Get a new host – weeks to months– Get the host setup with OS and software – hours to days– Setup db instance, monitoring and backups – hours to days
• Cloud DB Instance creation– From API calls to running DB instance – 5 to 10 minutes
• Demo deleting and creating 10 DB Instances
RAPID CREATION AND DELETION
• Automated Backups• Patching• Monitoring & Recovery of DB Instance and Hosts• Replication / HA
CLOUD DB SERVICE CAPABILITIES
• New methods of using databases• Allow us to build applications that would have been
difficult or impossible to do.– Scaling up or down based on workload– Large number of partitions or shards– A hierarchical storage database system– Testing
WHAT ARE ENDLESS POSSIBILITIES
ON DEMAND DAILY SCALING
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240
2
4
6
8
10
12
14Daily CPU Load
Hour
Load 25% Savings
ON DEMAND YEARLY SCALING
1 5 9 13 17 21 25 29 33 37 41 45 490
2
4
6
8
10
12
Yearly CPU Load
Week
Load
50% Savings
VERTICAL PARTITIONING
Very Large CPU
Customers
Orders
Addresses
Order Items
Catalog
Very Large CPU Very Large CPU Large CPU
Customers
Addresses
Orders
Order Items
Catalog
Before After
SHARDING HORIZONTALLY
Very Large CPU
Customers
Orders
Addresses
Order Items
Before After• Hashed by CustomerID
Very Large CPU
Customers
Orders
Addresses
Order Items
Very Large CPU
Customers
Orders
Addresses
Order Items
Very Large CPU
Customers
Orders
Addresses
Order Items
Very Large CPU
Customers
Orders
Addresses
Order Items
• Cloud Service Provides– Provisioning and management of large numbers of
instances– Backup/Restore to facilitate making new partitions– Allows you to both scale up and scale out – Should only have to scale for CPU/MEM
INITIAL PARTITIONING & RE-SHARDING
DATABASE OVER TIME
Storage
Small Server
Storage
Large Server
Storage
Very Large Server
Storage
Extra Large Server
Small
HSM DB SYSTEM
Application Layer
Large CPU
Week 1
Large CPU
Week 1
Week 2
Small
Week 1
Week 2
Week 3
Small
Week 1
Week 2
Week 3
Week 4
Small
Week 1
Week 2
Week 3
Week 4
Week 5
Week 1
Week 2
Week 3
Week4
Week 5
Week 6
Give me Week1 data
Writes
Reads
TYPICAL TESTING SETUP
PROD
Very Large Server
Storage
TEST
Large Server
Storage
WEEKLY REFRESH
CLOUD TESTING SETUP
PROD
Very Large Server
Storage
Bob Test
Very Small Server
Storage
Ted Test
Very Small Server
Storage
Mary Test
Very Small Server
Storage
7 AM6 PM
DEMO TESTING 10 DIFFERENT CONFIGURATIONS
Test Driver
Instance TPCC-0
innodb_thread_concurrency=0
Test Driver
Instance TPCC-1
innodb_thread_concurrency=1
Test Driver
Instance TPCC-2
innodb_thread_concurrency=2
Test Driver
Instance TPCC-3
innodb_thread_concurrency=3
Test Driver
Instance TPCC-4
innodb_thread_concurrency=4
Test Driver
Instance TPCC-5
innodb_thread_concurrency=5
Test Driver
Instance TPCC-6
innodb_thread_concurrency=6
Test Driver
Instance TPCC-7
innodb_thread_concurrency=7
Test Driver
Instance TPCC-8
innodb_thread_concurrency=8
Test Driver
Instance TPCC-9
innodb_thread_concurrency=9
• The Cloud provides very powerful features • The new features can enable new use cases
• Still very early in cloud computing for databases• Just scratching the surface
CONCLUSIONS
THANK [email protected]
