© Copyright 2015. Apps Associates LLC. 1
Deploying High-Performance Database Services
Amazon Aurora
November 5 2015
Bill Stewart Founder and CEO
PetTrax
Lynn Ferrante BDM, Databases
AWS Business Development
Kshitij Kumar CTO
Apps Associates
© Copyright 2015. Apps Associates LLC. 2
Agenda
• About Amazon Aurora RDS
• PetTrax’s move to Aurora
• Migration Best Practices
© Copyright 2015. Apps Associates LLC. 3
Amazon RDS Aurora
Lynn Ferrante BDM, Databases
AWS Business Development
© Copyright 2015. Apps Associates LLC. 4
Agenda
• What is Amazon RDS
• What is RDS Aurora
– Value Proposition
– Benefits
– Performance and Scalability
© Copyright 2015. Apps Associates LLC. 5
Amazon Aurora Summary
• MySQL compatible -existing MySQL applications can run without changes
• Fast – up to 5 times the throughput of standard MySQL
• Available and Durable--replicates 6 copies of data across 3 availability zones
• Highly Scalable -- scale up to 32 vCPUs,244GiB Memory with 15 replicas
• Highly Secure-Isolates data within VPC & encrypt data in-transit and at-rest.
• Inexpensive-- one tenth of the cost of the high end commercial databases
© Copyright 2015. Apps Associates LLC. 6
Fastest growing service
in AWS history
Amazon Aurora Customer Adoption
© Copyright 2015. Apps Associates LLC. 7
• Create a database in minutes
• Automated patching
• Push-button scale compute
• Automated continuous backups to S3
• Instantly create user snapshots
• Automatic failure detection and failover
• Read Replicas as failover targets
• Minimal replica lag
• Automatically scale up to 64 TB
• Pay only for storage you use
• 6 copies of data in 3 AZs
Amazon Aurora’s Recipe for Success
© Copyright 2015. Apps Associates LLC. 8
Write-only workload
1,000 connections
Query cache (default on for Amazon Aurora, off for MySQL)
-
10
20
30
40
50
60
70
10 100 1,000 10,000
Tho
usa
nd
s o
f W
rite
s p
er
Seco
nd
Number of Tables
Write Performance and Table Count
AuroraMySQL on I2.8XLMySQL on I2.8XL with RAM DiskRDS MySQL with 30,000 IOPS (Single AZ)
Up to
11x faster
Writes Scale with Table Count
© Copyright 2015. Apps Associates LLC. 9
OLTP Workload
Variable connection count
250 tables
Query cache (default on for Amazon Aurora, off for MySQL)
-
20
40
60
80
100
120
50 500 5,000
Tho
usa
nd
s o
f W
rite
s p
er
Seco
nd
Concurrent Connections
Write Performance and Concurrency
Aurora RDS MySQL with 30,000 IOPS (Single AZ)
Up to
8x faster
Better Concurrency
© Copyright 2015. Apps Associates LLC. 10
OLTP workload
1,000 connections
250 tables
Query cache on/off tested
-
50
100
150
200
250
300
350
400
100/0 50/50 0/100
Tho
usa
nd
s o
f O
pe
rati
on
s/Se
con
d
Read/Write Ratio
Performance with query cache on and off
Aurora without CachingAurora with CachingRDS MySQL;30,000 IOPS (Single AZ) - without cachingRDS MySQL;30,000 IOPS (Single AZ) - with caching
R/W Ratio
Amazon Aurora
Without Caching
Amazon Aurora With
Caching
RDS MySQL 30K IOPS Without Caching
RDS MySQL 30K IOPS
With Caching
100/0 160,000 375,000 35,000 19,000
50/50 130,000 93,000 24,000 20,000
0/100 64,000 64,000 16,000 16,000
Caching Improves Performance
© Copyright 2015. Apps Associates LLC. 11
Write workload
250 tables
Query cache on for Amazon Aurora, off for MySQL (best settings)
2.6 3.4 3.9 5.4
1,000 2,000 5,000 10,000
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
Updates per Second
Re
ad R
ep
lica
Lag
in M
illis
eco
nd
s
Read Replica Lag
Aurora RDS MySQL;30,000 IOPS (Single AZ)
Updates/ Second
Amazon Aurora
RDS MySQL 30K IOPS
(Single AZ)
1,000 2.62 ms 0 s
2,000 3.42 ms 1 s
5,000 3.94 ms 60 s
10,000 5.38 ms 300 s
Replicas have up to 400 times less Lag
© Copyright 2015. Apps Associates LLC. 12
• 4 client machines with 1,000 threads each
WRITE PERFORMANCE READ PERFORMANCE
• Single client with 1,000 threads
• MySQL SysBench
• R3.8XL with 32 cores and 244 GB RAM
SQL Benchmark Results
© Copyright 2015. Apps Associates LLC. 13
Connecting Pets and Families
Bill Stewart (617) 875-7360 [email protected] www.pettraxinc.com
© Copyright 2015. Apps Associates LLC. 14
Agenda
• Overview
• Problem
• Solution
• The AWS Advantage
• Apps Associates Partnership
• Beta Solution
• Summary
© Copyright 2015. Apps Associates LLC. 15
Ricky – The Gobbler Teddy – The Nibbler
Overview Our Story
© Copyright 2015. Apps Associates LLC. 17
Problem Pet families lead busy lives
• Out of the home much of the day
– Difficult to keep track of diet and exercise
• Multiple caregivers in the home
– People err on side of caution and end up overfeeding
• Need to work late tonight
– Leave out a big bowl of food
• Pet seems to be OK, no need to go in for check-up
– Only going to vet when health issue is advanced
• Multi-pet households
– Problem grows exponentially for each additional pet in home
© Copyright 2015. Apps Associates LLC. 18
Solution Intelligent feeding, monitoring, and notifications
• Intelligent feeding
– Pet’s ability to eat must be authorized and automated
– Authorization must evaluate multiple criteria
• Food type allowed
• Times during the day allowed
• Daily & feeding-session calorie allowance
• Intelligent monitoring
– Track all pet interactions
• Feeding/drinking sessions: duration, time of day, calories
• Feeding attempts: at own feeder and other pet’s feeder
• Level of activity: sleeping, moving
• Intelligent notifications
– Ricky just finished eating. 45 calories consumed; all normal
– Teddy has not eaten in 10 hours. This is not normal; check on Teddy
– Health concern! Bailey’s activity pattern has deviated significantly over the past 8 hours; check on Bailey
© Copyright 2015. Apps Associates LLC. 19
Solution Intelligent Feeder – Alpha Version
RFID Reader for pet identification
Motion Sensor
Food Storage and Dispensing
Camera
Retractable Food Tray
Temp/Humidity Sensor
Processor, Memory, I/O, and Comm
© Copyright 2015. Apps Associates LLC. 20
Solution Customized for any pet household
• Single cat or single dog homes
• Multi-cat or multi-dog homes
– Same food (same feeder)
– Different food (multiple feeders)
• Mixed cat and dog homes
– Multiple feeders
© Copyright 2015. Apps Associates LLC. 21
Solution What’s missing?
Connected Feeder
Scalable Compute Resources
Mobile Application
© Copyright 2015. Apps Associates LLC. 22
The AWS Advantage Why leverage cloud infrastructure?
• Up and running in a day
– Compute
– Database
– Storage
– Networking
– Management Tools
– Security and Identity
• Low start up costs
– Low monthly cost after first year
– Only pay for what you need
• Focus on your business
© Copyright 2015. Apps Associates LLC. 23
The AWS Advantage Add services and scale infrastructure as you grow
• Additional AWS services that may benefit PetTrax
– Analytics
– Internet of Things (IoT)
– Mobile Services
– Application Services
• Scale up existing infrastructure
– Supporting company growth
– Only pay for what you need
• Leverage 3rd party integrators
– Apps Associates and others
© Copyright 2015. Apps Associates LLC. 24
Apps Associates Partnership AWS Premier Consulting Partner
• Infrastructure sizing and deployment – Compute resources – Web (Apache) – Application (Tomcat) – Database (MySQL Aurora) – Messaging (MQTT)
• Application development – server – Java REST services – Database design and optimization – MQTT configuration
• Application development – mobile – iOS – Android
© Copyright 2015. Apps Associates LLC. 25
Migrating to Aurora
• Preparing for the future • MYSQL Version 5.6 • Stood up a separate application stack for Aurora to
avoid interference with ongoing development • Migration done in 7 days • Changed one line of code in the application
– Database connection string
• Testing – simulated load on application with Aurora • Everything worked in Aurora as in MYSQL • De-commissioned MYSQL technology stack • Tested failover and recovery • Ready for the future
© Copyright 2015. Apps Associates LLC. 26
Beta Solution 6 months later
Intelligent Feeders • Authorization-based feeder and monitor
Mobile Application
Cloud Infrastructure • Compute • Web Service • Application Service • Database • Messaging
© Copyright 2015. Apps Associates LLC. 27
Summary Convergence of forces pet awareness and enabling technology
• PetTrax business focus
– Pets are family members now
– Health of our loved ones is paramount
– Pet spending continues to increase
– Pet-Social Media interactions will drive sales
• AWS & Apps Associates technology focus
– Internet of Things having exponential growth
– Mobile monitoring is a reality
– Information will drive more services
– Robust/scalable AWS Cloud infrastructure enables growth
© Copyright 2015. Apps Associates LLC. 28
Migration to Aurora
Kshitij Kumar Global VP of BI/EPM and CTO Apps Associates LLC
© Copyright 2015. Apps Associates LLC. 29
About Apps Associates
• High Growth History
• HC Worldwide: 650
• Global Presence: US, Europe, Asia
• Market Leader in Several Segments of NE US with Rapid Expansion Across US
• Strong AWS Partnership
Market Leader: Oracle on AWS MSP Certification – within first 10 partners WW 1 of 7 partners for MMN Accelerator Program Recently selected for Partner Advisory Board &
DB Accelerator partner
© Copyright 2015. Apps Associates LLC. 30
• Migrate existing My SQL
– Lift and Shift
– MY SQL Version 5.6
– InnoDB Framework
– EC2 Classic to VPC
1. AWS Data Migration Service 2. AWS Schema Migration Service
Migrate Commercial Databases like Oracle
– Custom Application
– Web Application
– One Off Departmental Reporting Application
– Pilot project
Migrating to Aurora
© Copyright 2015. Apps Associates LLC. 31
Typical Migration Timeline (4 Weeks)
Assessment
( 2 days)
• Analysis of workload
• Analysis of current DB objects / code
• Understand Performance characteristics
• Develop TCO
• Prepare migration approach
Build and Test
( 8 + 4 days)
• Build/Configure AWS platform; security model
• Migrate from MyISAM to Innodb
• Setup replication from dev to Aurora Dev
• Complete migration
• Support testing
• Documentation & signoff
Production Migration & Support
( 5 days)
• Perform production migration
• Support Go-Live cutover process
• Provide post-production setup
• Configure maintenance jobs
• Knowledge Transfer & Sign-off
Part – I (2 to 3 weeks) Part – II (1 Week)
© Copyright 2015. Apps Associates LLC. 32
Define Design Move Develop Test Validate
Design Slice
Database
Application
Aurora
In-DB Code
Web App
Client Server
DB Objects (Tables etc.)
Packages etc.
Re-connect
Host on AWS
Ease of Migration
Backup / Recovery
Access
Perf
Security
Business
Success Criteria Business
Objective
Select DB
Test Criteria
Success Criteria Business
Objective
Demonstrate
Next Steps
Project Management / Oversight
Aurora Migration Approach
Co
re S
ervi
ces
Op
tio
nal
Ser
vice
s
DB
Tra
inin
g A
pp
Ch
ange
Tra
inin
g
© Copyright 2015. Apps Associates LLC. 33
Aurora Migration Utilities : Data Migration and Schema Conversion
Kshitij Kumar Global VP of BI/EPM and CTO Apps Associates LLC
© Copyright 2015. Apps Associates LLC. 40
Closing
Kshitij Kumar Global VP of BI/EPM and CTO Apps Associates LLC
© Copyright 2015. Apps Associates LLC. 43
Connect with Us
Web: www.appsassociates.com
Email: [email protected]
YouTube: www.youtube.com/user/AppsAssociates
LinkedIn: www.us.linkedin.com/company/Apps-Associates
Twitter: @AppsAssociates
Facebook: www.facebook.com/AppsAssociatesGlobal
Google+: www.plus.google.com/+AppsAssociatesGlobal/