Architecture and Design of MySQL Powered Applications

Peter ZaitsevMay 8, 2014

About Presentation

What's in your Toolbox ?

What Questions you should ask ?

Common MySQL Architecture

Patterns

2

LiveJournal Architecture 10 Years Ago3

10 Years ago we see

Load Balancers

FrontEnd vs BackEnd Web Server Separation

MemcacheD (Caching)

Global Database for Shared Data

Read Write Splitting

Sharding “Database Clusters”

4

History Lesson

MySQL is quite Mature

Battle tested Architecture Patterns

Most Problems have proven solutions

5

YOUR TOOLBOX NOW

Well… certain things have changed

6

Very Powerful Hardware

• 16GB to 512GBLots of

Memory

• 1000 IOPS to 100.000 IOPS

• 10ms latency to 0.1ms IO Latency

Very Fast Storage

• 4 Cores to 32 CoresMany CPU

Cores

7

Reasonable Software

•MySQL

•Linux Kernel

•File Systems

•Etc..

Can use this

Hardware Pretty Well

8

Cloud

Dynamic Scalability

Throw Away Server Mentality

Agility

Less involved Operations

9

Automation

Development Testing Deployment Operations

10

New Languages and Frameworks

More Agile

Less “system” code

More Client Side Development

Less Database Dependence

ORM

11

Multiple “Clients”

Desktop Web Browsers

Mobile Web Browsers

“Apps”

API

12

Not Only “MySQL”

• Memcache, Redis

• RabbitMQ

• Cassandra, MongoDB

• Hadoop

• Column Store Engines, Vertica

Right Tools

for Right Job!

13

More HA Options

Mature MySQL Replication

Percona XtraDB Cluster, Galera

MySQL Cluster

Proprietary cloud HA

14

Ready solutions for Replication Management

MHA

PRM

Continuent Tungsten

15

Making Sharding Better

• Clustrix

• MemSQL

• ScaleDB

Not Quite MySQL

• ScaleArc, ScaleBase, Tesora

• MySQL Proxy, MaxScale, ProxySQL

Proxy Solutions

• Vitess

• JetPants

• MySQL Fabric

OpenSource Frameworks

16

ARCHITECTURE QUESTIONS

17

Basics

Make some Right decisions Early

Your first Architecture will not be perfect

Some poor choices are very expensive to fix

18

Consultants are like Lawyers

You can waste a lot of money with them

Getting right advice on the right topics will save you a lot of trouble in the end

19

Dimensions to consider

Application Scale

High Availability

Team Experience

20

Approach to Scaling

Avoid Over-Engineering

Give yourself some

runway

Do Some Capacity Planning

21

High Availability

Real High Availability is Really Expensive

Medium Availability is getting more affordable

People are often leading cause of downtime

22

Team Experience

Chose Solutions which your Team can Support

Often Developers know little about Databases

Right Development, Operational Process Is a Key

Tools to Support your Process

• Percona Toolkit, Percona Cloud Tools

• Systematically find and review new queries

23

Simplicity

• Less complexity lower costsUse fewer

components when possible

• Monitor

• Troubleshoot

Each components needs experience

to

• “Follow the Crowd”

• Unless you have a reason not toThere is safety in

numbers

24

World of “Small” Applications

• Really

Majority of Applications are

Small

• Especially with support of caching

May well run on single database

instance

• Active intranet site for company with 200K employees

• Ecommerce site supporting $10M+ in Sales

Such “Small” Applications can

be significant

25

Lets do Some Math

Modern MySQL

Instance Handle 100K

QPS

20 Queries per User

Interaction

5K User Interactions Per Second

144M User Interactions a

Day (Considering

3x peak to average)

10M Daily Active users

(with Medium

Engagement)

26

PRACTICAL CHOICES

Basic Architecture Patterns

27

Baseline

Single MySQL Instance

No Caching

No HA

No Supplemental Technologies

28

Simple High Availability

HA with DBaaS (Amazon RDS)

HA with MySQL Replication

HA with Percona XtraDB Cluster

29

What do you need to Scale ?

Reads

Writes Data Size

30

Load Management Tricks

Move the load

In Time (Doing Batch Processing

at Night)

In Space (Use Slave for Reports)

31

Queueing

Used by Most high volume architectures

Deal with Load Spikes

Increase Reliability

Scale parts of the system which need to

Many solutions

• RabbitMQ

• Redis

• Gearman

32

Scaling Reads

Replication (and Read Write Splitting)

Caching (Memcache, Redis etc)

Percona XtraDB Cluster (Read Anywhere)

33

Scaling Writes

• Using Different Servers for Different Parts of Application

Functional Partitioning

• Horizontal Partitioning over many servers Sharding

34

Scaling Data Size

•Depends on the Application and Needs

•10TB good rule of thumb

How Much We Can

Practically Store in Single

MySQL Instance

these days

35

How Is it Possible

“Online” Schema Change

MySQL 5.6

Pt-online-schema-change (Percona

Toolkit)

Fast Backup

Percona Xtrabackup

MySQL Enterprise Backup

Modern Filesystem/Volume

Managers

Fast Network

10Gbps affordable

High Performance

Storage

Flash/SSD

Compression

Innodb

TokuDB

36

Summary

MySQL is Mature. Solutions Available for Many Problems

Make few choices right in the beginning

Hardware and Software advances allow to go a long way with simple architectures

37

38

Thank You!

Peter Zaitsevpz@percona.com