Scaling MySQL -- NY Code Camp

transcript

Insert Picture Here

MySQL Scaling§ Dave Stokes

MySQL Community Manager

David.Stokes@Oracle.com

@stoker

slideshare.net/davestokes

Safe Harbor Agreement

.The following is intended to outline our general product direction. It is

intended for information purposes only, and may not be incorporated into

any contract. It is not a commitment to deliver any material, code, or

functionality, and should not be relied upon in making purchasing decision.

The development, release, and timing of any features or functionality

described for Oracle’s products remains at the sole discretion of Oracle.

You want to get a bigger, badder MySQL --BUT what are the costs???

Budget

Complexity

Application

Opportunity

Sanity!

LAMP Stack https://en.wikipedia.org/wiki/LAMP_(software_bundle)

LAMP is an archetypal model of web service solution stacks, named as an acronym of the names of its original four open-source components: the Linux operating system, the Apache HTTP Server, the MySQL relational database management system (RDBMS), and the PHP programming language. The LAMP components are largely interchangeable and not limited to the original selection. As a solution stack, LAMP is suitable for building dynamic web sites and web applications.

Since its creation, the LAMP model has been adapted to other componentry, though typically consisting of free and open-source software. For example, an equivalent installation on the Microsoft Windows family of operating systems is known as WAMP.

Simple Stuff

Option 0

Run the Latest, Greatest MySQL

MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15%

--Make sure your config file matches your hardware

--ANALYSE TABLE – Maintenance of indexes, statistics

--Check for SLOW QUERIES, QUERIES w/o INDEXES

--TUNE QUERIES

This is the least complex, less upsetting, and usually lowest budget cost option

Option 0.1

Add in a Memcached Layer

● Cache for quick access

● Session Data

● Sports Scores

● Opens up NoSQL accessto InnoDB later

This is the second least complex, less upsetting, and usually lowest budget cost

option but does make you change your application to become 'cache aware'.

Insert Picture Here

What database serviceLevels do you Need toProvide?

UpTime versus Response Time versus Access

Things to be engineered/negotiated upfront

Up Time

– How much down time can you afford?

Response Time

– How long is too long to wait?

Access

– How many do you need to serve simultaneously.

Vertical Scaling

● Sometimes a newer, bigger server is the next logical step

● All eggs in newer basket

● Not limitless

● Tossing $$ at problem

● Transition pain

● May not openbottlenecks

Insert Picture Here

Simple Level 1 -Splitting Reads /Writes

Use MySQL Replication to Split Read/Writes

MySQL Replication

READ Splitting

Most Common Scaling Technique

● More complex environment

● Application Changes

● Easier with PHP & Java

● Low opportunity and sanity costs

● Less risk of one server stopping operations

https://blog.engineyard.com/2014/easy-read-write-splitting-php-mysqlnd

● PHP Developers Have it Easy!

● Use the mysqlnd_ms Native Driver Extension

– Read/write splits

– Load balancing

– No change to application

Read Splitting—Easy Way

Use MySQL Utilities to copy master, set up failover

● mysqlserverclone

● mysqlfailover

– When master dies, most up to date slave (GTID) promoted automatically

● Mysqlrplcheck & mysqlrplshow

MySQL Fabric

● Set up highly available groups of servers or shards

● 'Smarts' in the controller, no change to application

● Automatic fail over

● Re - shard on the fly

● As easy yo set up asreplication

Insert Picture Here

Simple Level 2Moderately Easy, BetterAvailability

Distributed ReplicatedBlock Device (DRBD)

● High Availability

More complex environment

● Proven, supported

● Please practice fail over

and back

● Easy to manage but

not HA panacea

High Availability, low cost (sometimes)

Slightly more complex environment

● Proven, supported

● Best for ephemeral data, may not make upper management

comfortable (or you)

● Easy to manage but not HA panacea

Insert Picture HereComplex Level N^NNever, Ever Down

MySQL ClusterFor 99.999% uptime, upper bound on response time

Extremely Complex

● No single point of

failure = lots of $$

hardware

● Not general

purpose MySQL

Auto sharding

● Add/Subtract nodes at will

● Replicate between

geographically separated

locations

● Need a smart staff

MySQL Cluster Shards Data Automatically

● Single or multiple data node failures do not bring down a cluster

● Easy to add/remove data nodes

MySQL Cluster Topology

● Nodes

– MGMT – startup/add node

– Data – Where data lives

– Mysql – the RDMS

● Can be engineered for no

single point of failure

● Easily replicate across data centers

Insert Picture HereBigData

Ask ten DBAs to define 'Big Data' and You WillGet At Least Eleven Answers

● BIG as massive amounts

– Define MASSIVE

● BIG as in fast

● Define fast

● BIG as lots of eyes

● BIG as all of the the above!!!

● How many simultaneous users

The Three Vs of Big Data

● Volume

● Variety

● Velocity

● (maybe) Veracity

End Goal of your Big Data?

● Never, ever toss any data away?

● How will be it managed, examined, maintained, documented, and used?

● Latent needs?

● Data warehouse or Digital Landfill

The Easiest Way to Get Speed

NoSQL/Memcached to InnoDB/NBD

● 9x faster by skipping parser, optimizer

● Key/Value pair access

and SQL access of same data

Easiest Way to Gain Size

Columnar Storage Engines for MySQL

● Not InnoDB

● Unique features, quirks

● Need > 26,000,000,000 rows to start seeing 'sweet spot'

● Use new server, don't scrimp

See Calpont and Infobright

Hadoop-- getting data from MySQL to Hadoop

80% of Hadoop Installations feed from MySQL

● Hadoop is made up of several components

● Twisty little maze, all alike

● Use Apache Bigtop

– Tested for interoperability

● Cloudera & Horton Works have great tutorials

● Kinda 'SQL-ish'

● Squoop and MySQL Hadoop Applier to move data

Partitioning and Sharding /Dividing and Conquering

Partitions built-in to MySQL

● Nice management tool

– Rolling quarters/months

– Fast drops

● No change to application but

higher complexity, low

opportunity cost

Sharding

● Much more complex

application and operations

● Not off shelf

● Hard to re-engineer

● Works

MySQL Fabric

● Lets you shard and reshard

● Very new

● Looking for feedback

● Http://labs.mysql.com

Insert Picture Here

Wrap up

Goals – What are you trying toaccomplishDefine problem completely

● Who produces, who consumes

● Negotiate performance rates

● Plan for failure, catastrophe

● Design for long life

What does end solution look/feel like?

● Platforms

● Speed

● Access areas

● Security

● Serenity

MySQL Marinate

● Free virtual class to learn MySQL run by the Boston MySQL Users Group

– 1 chapter a week

– Homework graded helpfully

MySQL Connect

● San Francisco● September 27th – October 2nd

● LA MySQL User Group will have some passes to raffle

● How do we discount for other groups?

David.Stokes@Oracle.com @stoker

slideshare.net/davestokes

Insert Picture Here

Graphic Section Divider

Scaling MySQL -- NY Code Camp

Software