Date post: | 08-Dec-2014 |
Category: |
Technology |
Upload: | karthik-pr |
View: | 158 times |
Download: | 1 times |
Scaling Persistent Store UsingMySQL FABRIC
P.R.KARTHIKMySQL DBA
About US
Karthik.P.R• 4 + years of Industry experience as MySQL DBA.
• 1+ year at Yahoo! Administrating MySQL servers.
• Area of Focus MySQL HA and MySQL Sharding
• Student : M.Sc ( FOSS)
• Blogger : remotemysqldba.blogspot.in
PROGRAM AGENDA
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
Scaling MySQL
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
• Handling Shards
Scaling MySQL
• Starts with a single node
Scaling MySQL
• As reads and writes grows
Scale Vertically.
• Increase system resources• Memory• CPU • Hard Disk
Scaling MySQL
Scaling reads using replication. • Replication is the best solution.• Replication is Async.• Failover has to be set.• Single writer
Read Queries Write Queries
Scaling MySQL
Scaling reads using Galera • Read/ Write on any node• Synchronous Replication• A good HA solution.• Easy to add nodes.
Write / Read Queries
Scaling MySQL
Scaling Writes
• Replication can’t scale writes• Partitioning is needed.• Distributes the writes• Horizontal scaling or Sharding.
ID 1 - 1000 ID 1001 - 2000
Scaling MySQL
Components .
•Shard Key– Range– Hash– List
•Meta data store•Managing data set•High Availability of shards.•Database and schema changes.
Scaling MySQL
Sharding Architecture
Proxy layer
State Store
Shard A
Shard B
Shard C
Application
Scaling MySQL
Advantages of Sharding
•Improved Performance•Smaller Data set on local node.•Handling large data set•Scale well horizontally.•Only small data set is affected on a node failure.
Disadvantages of Sharding
•Managing the shards •Code change is need at some cases.•Splitting Shards•Maintaining the HA
Available Sharding Tools
• Sharding is used widely in Large Scale MySQL Deployments.• Large organization built their own tools for sharding.
Popular MySQL Shards,
Facebook• Twitter• Tumblr• Flickr• Youtube• Dropbox
Available Sharding Tools
Sharding Tools.
1) Saclebase ( Closed Source)2) Jetpants (Tumblr ) 3) Vitess ( Youtube )4) MySQL Fabric ( Oracle MySQL )
MySQL FABRIC
MySQL FABRIC
• MySQL Fabric is the frame work to maintain shards and High Availability.• Downloaded from MySQL Utilities.• Good CLI commands.• Works on MySQL > 5.6.10• XML RPC for Python , php and Java.• Minimized downtime of shard.
MySQL FABRIC ARCHITECTURE
Application XML-RPC Connector
XML-RPC
FABRICNODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store
MySQL FABRIC
FABRIC Components.
• Fabric node.• Fabric aware connectors ( XML RPC )• Server Groups.
– Global Group– Shard Group
• Back Store
MySQL FABRIC
Fabric node
• Stores Fabric Config• Shard Moving• Shard splitting• High Availability monitoring
Back Store
• Global tables info• Shard key info and mapping• Shard monitoring
Fabric Node Back store
MySQL FABRIC
Fabric aware connectors
• Supports Python/ php / Java Connectors• Fetch Shard info• Local cache• Avoids the proxy hop.
Application XML-RPC Connector
MySQL FABRIC
Server Groups
• Stores actual data – Global Group– Shard Group
• Primary – Read / Write• Secondary – Reads ( failover)• Spare ( On request )
Primary Node Secondary Node
Spare Node
Server Group
MySQL FABRIC
Server Groups ( Global Group )
• Stores Global Tables and schema• Schema changes• Replicates to Shard groups
Global Group Shard Group
Global Writes
Fabric Node
MySQL FABRIC ARCHITECTURE ( Again )
Application XML-RPC Connector
XML-RPC
FABRICNODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store
MySQL FABRIC
Advantages:
• Ease of Failover with GTID.• Shard Split is easy • RPC avoid network hops.
Dis Advantages
• Fabric node will be single point of failure • No Cross Shard . ( Shard query can help here)• Limited Connectors.
THANK YOU