Managing a shared MySQL farmThijs FerynEvangelist+32 (0)9 218 79 [email protected]
Dutch PHP ConferenceSaturday May 21st 2011Amsterdam, The Netherlands
About me
I’m an evangelist at Combell
About me
I’m a board member at PHPBenelux
I live in the wonderful city of Bruges
MPBecker -‐ Bruges by Night hXp://www.flickr.com/photos/galverson2/3715965933
Give me feedback: hXp://joind.in/3247
Read my blog: hXp://blog.feryn.eu
Follow me on TwiXer: @ThijsFeryn
Managing a shared MySQL farmtekst
Managing a shared MySQL farmtekst
Provisioning/authentication/permissions
Managing a shared MySQL farmtekst
Several clients/apps connect to it
Managing a shared MySQL farmtekst
Multiple servers
The farm
Managing the farm
Managing the farm
User
Permissions
Database
Managing users
✓Create user✓Remove user✓Enable/disable user✓Reset password
Managing databases
✓Create database✓Remove database✓Enable/disable database✓Set quota
Managing permissions
✓Grant permissions✓Revoke permissions✓Enable wricng✓Disable wricng
MySQL authenccacon & privileges
MySQL privilege system
Global privileges
Database privileges
Table privileges
Field privileges
MySQL privilege system
Global privileges
Database privileges
Table privileges
Field privileges
mysql.user
mysql.db
mysql.tables_priv
mysql.columns_priv
General privileges
✓Select✓Insert✓Update✓Delete✓Create✓Drop✓Grant✓References✓Index
✓Alter✓Create tmp table✓Lock tables✓Create view✓Show view✓Create roucne✓Alter roucne✓Execute priv
Server privileges
✓Reload✓Shutdown✓Process✓File✓Show_db✓Super
✓Max quescons✓Max updates✓Max conneccons✓Max user conneccons
Which privileges to grant?
Which privileges to grant?
✓Select✓Insert✓Update✓Delete✓Create✓Drop✓Grant✓References✓Index
✓Alter✓Create tmp table✓Lock tables✓Create view✓Show view✓Create roucne✓Alter roucne✓Execute priv
✓Reload✓Shutdown✓Process✓File✓Show_db✓Super
Manage privileges
✓CREATE USER✓DROP USER✓GRANT✓RENAME USER✓REVOKE✓SET PASSWORD
Manage privileges
✓Manually in mysql.user✓Manually in mysql.db✓Manually in mysql.tables_priv✓Manually in mysql.columns_priv
Challenges
Challenges
✓Management across mulcple nodes✓Aggregacng data from mulcple nodes✓Name clashes✓Quota management
Solucons
Solucons
✓Centralized provisioning database✓GeXers on the provisioning database✓Node mapper for user/db/privilege management✓INFORMATION_SCHEMA for quota management✓Prefixes to avoid name clashes
Provisioning plan
User✓Id✓Prefix✓Username✓Password✓Enabled✓DatabaseId✓Write✓CreateDate✓UpdateDate
Database✓Id✓Node✓Prefix✓Database✓Quota✓Enabled✓Down✓Overquota✓CreateDate✓UpdateDate
User✓Id✓Prefix✓Username✓Password✓Enabled✓DatabaseId✓Write✓CreateDate✓UpdateDate
Database✓Id✓Node✓Prefix✓Database✓Quota✓Enabled✓Down✓Overquota✓CreateDate✓UpdateDate
Mulcpleservers
Database on single node
Mapping uses cases to SQL
✓Add user✓Delete user✓Reset user password✓Enable user✓Disable user✓Get user
Add user
INSERT INTO `user`(`prefix`,`username`,`password`,`createdate`) VALUES(‘test’,‘test_user’,‘mypass123’,NOW());
Delete user
DELETE FROM `user` WHERE username=‘test_user’;
DELETE u.*, db.* FROM `mysql`.`user` u LEFT JOIN `mysql`.`db` db ON(db.`User` = u.`User`) WHERE u.`User` = ‘test_user’;
Reset user password
UPDATE `user` SET `password` = ‘newpass123’ WHERE `username` = ‘test_user’;
UPDATE `mysql`.`user` SET `Password` = PASSWORD(‘newpass123’) WHERE `User`= ‘test_user’;
Enable user
UPDATE `user` SET `enabled` = '1' WHERE `username` = ‘test_user’;
UPDATE `mysql`.`user` SET `Host` = ‘%’ WHERE `User`= ‘test_user’
Disable user
UPDATE `user` SET `enabled` = '0' WHERE `username` = ‘test_user’;
UPDATE `mysql`.`user` SET `Host` = ‘localhost’ WHERE `User`= ‘test_user’
Get user
SELECT * FROM `user` WHERE `username` = ‘test_user’;
✓Add database✓Delete database✓Set database quota✓Enable database✓Disable database✓Get database
Add database
INSERT INTO `database`(`node`,`prefix`,`database`,`quota`,`createdate`) VALUES(1,‘test’,‘test_db’,10,NOW());
CREATE DATABASE test_db1;
Delete database
DELETE FROM `database` WHERE `database` = ‘test_db’;
Delete database
SELECT u.usernameFROM `user` uWHERE u.databaseId = 123GROUP BY u.username; Find
deletable users to delete from MySQL privileges system
Are linked to this database
Delete database
DELETE u.*, db.* FROM `user` u LEFT JOIN `db` db ON(db.`User` = u.`User`) WHERE u.`User` IN('test_user’);
Deletethese users from MySQL privileges
system
Delete database
DROP DATABASE test_db;
Set database quota
UPDATE `database` SET `quota` = 100WHERE `database` = ‘test_db’;
Enable database
UPDATE `database` SET `enabled` = '1' WHERE `database` = ‘test_db’;
Enable database
SELECT u.username, u.writeFROM user uWHERE u.databaseId = 123 Find
user mappings to re-‐enable
Enable database
INSERT INTO `db`(Host,Db,User,Select_priv,Insert_priv, Update_priv,Delete_priv,Create_priv,Drop_priv,Grant_priv,References_priv,Index_priv,Alter_priv,Create_tmp_table_priv,Lock_tables_priv, Create_view_priv,Show_view_priv,Create_routine_priv, Alter_routine_priv,Execute_priv)
Enable database
VALUES(‘%’,‘test_db’,‘test_user’,'Y','Y','Y','Y','Y','Y','N','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y');
VALUES(‘%’,‘test_db’,‘test_user’,'Y','N','N','N','N','N','N','N','N','N','N','N','N','Y','N','N','Y');
Write permissions
Read-‐only
permissions
Disable database
UPDATE `database` SET `enabled` = '0' WHERE `database` = ‘test_db’;
DELETE FROM `db` WHERE db = 'test_db’;
Get database
SELECT * FROM `database` WHERE `database` = ‘test_db’;
✓Grant privilege✓Revoke privilege✓Enable database wricng✓Disable database wricng
Grant privilege
UPDATE `user` SET `databaseId`=123, `write`='1' WHERE `username`= ‘test_user’;
UPDATE `user` SET `databaseId`=123, `write`='0' WHERE `username`= ‘test_user’;
Write permissions
Read-‐only
permissions
Grant privilege
INSERT INTO `user`(Host,User,Password) VALUES(‘%’,‘test_user’,PASSWORD(‘password’));
Try adding user or catch duplicate user error
Grant privilege
INSERT INTO `db`(Host,Db,User,Select_priv,Insert_priv, Update_priv,Delete_priv,Create_priv,Drop_priv,Grant_priv,References_priv,Index_priv,Alter_priv,Create_tmp_table_priv,Lock_tables_priv, Create_view_priv,Show_view_priv,Create_routine_priv, Alter_routine_priv,Execute_priv)
Grant privilege
VALUES(‘%’,‘test_db’,‘test_user’,'Y','Y','Y','Y','Y','Y','N','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y');
VALUES(‘%’,‘test_db’,‘test_user’,'Y','N','N','N','N','N','N','N','N','N','N','N','N','Y','N','N','Y');
Write permissions
Read-‐only
permissions
Revoke privilege
UPDATE `user` SET `databaseId`= NULL, `write`= NULL WHERE `user`= ‘test_user’;
DELETE u.*, db.* FROM `user` u LEFT JOIN `db` db ON(db.`User` = u.`User`) WHERE u.`User` = ‘test_user’;
Enable database wricng
UPDATE `user` SET `write`= '1' WHERE `username` = ‘test_user’;
Enable database wricng
UPDATE `user` SET `write`= '1' WHERE `username` = ‘test_user’;
UPDATE `db` SET`Select_priv` = 'Y',`Insert_priv` = 'Y',`Update_priv` = 'Y',`Delete_priv` = 'Y',`Create_priv` = 'Y',`Drop_priv` = 'Y',`Grant_priv` = 'N',`References_priv` = 'Y',`Index_priv` = 'Y',`Alter_priv` = 'Y',`Create_tmp_table_priv`='Y',`Lock_tables_priv` = 'Y',`Create_view_priv` = 'Y',`Show_view_priv` = 'Y',`Create_routine_priv` = 'Y',`Alter_routine_priv` = 'Y',`Execute_priv` = 'Y'WHERE `db`= ‘test_db’ AND `user` = ‘test_user’;
Disable database wricng
UPDATE `user` SET `write`= '0' WHERE `username` = ‘test_user’;
Disable database wricng
UPDATE `user` SET `write`= '1' WHERE `username` = ‘test_user’;
UPDATE `db` SET`Select_priv` = 'Y',`Insert_priv` = 'N',`Update_priv` = 'N',`Delete_priv` = 'N',`Create_priv` = 'N',`Drop_priv` = 'N',`Grant_priv` = 'N',`References_priv` = 'N',`Index_priv` = 'N',`Alter_priv` = 'N',`Create_tmp_table_priv`='N',`Lock_tables_priv` = 'N',`Create_view_priv` = 'N',`Show_view_priv` = 'Y',`Create_routine_priv` = 'N',`Alter_routine_priv` = 'N',`Execute_priv` = 'Y'WHERE `db`= ‘test_db’ AND `user` = ‘test_user’;
Quota management
Quota management
✓Limits in provisioning database✓Current usage stored in INFORMATION_SCHEMA✓Raco calculated via cron task✓Write permissions disabled while over quota
Quota management
SELECT `database`,`quota` FROM `database`
SELECT TABLE_SCHEMA as `database`,ROUND(SUM(DATA_LENGTH + INDEX_LENGTH)/1048576,2) as `usage`FROM `information_schema`.`TABLES`GROUP BY TABLE_SCHEMA
Quota management
UPDATE `database` SET `overquota` = '1' WHERE `database` = ‘test_db’;
Quota management
UPDATE `db` SET`Select_priv` = 'Y',`Insert_priv` = 'N',`Update_priv` = 'N',`Delete_priv` = 'Y',`Create_priv` = 'N',`Drop_priv` = 'Y',`Grant_priv` = 'N',`References_priv` = 'N',`Index_priv` = 'N',`Alter_priv` = 'N',`Create_tmp_table_priv` = 'N',`Lock_tables_priv` = 'N',`Create_view_priv` = 'N',`Show_view_priv` = 'Y',`Create_routine_priv` = 'N',`Alter_routine_priv` = 'N',`Execute_priv` = 'Y' WHERE `db`= ‘test_database’
Quota management
UPDATE `database` SET `overquota` = '0' WHERE `database` = ‘test_db’;
Quota management
UPDATE `db` SET`Select_priv` = 'Y',`Insert_priv` = 'Y',`Update_priv` = 'Y',`Delete_priv` = 'Y',`Create_priv` = 'Y',`Drop_priv` = 'Y',`Grant_priv` = 'N',`References_priv` = 'Y',`Index_priv` = 'Y',`Alter_priv` = 'Y',`Create_tmp_table_priv`='Y',`Lock_tables_priv` = 'Y',`Create_view_priv` = 'Y',`Show_view_priv` = 'Y',`Create_routine_priv` = 'Y',`Alter_routine_priv` = 'Y',`Execute_priv` = 'Y' WHERE `db`= ‘test_db’
Goals
Single point of management
Single point of conneccon
Replicacon & loadbalancing
Replicacon & loadbalancing
✓Minimizes risk✓Ensures stability, scalability & performance✓Copies databases across nodes✓Doesn’t parccon/shard databases✓Will require mulcple independent clusters
Proxying strategies
Server proxy
Server proxy
MySQL Proxy is a simple program that sits between your client and MySQL server(s) that can monitor, analyze or transform their communicacon.
MySQL Proxy features
✓ Load balancing✓ Failover✓ Query analysis✓ Query filtering and modificacon
Installacon
APT-‐GET INSTALL✓mysql-‐proxy✓lua5.1✓liblua5.1-‐0-‐dev✓liblua5.1-‐sql-‐mysql-‐2✓liblua5.1-‐memcached0✓liblua5.1-‐md5-‐0
Startup
/usr/bin/mysql-‐proxy \-‐-‐proxy-‐lua-‐script=/var/www/mysqlproxy.dev/ \proxy.lua -‐-‐proxy-‐address=:3307 \ -‐-‐proxy-‐backend-‐addresses=172.16.26.133:3306 \-‐-‐proxy-‐backend-‐addresses=172.16.26.134:3306 \-‐-‐lua-‐path=/usr/share/lua/5.1/?.lua \-‐-‐lua-‐cpath=/usr/lib/lua/5.1/?.so Custom
LUA library
/etc/default/mysql-‐proxy
Hooks
✓connect_server✓read_handshake✓read_auth✓read_auth_result✓read_query✓read_query_result✓disconnect_client
Goal
Goal
✓ Accept conneccon using the proxy✓Hook into the authenccacon✓Match user to the provisioning DB✓Fetch node from provisioning✓Switch to the right node
➡Effeccve proxying solucon
Reality
Reality
✓ Accept conneccon using the proxy✓Hook into the authenccacon✓Match user to the provisioning DB✓Fetch node from provisioning✓Switch to the right node
➡Effeccve proxying solucon
Reality
Conneccon switching only happens in the connect_server hook
Auth info is only available starcng from the read_auth hook
Bridge the gap
Bridge the gap
Redirect to node based on client IP
Let’s see some code !
Coderequire('luarocks.require')require('md5')require('Memcached')require('luasql.mysql')local memcache = Memcached.Connect()-‐-‐-‐ configlocal mysqlhost = "localhost"local mysqluser = "myUser"local mysqlpassword = "MyPwDsesd"local mysqldatabase = "test"-‐-‐ debuglocal debug = true
Dependencies & config
Code
function error_result (msg) proxy.response = { type = proxy.MYSQLD_PACKET_ERR, errmsg = msg, errcode = 7777, sqlstate = 'X7777', } return proxy.PROXY_SEND_RESULTend
Custom MySQL errors
Codefunction node_get(ip) local node = memcache:get(md5.sumhexa(ip)) if not node == nil then return loadstring('return '..memcache:get(md5.sumhexa(ip)))() end node = sql_get(ip) if node == nil then return nil end
memcache:set(md5.sumhexa(ip), node, 3600) return node
end
Get node from cache or database
Codefunction sql_get(ip) env = assert (luasql.mysql()) con = assert (env:connect(mysqldatabase,mysqluser,mysqlpassword,mysqlhost)) cur = assert (con:execute(string.format("SELECT n.`id` FROM `accesslist` a JOIN `node` n ON(n.id=a.node) WHERE a.`ip` = '%s'",ip))) row = cur:fetch ({}, "a") if cur:numrows() == 0 then return nil end cur:close() con:close() env:close() return row.idend
Get node from provisioning database
Code
function connect_server() selectedNode = node_get(proxy.connection.client.src.address)
if selectedNode == nil then return error_result(string.format("No info found in the cluster for IP '%s'",proxy.connection.client.src.address)) end
proxy.connection.backend_ndx = selectedNode end
Retrieve and switch to node
Reality
MySQL Proxy is not accvely supported
Client proxy
MySQL Nacve Driver
MySQL Nacve Driver
✓Replacement for libmysql✓Full client protocol as a PHP extension✓Official since PHP 5.3.0✓No API✓Mysql, Mysqli & PDO use it✓Supports plugins
MySQL Nacve Driver
hXp://schlueters.de/blog/archives/146-‐mysqlnd-‐plugins-‐for-‐PHP-‐in-‐praccce.html
hXp://blog.ulf-‐wendel.de/?p=284
Read these blog posts
MySQL Nacve Driver
✓ Accept conneccon using the proxy✓Hook into the authenccacon✓Match user to the provisioning DB✓Fetch node from provisioning✓Switch to the right node✓Doesn’t work for remote conneccons
➡Effeccve proxying solucon
DNS & hostnames
Hostname per account
What about PhpMyAdmin?
What about PhpMyAdmin?
✓Use single signon auth module✓Use customized fallback auth module✓Detect linked database & node✓Switch to node
config.inc.php
<?php$cfg['Servers'][1]['auth_type'] = 'httpsoap';$cfg['Servers'][1]['host'] = '1.2.3.4';$cfg['Servers'][1]['connect_type'] = 'tcp';$cfg['Servers'][1]['compress'] = false;$cfg['Servers'][1]['extension'] = 'mysql';$cfg['Servers'][1]['AllowNoPassword'] = false;$cfg['Servers'][2]['auth_type'] = 'httpsoap';$cfg['Servers'][2]['host'] = '1.2.3.4';$cfg['Servers'][2]['connect_type'] = 'tcp';$cfg['Servers'][2]['compress'] = false;$cfg['Servers'][2]['extension'] = 'mysql';$cfg['Servers'][2]['AllowNoPassword'] = false;$cfg['Servers'][3]['extension'] = 'mysql';$cfg['Servers'][3]['auth_type'] = 'signon';$cfg['Servers'][3]['SignonSession'] = 'SSOSession';$cfg['Servers'][3]['SignonURL'] = 'scripts/signon.php';$cfg['Servers'][3]['LogoutURL'] = 'scripts/signon-logout.php';
scripts/signon.php
<?phpif (isset($_REQUEST['user'])) { try{ $soap = new SoapClient('http://my.soap-webservice.net/?WSDL'); $user = $soap->user_getByUsername($_REQUEST['user']); if(!isset($_REQUEST['hash'])){ die("No hash submitted"); } if(sha1($user->username.$user->password.'azertyuiop') !== $_REQUEST['hash']){ die("Invalid hash"); } } catch (Exception $e){ die("No such user"); }...
scripts/signon.php
session_set_cookie_params(0, '/', '', 0); $session_name = 'SSOSession'; session_name($session_name); session_start(); $_SESSION['PMA_single_signon_user'] = $user->username; $_SESSION['PMA_single_signon_password'] = $user->password; $_SESSION['PMA_single_signon_host'] = $user->node; $_SESSION['PMA_single_signon_port'] = '3306'; $id = session_id(); session_write_close(); header('Location: ../index.php?server=3');} else { exit(); header('Location: ../index.php?server=1');}
scripts/signon-‐logout.php
<?phpsession_set_cookie_params(0, '/', '', 0);$session_name = 'SSOSession';session_name($session_name);session_start();session_destroy();header('Location: ../index.php?server=1');
Customized fallback auth module
✓Copy of ./libraries/auth/h>p.auth.lib.php✓Modify PMA_auth_set_user() funccon✓Implement deteccon logic✓Communicates with provisioning service✓Retrieves database & node✓Switches to node
libraries/auth/hXpsoap.auth.lib.php<?phpfunction PMA_auth_set_user(){ global $cfg, $server; global $PHP_AUTH_USER, $PHP_AUTH_PW; try{ $soap = new SoapClient('http://my.soap-webservice.net/?WSDL'); $user = $soap->user_getByUsername($PHP_AUTH_USER); $cfg['Server']['host'] = $user->node; } catch (Exception $e){ PMA_auth(); return true; }...
libraries/auth/hXpsoap.auth.lib.phpif ($cfg['Server']['user'] != $PHP_AUTH_USER) { $servers_cnt = count($cfg['Servers']); for ($i = 1; $i <= $servers_cnt; $i++) { if (isset($cfg['Servers'][$i]) && ($cfg['Servers'][$i]['host'] == $cfg['Server']['host'] && $cfg['Servers'][$i]['user'] == $PHP_AUTH_USER)) { $server = $i; $cfg['Server'] = $cfg['Servers'][$i]; break; } } } $cfg['Server']['user'] = $PHP_AUTH_USER; $cfg['Server']['password'] = $PHP_AUTH_PW; return true;}
Q&A