Introducing Lua for MySQL Proxy - datacharmer.comdatacharmer.com/downloads/introducing_lua.pdf ·...

Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database

Giuseppe MaxiaQA Developer

December 2007

Introducing Lua for MySQL Proxy

programming


Giuseppe Maxia

MySQL QA Developer

a.k.a. The Data Charmer

✦ Despite popular belief, not a Lua expert (less than six months experience)

About me


• MySQL Proxy architecture (very quickly)• Introducing Lua• Language fundamentals• Examples• Comments

Agenda


• MySQL Forge • http://forge.mysql.com/wiki/MySQL_Proxy

• Getting started article• http://dev.mysql.com/tech-resources/articles/proxy-gettingstarted.html

• Lua• http://www.lua.org/docs.html• http://lua-users.org/wiki/TutorialDirectory

References

http://forge.mysql.com/wiki/MySQL_Proxy

http://forge.mysql.com/wiki/MySQL_Proxy

http://dev.mysql.com/tech-resources/articles/proxy-gettingstarted.html




http://www.lua.org/docs.html%0Dhttp://www.lua.org/docs.html%0D

http://www.lua.org/docs.html%0Dhttp://www.lua.org/docs.html%0D

http://lua-users.org/wiki/TutorialDirectory

http://lua-users.org/wiki/TutorialDirectory


Proxy (< lat. procuratio)

5

= Someone taking care of someone else's interests

A server proxy is something acting on behalf of another server


Proxy Overview

6


Proxy Overview

7

PROXY CORE

connection hook

read query hook

read result hook

function

Lua script

function

functionfunctionfunction


Lua Overview

8

??Why not ...{Perl ?

PHP?Javascript?[whatever]?


Lua Overview

9

• SMALL ( < 200 KB)• DESIGNED for

EMBEDDED systems• Widely used (lighttpd)

lighttpd, like MySQL Proxy, was created by Jan Kneschke


Lua Overview

10

Very popular among game writers


Proxy Overview

11

connect_server

Lua script

read_auth

read_auth_result

read_handshake

read_query

read_query_result

disconnect_client

global context

session contextsession

contextsession contextsession

contextsession contextsession

contextsession context


Using Lua files

12

/usr/local/sbin/mysql-proxy \ --proxy-lua-script=/path/name.lua

IMPORTANT!THE SCRIPT DOES NOT START UNTIL THE

FIRST CLIENT CONNECTION


Lua types

13

• nil• number• string• table• function• userdata

a = nilb = 1c = 'abc't = { a,b,c }f = printu = some_C_struct


Lua comments

14

-- simple comment

print(1)--[[print(2)print('hello')--]]print(3)


Lua comments

15

-- simple comment

--[=[ print(1)--[[print(2)print('hello')--]]print(3)--]=]


numbers and strings

16

• nil = no value (false)• number = floating point• '5' =/= 5 a = nil

b = 5; c = '5'print (b == c)falseprint (b == c +0)true


numbers and strings

17

• conversion on demand

a = 5 ; b = '5'

print(type(a), type(b))number stringprint(type(b+0))numberprint(type(a .. ""))string


numbers and strings

18

• conversion on demand

a = 5 ; b = '5'

print(type(tostring(a)))stringprint(type(tonumber(b)))number


strings

19

• smart quoting

a = 'Hello' b = "World"c = "Can't"d = [[Don't say "Hello"]]e = [=["d'oh" [[braces]]!]=]


tables

20

• associative arrays• can be used as arrays• can create complex

structurest1 = {10, 20, 30 }t2 = { a = 'abc', b = 2, c = { 3, 4}}


functions

21

• can be assigned to variables

• new functions can override existing ones

function f (x) print(x)endg = fg(10)


userdata

22

• containers to exchange data between Lua and host language

• can have "tag methods"


Statements

23

• normal assignmentsa = 3b = get_num() -- func return

• multiple assignmentsa,b = 3,2

• multiple return valuesfunction x () return 1, 'OK'end

a, b, c = x() -- a = 1, b = 'OK', c = nil


statement blocks

24

• ifif condition then statementsend

• whilewhile condition do statementsend


statement blocks

25

• forfor var = 1, 10 [,step] do statementsend

• forfor n,v in pairs(table_var) do statementsend


sample function

26

• read_query

1 function read_query(packet) 2 if packet:byte() ~= 3 proxy.PROXY_COM_QUERY 4 then 5 return 6 end 7 local query = packet:sub(2) 8 print('received ' .. query) 9 end


some details

27

== equals ~= not equal string.byte(packet) packet:byte()

string.sub(packet,2) packet:sub(2) 'abc' .. '123' == 'abc123'


Using tables

28

t = {}t[1] = 'a' --First element 1, ! 0t[2] = 'b'table.insert(t, 'c') -- or t[ #t +1 ] = 'c'

t = {'a', 'b', 'c' }t = {1 = 'a', 2 = 'b', 3 = 'c'}print (t[2])b


Using tables

29

sizes = {}sizes['john'] = 'XL'sizes['paul'] = 'M'sizes['fred'] = 'L'

sizes = { john = 'XL', paul = 'M', fred = 'L',}


Using tables

30

sizes = { john = 'XL', paul = 'M', fred = 'L',} print(sizes['john'])XLprint(sizes.paul)M


Using tables

31

for i, v in ipairs(t) do print (i ..' -> ' .. v)end

for name,size in pairs(sizes) do print(name .. ' ' .. wears .. ' ' .. size)end


Watch out

32

/* C / C++ */ int a = 0;printf("%s\n", a ? "true" : "false");false

-- Luaa = 0print ( a and "true" or "false")true


Watch out

33

-- Luaa = falseprint ( a and "true" or "false")false

a = nilprint ( a and "true" or "false")false


Finding text

34

query = 'SELECT id FROM t1'

local cmd, column = query:match("(SELECT)%s+(%w+)")

if cmd then -- do something with queryend


Finding text

35

• Regular expressions• similar to Perl/PHP, but simpler

• % instead of \• (captures)• [character classes]• ^ $ + - ? * • no alternation (a|b)• no modifiers /i


Finding text (Proxy way)

36

local tk = require('proxy.tokenizer')

local tokens = tk.tokenize(query)

if tokens[1].token_name == 'TK_SQL_SELECT' then -- do something with queryend


Finding text (Proxy way)

37

-- each token is a table

token = { token_name = 'TK_SQL_SELECT', text = 'select', token_id = 204}


I/O

38

-- files are objects

local fname = '/tmp/test.txt'assert(fh = io.open(fname,'r'))

for line in fh:lines() do print(line)endfh:close()


I/O

39

-- files are objects

local fname = '/tmp/test.txt'

assert(fh = io.open(fname,'w'))for x = 1, 100 do fh:write('new row ' .. x) fh:flush()endfh:close()


Examples

40

http://forge.mysql.com/wiki/Lua_Scripts_For_MySQL_Proxy_Examples

• all hooks• session bandwidth• user bandwidth• blocking commands




all-hooks (1)

41

source: all-hooks.lua

function read_auth( auth )...function connect_server()...function read_handshake( auth )...function read_auth_result( auth )...function disconnect_client()...function read_query (packet)...function read_query_result (inj)...

http://forge.mysql.com/snippets/download.php?id=109



all-hooks (2)

42


local access_ndx = 0

function print_access(msg) access_ndx = access_ndx + 1 print( string.format('%3d %-30s',access_ndx,msg))end

function read_auth( auth ) print_access ('inside read_auth ')end

function connect_server() print_access ('inside connect_server')end

function read_handshake( auth ) print_access ('inside read_handshake' )end-- ... to be continued




all-hooks (3)

43


function read_auth_result( auth ) print_access ('inside read_auth_result ')end

function read_query (packet) print_access('inside read_query \t' .. packet:sub(2)) proxy.queries:append(1, packet) return proxy.PROXY_SEND_QUERYend

function read_query_result (inj) print_access('inside read_query_result \t' .. inj.query)end

function disconnect_client() print_access('inside disconnect_client')end




read_query & read_query_result

44

client

MySQL Proxy

SERVER

query

functionread_query

function

read_query_result

query

result

result

if a query is passed directly to the server,its result is NOT evaluated by read_query_result


read_query & read_query_result

45

client

MySQL Proxy

SERVER

query

function read_query

functionread_query_result

query queue

query

result

result

only if a query is added to the query queue,its result is evaluated by read_query_result

queryquery


all-hooks (4)

46


sample output/usr/local/sbin/mysql-proxy --proxy-lua-script=all-hooks.lua 1 inside connect_server 2 inside read_handshake 3 inside read_auth 4 inside read_auth_result 5 inside read_query select @@version_comment limit 1 6 inside read_query_result select @@version_comment limit 1 7 inside read_query select now() 8 inside read_query_result select now() 9 inside read_query 10 inside disconnect_client




session-bandwidth (1)

47

source: session-bandwidth.lua

local bandwidth = 0

function read_query (packet ) bandwidth = bandwidth + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERYend

-- to be continued ...




session-bandwidth (2)

48


-- (continue)

function read_query_result(inj) local fields = inj.resultset.fields if (fields) then for i = 1, #fields do bandwidth = bandwidth + (fields[i] and fields[i].name:len() or 0) end for row in inj.resultset.rows do for i = 1, #inj.resultset.fields do bandwidth = bandwidth + (row[i] and row[i]:len() or 0) end end end print (proxy.connection.server['thread_id'],bandwidth)end




session-bandwidth

49


sample output:

/usr/local/sbin/mysql-proxy --proxy-lua-script=bandwidth.lua64 7864 115

65 7865 115

66 7866 1118




user-bandwidth (1)

50

source: user-bandwidth.lua

proxy.global.bandwidth = proxy.global.bandwidth or {}

local session_user

function read_auth( auth ) session_user = auth.username proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] or 0end





user-bandwidth (2)

51


function read_query (packet ) proxy.global.bandwidth[session_user ] = proxy.global.bandwidth[session_user] + packet:len() proxy.queries:append(1, packet ) return proxy.PROXY_SEND_QUERYend




Copyright 2004-2007 MySQL AB The World’s Most Popular Open Source Database 52

function read_query_result(inj) local fields = inj.resultset.fields local rows = inj.resultset.rows if fields then for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (fields[i] and fields[i].name:len() or 0) end if rows then for row in rows do for i = 1, #fields do proxy.global.bandwidth[session_user] = proxy.global.bandwidth[session_user] + (row[i] and row[i]:len() or 0) end end end end print (session_user .. ' -> ' .. proxy.global.bandwidth[session_user])end

user-bandwidth (3)



Sample session

gmax -> 376gmax -> 399

root -> 599root -> 10641

gmax -> 478gmax -> 20520

user-bandwidth (4)




source: query-block.lua

local tokenizer = require("proxy.tokenizer")----- make_regexp_from_command()---- creates a regular expression for fast scanning of the command-- -- @param cmd the command to be converted to regexp--function make_regexp_from_command(cmd) local regexp= '^%s*'; for ch in cmd:gmatch('(.)') do regexp = regexp .. '[' .. ch:upper() .. ch:lower() .. ']' end return regexpend-- to be continued ...

query-block (1)





local SHOW_REGEXP = make_regexp_from_command('show')

queries_to_filter = { { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLE', 'STATUS'} , }, { prefix = SHOW_REGEXP, keywords = { 'SHOW', 'TABLES'} , },}


query-block (2)





function error_result (msg) proxy.response = { type = proxy.MYSQLD_PACKET_ERR, errmsg = msg, errcode = 7777, sqlstate = 'X7777', } return proxy.PROXY_SEND_RESULTend


query-block (3)




function read_query( packet ) if (packet:byte() ~= proxy.COM_QUERY) then return end local query = packet:sub(2) for i,query_to_filter in pairs(queries_to_filter) do if query:match(query_to_filter.prefix) then local full_tokens = tokenizer.tokenize(query) local tokens = tokenizer.bare_tokens(full_tokens, true) local found = 0 local requested = #query_to_filter.keywords for j,keyword in pairs(query_to_filter.keywords) do for k, token in pairs(tokens) do if token:upper() == keyword then found = found + 1 break end end end-- to be continued ...

query-block (4)


function read_query( packet ) -- ...

if found == requested then -- to be filtered off return error_result('command <' .. table.concat(query_to_filter.keywords,' ') .. '> is not allowed' ) end end endend

query-block (5)



sample output:

show tables;ERROR 7777 (X7777): command <SHOW TABLES> is not allowed

query-block (6)




Read more

60

http://www.lua.org/docs.html

online Lua documentation




Read more

61

http://www.inf.puc-rio.br/~roberto/pil2/




Read more

62

http://www.wrox.com/WileyCDA/WroxTitle/productCd-0470069171.html




Giuseppe MaxiaQA Developer

December 2007

QUESTIONS

?

Date post:	10-Jul-2020
Category:	Documents
Upload:	others
View:	9 times
Download:	0 times

Introducing Lua for MySQL Proxy - datacharmer.comdatacharmer.com/downloads/introducing_lua.pdf ·...

Documents