libdrizzle lua nginx

8/2/2019 Libdrizzle Lua Nginx

1/140

Scripting libdrizzle with Lua inside Nginx


2/140

Scripting libdrizzle with Lua

inside Nginx

(agentzh)[email protected]

2012.4


3/140

"MySQL is always the bottleneck !"Really?!"


4/140


5/140


6/140

Some benchmarks onAmazon EC2 Small instances


7/140


8/140

A Slow MySQL Query

select sleep (1)


9/140

Amazon Linux AMI 2011.09

nginx 1.0.14

php-fpm 5.3.10


10/140


11/140


12/140


13/140


14/140

A Fast MySQL Query

with a Small Resultset

select *

from world.City order by ID limit 1


15/140


16/140


17/140

A Fast MySQL Query

with a Big Resultset (100 KBytes)

select * from world.City order by ID limit 1000


18/140


19/140


20/140


21/140

We integrated libdrizzle directly into Nginx !

http://wiki.nginx.org/HttpDrizzleModule


22/140


23/140


24/140


25/140


26/140


27/140

Let's just mud with nginx.conf ,the Nginx configuration file


28/140

upstream my_mysql_backend {

drizzle_server 127.0.0.1:3306 dbname= test

password= some_pass user= monty protocol= mysql ;

# a connection pool that can cache up to # 200 mysql TCP connections drizzle_keepalive max=200 overflow= reject

}


29/140

location ~ '^/cat/(.*)' {

set $name $1 ; set_quote_sql_str $quoted_name $name ; drizzle_query "select * from cats where name= $quoted_name " ;

drizzle_pass my_mysql_backend;

rds_json on;}


30/140

$ curl 'http://localhost/cat/Jerry'[{"name":"Jerry","age":1}]


31/140

The dynamic SQL Query for

This Request

select *

from cats where name= 'Jerry'


32/140


33/140

The Slow MySQL Query again!

select sleep (1)


34/140


35/140


36/140


37/140

The Fast MySQL Query

with a Small Resultset Again!



38/140


39/140


40/140


with a Big Resultset (100 KBytes) Again!



41/140


42/140


43/140

We also embedded Lua and LuaJIT directly into Nginx !

http://wiki.nginx.org/HttpLuaModule


44/140


45/140

Use the Lua language to accessthe ngx_drizzle module!


46/140


47/140

location = /api { content_by_lua '

local rds_parser = require "rds.parser"local cjson = require "cjson"

local resp = ngx.location.capture ("/cat/Jerrylocal data, err = rds_parser.parse(res.body)ngx.print(cjson.encode(data.resultset))

';}


48/140

$ curl 'http://localhost/api'[{"name":"Jerry","age":1}]


49/140


with a Small Resultset Revisited!



50/140


51/140


52/140


with a Big Resultset (100 KBytes) Again!



53/140


54/140


55/140

I just implemented the Lua cosocket API !

http://wiki.nginx.org/HttpLuaModule#ngx.socket.tcp


56/140

!! a socket API based on Lua coroutines!! a socket API that is synchronous !! a socket API that is nonblocking


57/140


58/140

I wrote the lua-resty-mysql librarybased on the cosocket API.

http://github.com/agentzh/lua-resty-mysql


59/140

It is a pure Lua MySQL driverwritten from scratch !

l l l i " l"


60/140

local resty_mysql = require "resty.mysql"

local mysql = resty_mysql:new()

local ok, err = mysql:connect{host = "127.0.0.1" ,port = 3306 ,

database = "world" ,user = "monty" ,password = "some_pass"

}


61/140

local query = "select * from cats"

local rows, err, errno, sqlstate = mysql:query(query)

for i, row in ipairs(rows) do

-- process the row table end


62/140

-- add the current MySQL connection

-- into the per-worker connection pool, -- with total capacity of 1024 connections and -- 60 seconds maximal connection idle time

local ok, err = mysql:set_keepalive( 60000 , 1024


63/140

The Slow MySQL Query Revisited!

select sleep (1)


64/140


65/140


66/140


67/140

The Fast MySQL Querywith a Small Resultset Revisited!



68/140


69/140


70/140

The Fast MySQL Querywith a Big Resultset (100 KBytes) Revisited!

select * from world.City

order by ID limit 1000


71/140


72/140


73/140

How about comparing withthe NodeJS world?


74/140


75/140


76/140


77/140


78/140


79/140

Caching responses with

ngx_srcache + ngx_memc http://wiki.nginx.org/HttpSRCacheModule

http://wiki.nginx.org/HttpMemcModule


80/140

# configure the cache storage location


81/140

g g location /memc {

internal ; set $memc_key $query_string ;

set $memc_exptime 300;

memc_pass 127.0.0.1:11211;}

location = /api {


82/140

location = /api {

set $key "$uri?$args" ;

srcache_fetch GET /memc $key ; srcache_store PUT /memc $key ;

# drizzle_pass/fastcgi_pass/content_by_lua/...}


83/140


84/140


85/140


86/140

Find the source forall the benchmarks given here:

http://github.com/agentzh/mysql-driver-benchmark


87/140

Any questions ?

http://openresty.org

https://groups.google.com/group/openresty


88/140


89/140


90/140


91/140


92/140


93/140


94/140


95/140


96/140


97/140


98/140


99/140


100/140


101/140


102/140


103/140


104/140


105/140


106/140


107/140


108/140


109/140


110/140


111/140


112/140


113/140


114/140


115/140


116/140


117/140


118/140


119/140


120/140


121/140


122/140


123/140


124/140


125/140


126/140


127/140


128/140


129/140


130/140


131/140


132/140


133/140


134/140


135/140


136/140


137/140


138/140


139/140


140/140

libdrizzle lua nginx

Documents