Lighttpd & Lighttpd & ModcacheModcache

http://blog.quehy.com/http://blog.quehy.com/

2009/06/282009/06/28

Beta技术沙龙http://club.blogbeta.com官方 twitter：@betasalon

Lighttpd & Lighttpd & ModcacheModcache

2009/06/282009/06/28

Basic lighttpd infoBasic lighttpd info Event-driven, single processEvent-driven, single process Uses non-block io (network) + writev Uses non-block io (network) + writev

(memory) + sendfile (local files)(memory) + sendfile (local files) Light weighted, simple module Light weighted, simple module

structurestructure HomepageHomepage

http://redmine.lighttpd.net/projects/http://redmine.lighttpd.net/projects/lighttpdlighttpd

LighttpdLighttpd’’s advantagess advantages

No need to sync data and variables No need to sync data and variables between modulesbetween modules

Modern OS/Hardware are so powerful Modern OS/Hardware are so powerful that event-driven model works greatthat event-driven model works great

Handy conditional configurationHandy conditional configuration Easy to write modules (Easy to write modules (

http://redmine.lighttpd.net/projects/lihttp://redmine.lighttpd.net/projects/lighttpd/wiki/Develghttpd/wiki/Devel ) )

LighttpdLighttpd’’s disadvantagess disadvantages

Sometimes one process isnSometimes one process isn’’t t enough (try server.max-workers)enough (try server.max-workers)

One block io operation may One block io operation may block whole http serviceblock whole http service

mod_fastcgi doesnmod_fastcgi doesn’’t work as fast t work as fast as it should be because of poor as it should be because of poor request distribution algorithmrequest distribution algorithm

General lighttpd tuningGeneral lighttpd tuning

server.event-handler = "linux-sysepoll“ server.event-handler = "linux-sysepoll“ server.max-fds = 16384 # set max open server.max-fds = 16384 # set max open

fd limit to 16kfd limit to 16k server.max-keep-alive-idle = 0 #don’t server.max-keep-alive-idle = 0 #don’t

use keep-aliveuse keep-alive server.max-read-idle = 30 # smaller server.max-read-idle = 30 # smaller

read timeoutread timeout server.max-write-idle = 180 # smaller server.max-write-idle = 180 # smaller

write timeoutwrite timeout

Mod_fastcgi TuningMod_fastcgi Tuningserver.max-request-size = 40960 #maximum 40M post content lengthserver.max-request-size = 40960 #maximum 40M post content length

fastcgi.server = ( ".php" => ((fastcgi.server = ( ".php" => (( "socket" => "/tmp/php-fastcgi.socket","socket" => "/tmp/php-fastcgi.socket", "bin-path" => "/usr/bin/php-cgi -c /etc/php.ini","bin-path" => "/usr/bin/php-cgi -c /etc/php.ini", "max-procs" => 20,"max-procs" => 20, "bin-environment" => ("bin-environment" => ( "PHP_FCGI_CHILDREN" => "9", #lighttpd will fork "PHP_FCGI_CHILDREN" => "9", #lighttpd will fork

20*10=200 php-cgi children20*10=200 php-cgi children "PHP_FCGI_MAX_REQUESTS" => "50000“ # php child "PHP_FCGI_MAX_REQUESTS" => "50000“ # php child

restart after processing 50k requestsrestart after processing 50k requests ),), "broken-scriptfilename" => "enable","broken-scriptfilename" => "enable", "allow-x-send-file" => "enable","allow-x-send-file" => "enable", "check-local" => "disable","check-local" => "disable", "bin-copy-environment" => ( "PATH", "SHELL", "USER" )"bin-copy-environment" => ( "PATH", "SHELL", "USER" ) ))))))

Basic modcache infoBasic modcache info

Project started when squid canProject started when squid can ’’t fulfill t fulfill needneed

v1.0 released at 10/2006, v1.8 released v1.0 released at 10/2006, v1.8 released at 06/2009at 06/2009

Open source, BSD LicenseOpen source, BSD License Supports both disk-based cache and Supports both disk-based cache and

memory-based cachememory-based cache HomepageHomepage

http://www.linux.com.cn/modcache/http://www.linux.com.cn/modcache/

ModcacheModcache’’s advantagess advantages

Lighttpd based (itLighttpd based (it’’s a powerful web s a powerful web server, far away beyond normal proxy server, far away beyond normal proxy server)server)

Works with other lighttpd modules, Works with other lighttpd modules, such as mod_compress / mod_accesssuch as mod_compress / mod_access

Keep it as simple as possibleKeep it as simple as possible Small source code base (< 3000 lines)Small source code base (< 3000 lines) DonDon’’t use local db to store cache t use local db to store cache

informationinformation

ModcacheModcache’’s s disadvantagesdisadvantages

No internal cache files management No internal cache files management (try /usr/sbin/tmpwatch)(try /usr/sbin/tmpwatch)

Rules of cache are controlled by Rules of cache are controlled by modcache, not by backend serversmodcache, not by backend servers

Memory cache doesnMemory cache doesn’’t share t share between different lighttpd processesbetween different lighttpd processes

ItIt’’s a patch, not included in official s a patch, not included in official lighttpd releaselighttpd release

Modcache vs SquidModcache vs Squid ModcacheModcache

1.1. Event basedEvent based

2.2. Directories basedDirectories based

3.3. Without disk Without disk managementmanagement

4.4. Reverse Cache onlyReverse Cache only

5.5. No ICPNo ICP

6.6. Expire time based on Expire time based on refresh.patternrefresh.pattern

7.7. Conditional config /ACLConditional config /ACL

8.8. Small source codebaseSmall source codebase

SquidSquid

1.1. Thread basedThread based

2.2. DB basedDB based

3.3. With disk managementWith disk management

4.4. Reverse/Forward CacheReverse/Forward Cache

5.5. Supports ICPSupports ICP

6.6. Expire time based on Expire time based on backend response and backend response and refresh-patternrefresh-pattern

7.7. ACLACL

8.8. Large source codebaseLarge source codebase

Modcache config(1)Modcache config(1) cache.basescache.bases

List of cache List of cache directories. For directories. For examples: cache.bases = examples: cache.bases = ("/data/cache", ("/data/cache", "/data2/cache")"/data2/cache")

cache.domainscache.domains List of domains which List of domains which

modcache would try to modcache would try to cache for. When it isn’t cache for. When it isn’t set, modcache would cache set, modcache would cache files of every domains. files of every domains. for examples: for examples: cache.domains = cache.domains = ("(^|\.)linux\.com\.cn$") ("(^|\.)linux\.com\.cn$") caches *.linux.com.cn. caches *.linux.com.cn. domains.domains.

cache.ignore-cache.ignore-hostnamehostname. . Includes hostname in Includes hostname in

saved filename or not. saved filename or not.

default is disable.default is disable. cache.debugcache.debug

Default is disableDefault is disable

cache.enablecache.enable Default is enableDefault is enable

Modcache config(2)Modcache config(2) cache.support-queriescache.support-queries

Try to cache query with Try to cache query with '?'. default is '?'. default is 'disable'. When 'disable'. When cache.support-queries = cache.support-queries = "enable" and "enable" and cache.dynamic-mode = cache.dynamic-mode = "disable", modcache will "disable", modcache will treat "/uri?q1" or "/uri?treat "/uri?q1" or "/uri?q2" as "/uri" and save q2" as "/uri" and save them to local cache file them to local cache file of "/uri". When of "/uri". When cache.support-queries and cache.support-queries and cache.dynamic-mode are cache.dynamic-mode are both enabled, modcache both enabled, modcache will treat "/uri?q1" and will treat "/uri?q1" and "/uri?q2" as different "/uri?q2" as different resource and save them to resource and save them to different local cache different local cache files. files.

cache.dynamic-modecache.dynamic-mode To support dynamic To support dynamic

generated web page such as generated web page such as forum/bbs. default is forum/bbs. default is "disable". Note: "disable". Note: cache.support-queries has cache.support-queries has to be enabled when to be enabled when dynamic-mode is enabled.dynamic-mode is enabled.

cache.programs-extcache.programs-ext List of local program List of local program

extension, such as ".php". extension, such as ".php". modcache will replace them modcache will replace them with ".cache.html" to with ".cache.html" to aviod confliction with aviod confliction with other modules, such as other modules, such as mod_fastcgi. default is mod_fastcgi. default is not setnot set

Modcache config(3)Modcache config(3)

cache.max-memory-cache.max-memory-sizesize Number of MBytes Number of MBytes which modcache will which modcache will use to save cached use to save cached content in memory. content in memory. default is default is 256Mbytes.256Mbytes.

cache.lru-remove-cache.lru-remove-countcount Number of content Number of content removed from memory removed from memory when memory is full. when memory is full. default is 256.default is 256.

cache.purge-hostcache.purge-host pcre regex hosts ip pcre regex hosts ip which are allowed to which are allowed to PURGE cache files. PURGE cache files. for examples: for examples: cache.purge-cache.purge-host="^200\.100\.1\.host="^200\.100\.1\." allow " allow 200.100.1.0/24 to 200.100.1.0/24 to purge cache. Default purge cache. Default is not setis not set

Cache.refresh-patternCache.refresh-pattern

Key of modcache, idea from Key of modcache, idea from squidsquid’’s refresh-patterns refresh-pattern

cache.refresh-pattern format is cache.refresh-pattern format is "url_pcre_regex" => "minutes "url_pcre_regex" => "minutes options". Note: options". Note: zero 'minutes'zero 'minutes' means cache forever and means cache forever and minutes option is mandatory.minutes option is mandatory.

Refresh Pattern Refresh Pattern options(1)options(1)

ignore-reloadignore-reload Don't update cache when browser sends Don't update cache when browser sends 'Cache-Control: 'Cache-Control:

no-cache'no-cache' header. It's default behaviour header. It's default behaviour update-on-refreshupdate-on-refresh

Update cache when browser sends Update cache when browser sends 'Cache-Control: no-'Cache-Control: no-cache'cache' header. header.

NocacheNocache Don't cache matched urlDon't cache matched url. .

fetchall-for-range-requestfetchall-for-range-request Download all content of file if client sends Download all content of file if client sends 'Range: xxx-yyybytes''Range: xxx-yyybytes' header. Useful for multi- header. Useful for multi-thread downloaders(such as flashget).thread downloaders(such as flashget).

Refresh Pattern Refresh Pattern options(2)options(2)

rfc-violated options:rfc-violated options: override-expireoverride-expire

Ignore backend's Ignore backend's 'Expire''Expire' header while header while determining whether to cache.determining whether to cache.

ignore-varyignore-vary Igore backend’s ‘Vary’ header while Igore backend’s ‘Vary’ header while determining whether to cachedetermining whether to cache

ignore-cache-control-headerignore-cache-control-header Ignore backend's Ignore backend's 'Cache-Control''Cache-Control' headers while determining whether to headers while determining whether to cache.cache.

Refresh Pattern Refresh Pattern options(3)options(3)

flv-streamingflv-streaming to work with mod_flv_streaming.to work with mod_flv_streaming.

memory-compressmemory-compress compress memory-saved content for compress memory-saved content for ‘Accept-Encoding: gzip, deflate’ ‘Accept-Encoding: gzip, deflate’ requestrequest

use-memoryuse-memory use memory to save caches instead use memory to save caches instead of local disk of local disk

Modcache Cook Books Modcache Cook Books (1)(1)

Site with several millions image filesSite with several millions image filesserver.modules = (server.modules = (## ...., # other modules ...., # other modules

"mod_cache", # make sure mod_cache loaded before "mod_cache", # make sure mod_cache loaded before mod_proxymod_proxy"mod_proxy""mod_proxy"

))proxy.worked-with-mod-cache = "enable"proxy.worked-with-mod-cache = "enable"cache.bases = ("/data/cache")cache.bases = ("/data/cache")cache.support-queries = "enable"cache.support-queries = "enable"cache.max-memory = 2000 # 2000M suitable for 4G memory cache.max-memory = 2000 # 2000M suitable for 4G memory

serverservercache.refresh-pattern = (cache.refresh-pattern = (

"\.(?i)(jpg|bmp|jpeg|gif|png|ico)$" => "0 use-memory", #cache "\.(?i)(jpg|bmp|jpeg|gif|png|ico)$" => "0 use-memory", #cache for ever until removed from memory for ever until removed from memory "." => "120 " # other uses disk based cache"." => "120 " # other uses disk based cache

))

Modcache Cook Books Modcache Cook Books (2)(2)

Softwares/Music/Video download siteSoftwares/Music/Video download site

# uses several separate disks# uses several separate diskscache.bases = ("/data/cache1", cache.bases = ("/data/cache1",

"/data/cache2", "/data/cache3", "/data/cache2", "/data/cache3", "/data/cache4") "/data/cache4")

cache.support-queries = "enable"cache.support-queries = "enable"cache.refresh-pattern = (cache.refresh-pattern = (

"\.(?i)(rar|zip|exe|wmv|avi|mp3|ape|rm|"\.(?i)(rar|zip|exe|wmv|avi|mp3|ape|rm|mpeg|mpg|wma|asf|rmvb)$" => "10080 mpeg|mpg|wma|asf|rmvb)$" => "10080 fetchall-for-range-request",fetchall-for-range-request","." => "120""." => "120"

))

Modcache Cook Books Modcache Cook Books (3)(3)

BBS Site

server.modules = (server.modules = ("mod_compress", #use it to compress html/css "mod_compress", #use it to compress html/css filesfiles"mod_cache", "mod_proxy""mod_cache", "mod_proxy"

))mimetype.assign = (mimetype.assign = (

".css" => "text/css",".css" => "text/css",".js" => "application/x-javascript",".js" => "application/x-javascript",".html" => "text/html",".html" => "text/html",".htm" => "text/html"".htm" => "text/html"

))

Modcache Cook Books Modcache Cook Books (3)(3)

BBS Site(continued)BBS Site(continued)# compress output for css/js/html/htm# compress output for css/js/html/htmcompress.filetype = ("application/x-javascript", "text/css", compress.filetype = ("application/x-javascript", "text/css",

"text/html")"text/html")cache.support-queries = "enable"cache.support-queries = "enable"cache.dynamic-mode = "enable"cache.dynamic-mode = "enable"cache.bases = ("/data/cache") cache.bases = ("/data/cache") cache.refresh-pattern = (cache.refresh-pattern = (

"\.(?i)(js|css|xml)$" => "240", "\.(?i)(js|css|xml)$" => "240", "\.(?i)(htm|html|shtml)$" => "30 ignore-vary", "\.(?i)(htm|html|shtml)$" => "30 ignore-vary", ".(?i)(/|php)$" => "5 use-memory update-on-refresh ".(?i)(/|php)$" => "5 use-memory update-on-refresh memory-compress", memory-compress", "." => "30 update-on-refresh" # default to update every 30 "." => "30 update-on-refresh" # default to update every 30 minutes and on refresh requestsminutes and on refresh requests

))

Modcache Cook Books Modcache Cook Books (4)(4)

Video share siteVideo share siteserver.modules = (server.modules = (

"mod_flv_streaming", "mod_cache", "mod_proxy""mod_flv_streaming", "mod_cache", "mod_proxy"

))

flv-streaming.extensions = (".flv")flv-streaming.extensions = (".flv")

cache.bases = ("/data/cache1", "/data/cache2", cache.bases = ("/data/cache1", "/data/cache2", "/data/cache3“) "/data/cache3“)

cache.support-queries = "enable"cache.support-queries = "enable"

cache.refresh-pattern = (cache.refresh-pattern = (

"\.(?i)(flv)$" => "0 flv-streaming", # flv_streaming for flv "\.(?i)(flv)$" => "0 flv-streaming", # flv_streaming for flv filesfiles

"." => "120""." => "120"

))

Thanks for Thanks for listeninglistening

Q&AQ&A