Apache Web PerformanceLeveraging Apache to make your site fly

Who am I?&

Why would you listen to me?

Devon HillardWeb Application Architect

Built 10MinuteMail

Run Spark::red, an enterprise eCommerce Hosting company

Complex Web Apps with extremely high traffic and critical performance needs

devon@sparkred.com

What is Performance?

Server-Side Performance

• Traffic Capacity

• Request handling speed

• Request handling throughput

• Lower CPU/memory/bandwidth usage

• Scalability

Client-Side Performance

• Page Load Time

• First draw

• Inter-actable

• Complete

• Page Interaction Responsiveness

• Time to Complete Use Case

These two are TIED

• Browser caching of static assets reduces page load time AND reduces the number of requests the server has to handle for the same number of page views

• AJAX requests can typically be handled with far fewer resources than full page requests

• Reduced asset sizes means less bandwidth used and shorter request response sending times

• Solving for the Client brings Server performance gains!

Why Should I Care?

Everyone Wins!

• Client-Side Performance means

• happier users

• increased conversions

• increased SEO ranking

• Server-Side Performance means

• more capacity on same hardware

• saves money

• scaling is easier

Increased Conversions•+100 ms of page load time = 1% drop in sales

•+500 ms of page load time = 20% drop in searches

•+400 ms of page load time = 5-9% increase in clicking “back” button before the page finished loading

Why Apache?

Why I Use Apache• Popular web server - it’s everywhere

• Easy to install, troubleshoot, and find information on

• Mature and stable

• Lots of extensions

• Enterprise support requirements

• It Is fast enough to max out your hardware!

CPU Util @ 930 Mbit/sechttp://www.webperformance.com/load-testing/blog/2011/11/what-is-the-fastest-webserver/

PCA Awards• Saw 4x the planned for

traffic during spikes

• 140,000+ pages/minute - 2,333 pps (2012)

• 3,000+ Mbit/sec (2011)

• 1,200+ Mbit/sec PLUS Akamai CDN offloaded traffic (2012)

• The site stayed up and was quick to load and responsive to interact with the whole time

iftop output - 613 Mbit/sec

Puppies

Basic Apache Configs

Every Appis

Different

Which MPM?• Worker MPM scales for high traffic without

running out of memory better

• Less time spent tuning worker configs

• Unless you’re still using non-threadsafe Apache code (some PHP, etc..)

• New in Apache 2.4 is the Event MPM

Default<IfModule worker.c> StartServers 2 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0</IfModule>

Mine<IfModule worker.c> ThreadLimit 100 StartServers 5 MaxClients 5500 ServerLimit 200 MinSpareThreads 100 MaxSpareThreads 1000 ThreadsPerChild 100 MaxRequestsPerChild 0</IfModule>

MPM Worker Configs

ab with small fileVersion

Apache 2.2.3 MPM Worker

Document Size 119 bytes

Concurrency Level 1,000

Total Requests 100,000

Requests Per Second 20,790 r/s

Throughput 8,077 kb/s

hex core X5675

ab with large fileVersion

Apache 2.2.3 MPM Worker

Document Size 95,002 bytes

Concurrency Level 1,000

Total Requests 100,000

Requests Per Second 1,258 r/s

Throughput 119,966 kb/s

hex core X5675

Linux Kernel Configsnet.ipv4.tcp_max_syn_backlog=30000

net.core.netdev_max_backlog=1200

net.ipv4.tcp_timestamps=0

net.ipv4.tcp_sack=0

net.ipv4.tcp_window_scaling=0

net.ipv4.tcp_max_tw_buckets=2000000

net.ipv4.tcp_mem=100000000 100000000 100000000

net.ipv4.tcp_wmem=100000000 100000000 100000000

net.ipv4.tcp_rmem=30000000 30000000 30000000

net.ipv4.ip_conntrack_max = 231072

Keepalive - On or Off?• reduces overhead of establishing new

connections for each request from the browser

• can waste memory and other resources if left open too long

• Common practice is to disable them

• I turn keepalive on, set to 6 seconds or 500 requests

• For CDNs like Akamai, you’ll want to turn up the time to more like 120 seconds

GZip - mod_deflate• gzip compressing text resources - html, js, xml,

css dramatically decreases the size of the response for those assets: often up to 90% reduction in size

• reduces transfer time, especially for clients with slower connections (or big files)

• This decreases page load time (Client) and reduces the time the server thread is sending the response (Server)

# Removing Hosts varyHeader unset Vary

<IfModule mod_deflate.c>AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css application/x-javascriptBrowserMatch ^Mozilla/4 gzip-only-text/htmlBrowserMatch ^Mozilla/4\.0[678] no-gzipBrowserMatch \bMSIE !no-gzip !gzip-only-text/html

</IfModule>

Browser Caching• Tell the browser to cache all static assets, for as

long as you can bear it

• Set the ETag

• Set cache control and expiration response headers

<LocationMatch '^/pca/images/.*'>FileETag MTime SizeHeader set Cache-Control "max-age=2764800, public"<IfModule mod_expires.c>

ExpiresActive OnExpiresByType image/jpg A2764800ExpiresByType image/jpeg A2764800

ExpiresByType image/png A2764800</IfModule>

</LocationMatch>

Disk Caching• Why not use mod_mem_cache?

• Transparent web server caching of assets reduces load without deployment and development complexities

• Some warnings!

• Doesn’t auto clean itself

• Can grow to be huge if you don’t check the headers being cached

• Can grow to be huge if you have dynamic URI content - blah.jpg;jsessionid=foo

• Another place to purge cache for changed files

Disk Caching - Config<IfModule mod_disk_cache.c>

CacheRoot /var/cache/mod_disk_cache CacheDirLevels 2 CacheDirLength 1

CacheEnable disk /pca/img CacheEnable disk /pca/flash CacheEnable disk /pca/css CacheEnable disk /pca/js CacheEnable disk /pca/images CacheMaxFileSize 200715200 CacheDefaultExpire 3600 CacheIgnoreHeaders Set-Cookie</IfModule>

Mod_PageSpeed• Open Source Apache Module• Lots and Lots of filters

• image compression• combine js/css• sprite images• domain sharding• etc...

Three Tier Architecture

• SSL Termination• Load Balancing• Mod_proxy/mod_cluster• Mod_Security

Building Scalable Clusters w/Apache

• VIP• heartbeat• haproxy• DNS LB

• simple round robin• smart

Questions?

Thank you!

devon@sparkred.com