Download - Drupal Mysql Webinar Percona
Drupal and MySQLMike Benshoof
November 13, 2013
www.percona.com
Agenda
● Overview● Version Differences● Database Configuration● Scaling/HA● Tuning/Performance
www.percona.com
What is Drupal?
● Open Source CMS● LAMP Stack Based
● Originally PHP/MySQL● Now, includes any DB supported by PDO
● http://www.drupal.org
www.percona.com
Version Differences
● Limiting Discussion to MySQL Differences● Per Drupal's “Converting 6.x to 7.x” Doc:
● “Drupal 7 introduces a completely new database API”
● Some of the biggest differences include:● Dynamic Query Builders● Formal Prepared Statements● Specific INSERT/UPDATE/DELETE functions● PDO is now REQUIRED (major change)
www.percona.com
Sample Differences
● Standard SELECT – note the bound variables (i.e. :uid, :type)
● Update – uses db_update with bound variables
www.percona.com
Conceptual Differences
● Drupal 6● database.inc & database.mysql.inc are more procedural● Native MySQL interaction (mysql/mysqli)● mysqli_connect, mysqli_query, etc
● Drupal 7● database.inc is object oriented and driver based● PDO MySQL interaction with bound parameters and
prepared statements ● Database::GetConnection()->prepare()->execute()
www.percona.com
Database Configuration
● Database configuration is broken down into key/target pairs● Key represents active database “pool”● Target represents single database in pool
● Valid targets are “default” or “slave”● Keys are user defined● Full configuration located in
sites/default/settings.php (or specific site)
www.percona.com
Configuration (cont)
● Basic Configuration – Single server, all traffic sent to this host
● Note, this is set up by default with the installer
www.percona.com
Configuration (cont)
● Secondary Database – Manually specify this connection in a module
● Can be MySQL or other PDO data source
www.percona.com
Configuration (cont)
● Multiple Targets – This is set up for master/slave
● Default/Slave are the only options
● NOTE: if using multiple slaves, one will be chosen at random for each request
www.percona.com
Great... Now What??
How do you utilize different databases from code?
www.percona.com
Scaling the Database
● This is where the Default/Slave configuration comes into play● “Drupal may try to connect to a slave server when
appropriate and if one is not available will simply fall back to the single master server”
● Send reads to the slave (with read-only user)● Send writes to the master (with write user)● “Stick” delay-sensitive queries to master● Support for this is limited in Drupal Core
www.percona.com
In code, por favor?
● As parts of the Core don't utilize by default, it may be needed to manually send reads to the slave
● Simply add 'target' => 'slave' to the options
● Note – this can impact replication delay-sensitive queries
www.percona.com
Disabling Slave Servers
● In lag-sensitive environments, you can explicitly disable the use of slave servers:● db_ignore_slave()
● You can also force to the master for specific by using the same logic as slave:● array('target' => 'default')
● $_SESSION['ignore_slave_server']● Skips slave for a set number of seconds● Good for write followed by read(s)
www.percona.com
And External Connections?
● This is where the distinct database “key” comes into play
● Change the active database
● You can also utilize default/slave databases here as well
www.percona.com
Availability
● Splitting reads/writes can be complicated and in some cases, require manual code changes
● Fortunately, HA is simple (from application standpoint)● Use base configuration (default, default)● Default db server is actually a VIP● Failover is hidden from the application● Also, manual failover via the application is an option
www.percona.com
Availability
● Different options● Simple master/slave with manual failover
(reconfigure application config)● Master/Slave and shift IP on database (MHA)● Master/Master behind VIP (HAProxy, BigIP, etc)● PXC behind a VIP● PRM to manage failover and VIP
● In most cases, the application will just shift traffic and not need to be reconfigured
www.percona.com
Default Install Tuning
● Use Memcache!!● Semaphore/Locking API● Profile early and often!
www.percona.com
Cache != Faster
● By default, cache is stored in MySQL● This actually DOUBLES the number of
database queries in normal page browsing● This assumes populated cache
● Cache misses will actually increase the MySQL traffic even more, adding WRITE traffic● Even with no content being created, you have major
write impact on the database
www.percona.com
Benchmarking
● Simple JMeter Test Plan● Browse home page nodes● Randomly select a few nodes/pages● Data populated using the development module● 10 iterations of proxy-captured test plan
www.percona.com
Default Install Profiling
# 1.3s user time, 10ms system time, 24.32M rss, 200.27M vsz# Current date: Mon Sep 30 19:46:41 2013# Hostname: centos6-webdev# Files: /var/lib/mysql/centos6-webdev-slow.log# Overall: 3.50k total, 29 unique, 17.84 QPS, 0.06x concurrency __________# Time range: 2013-09-30 19:39:54 to 19:43:10
# Profile# Rank Query ID Response time Calls R/Call V/M Item# ==== ================== ============= ===== ====== ===== ===============# 1 0xCE1FFB25CB1ACF8E 0.5009 3.9% 530 0.0009 0.04 SELECT cache_bootstrap# 2 0xC854EADB9190C72F 1.8323 14.4% 470 0.0039 0.43 SELECT cache_menu# 3 0xE9D33E090967923F 4.7602 37.5% 403 0.0118 1.11 SELECT cache_field# 4 0x09630F7EE44B0750 0.3089 2.4% 342 0.0009 0.07 SELECT cache# 5 0xAD1BA3CA5E2D6F3B 0.1682 1.3% 139 0.0012 0.03 SELECT url_alias
● Full query capture (long_query_time=0)● Running in VM, so sorted by count● pt-query-digest --limit=30 /var/lib/mysql/slow.log –order-by=Query_time:cnt● Note: This assumes a populated cache
www.percona.com
Default Install - Continued
● No indexing issues in top queries ● Primary key or CONST index hits
● Top 4 queries are all SELECT … cache_%● 48% of all queries (1745 out of 3.5k)● 90% of bytes sent to the client● 22% of rows examined
● When considering query optimization, the best query is one that is never run...
www.percona.com
Enter memcached...
# 800ms user time, 100ms system time, 22.31M rss, 199.78M vsz# Current date: Mon Sep 30 20:14:26 2013# Hostname: centos6-webdev# Files: /var/lib/mysql/centos6-webdev-slow.log# Overall: 1.74k total, 25 unique, 16.59 QPS, 0.05x concurrency __________# Time range: 2013-09-30 20:12:08 to 20:13:53
# Profile# Rank Query ID Response time Calls R/Call V/M Item# ==== ================== ============= ===== ====== ===== ===============# 1 0xAD1BA3CA5E2D6F3B 0.0304 0.6% 139 0.0002 0.00 SELECT url_alias# 2 0xB83088CAE4EA7A29 0.1669 3.2% 90 0.0019 0.06 SELECT url_alias# 3 0xBAD26E72FBCD69FE 0.3140 6.0% 90 0.0035 0.03 SELECT blocked_ips# 4 0xCC47B42511EA22DD 0.2465 4.7% 90 0.0027 0.21 SET# 5 0x09E2CE64B0B5C525 0.0758 1.4% 89 0.0009 0.03 SET
● Note the huge drop in total queries
● 3.5k → 1.74k
● Also note the 4 query difference in unique
● 29 unique → 25 unique
● Only queries missing are the cache_% tables
www.percona.com
Locking API
● Important for preventing cache stampedes● Wrap expensive calls with
locking (i.e. variable_initialize)
● Default uses semaphore table in MySQL (InnoDB)● Write overhead to database
(flushing, redo log, etc)
● Does it need to be durable?
www.percona.com
Locking API (cont)
● Sample DB calls for a single page load:[root@centos6-webdev drupal]# tail -f /var/lib/mysql/centos6-webdev-slow.log | grep -i semaphore
INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ... INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...DELETE FROM semaphore WHERE name ...INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...INSERT INTO semaphore (name, value, expire) VALUES ...DELETE FROM semaphore WHERE name ...
● 8 write queries for single page load (redo log, page flushing, etc)● Depending on InnoDB configuration (innodb_flush_log_at_trx_commit, etc),
this can lead to several IOPs (expensive)
www.percona.com
Locking Alternatives
● Memory engine for semaphore table
● Removes InnoDB overhead
● No disk access needed – table in RAM
● Native implementation (no plugin or modules)
● Non-durable
● Negative – table-level locking can reduce concurrency, but this table is typically small (and hopefully not used often)
● Memcache locking
● Since you are already using memcache... (right?)
● Uses the (global) Memcache::Add operation
● Non-durable as well
www.percona.com
Profiling – Early and Often
● Finding problems early is key● Missing indexes, etc
● long_query_time = 0 ● Full verbosity (if using Percona Server)
● Pt-query-digest● Rows_examined > Rows_sent (suboptimal index)● Execution time● --review – track new queries over time (new
modules, etc)
www.percona.com
Questions?