mysql replication outside the box

MySQL replication outside the box

Giuseppe MaxiaQA DirectorContinuent, Inc@datacharmer

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. 1Tuesday, October 25, 11

about me -Giuseppe Maxia§ a.k.a. The Data Charmer§ QA Director, Continuent, Inc§ Long time hacking with database features§ Formerly, database consultant, designer, coder.§ A passion for QA§ An even greater passion for open source§ ... and community§ Passionate blogger§ http://datacharmer.blogspot.com

2Tuesday, October 25, 11

Why Tungsten?

BINARY LOG

REPLICATION

MySQLDBMS

transactiontransaction

MySQL replicationis single threaded

MySQLDBMS

master

slave slaveslave

MySQLDBMS

master master master

single source multi source (fan-in)

multiple sources?5Tuesday, October 25, 11

MySQLDBMS

mastermaster

master master

from this to this

multiple masters?

MySQLDBMS

mastermaster

master master

Avoiding conflicts?

INSERT RECORD

MODIFY RECORD

MySQLDBMS

master

slaveslave

MySQLDBMS

master

slaveslave

MySQLDBMS

master

slaveslave

Seamless failover?

MySQLDBMS

mysql master

postgresqlmysql

Replicating to something else?

oracle mongodb

All these examples tell us:

Nice dream, but MySQL can’t do it

Enter Tungsten Replicator

http://code.google.com/p/tungsten-replicator

Open Source

100% GPL v2Open Source

What can it do?

• Easy failover

• Multiple masters

• Multiple sources to a single slave

• Conflict prevention

• Parallel replication

• Replicate to Oracle and PostgreSQL database

MySQL to foreign services

BinLogs

Master DB

(Binlogs enabled) Slave DB

MySQL setup to run as

MySQL master

Data is applied to PostgreSQL or

Oracle or to NoSQL entities

Tungsten Replicator in a nutshell

binlog THL

slavemaster

host1 host2

trep_commit_seqnotrep_commit_seqnotrep_commit_seqnoorigin seqno eventid

global transaction ID

Main components

• Transaction History Logs (THL)

• roughly corresponding to MySQL relay logs

• have a lot of metadata

• Service database

• contains metadata for latest transactions

• Metadata is committed together with data

• Makes slaves crash proof

A failover scenario1: MySQL native replication

1. one Master, two slaves

/ Loading the “employees” test database

2. Master goes away

/ Stop replication

/ Slaves are updated at different levels

# 2select count(*) from titles 333,145

select count(*) from titles 443,308

3. Look into Slave #2 binary logs

/ find the last transaction

4. Look into Slave #3 binary logs

1. find the transaction that was last in slave #22. Recognize that last transaction in the log of slave #3

(This can actually take you a LOOOONG TIME)3. Get the position immediately after this transaction 4. (e.g. 134000 in file mysql-bin.000018)

5. promote Slave #3 to master

/ in slave #2

CHANGE MASTER TO master_host=‘slave_3_IP’,

master_user=‘slavename’,master_password=‘slavepassword’,master_log_file=‘mysql-bin.000018’,master_log_pos=134000;

A failover scenario1: Tungsten Replicator

1. one master, two slaves

/ loading the ‘employees’ test database

2. Master goes away

/ Stop replication

/ Slaves are updated at different levels

# 2select count(*) from titles 333,145

select count(*) from titles 443,308

3. no need to find the last transaction

/ simply change roles

trepctl -host slave3 setrole -role mastertrepctl -host slave2 setrole -role slave \

-uri thl://slave3

trepctl -host slave3 online State: ONLINE

trepctl -host slave2 online State: GOING-ONLINE:SYNCHRONIZING

4. Check that the slave has synchronized

# new masterselect seqno from tungsten.trep_commit_seqno;78

# new slaveselect seqno from tungsten.trep_commit_seqno;64

4. Tell the replicator to hurry up

# new mastertrepctl -node slave3 flushMaster log is synchronized with database at log sequence number: 78

# new slavetrepctl host slave2 wait -applied 78ONLINE

select seqno from tungsten.trep_commit_seqno;78

4. ... and we’re done

# new masterselect count(*) from employees.titlescount(*)443308

# new slave: count(*)443308

Parallel replication

parallel replication

From here ...

parallel replication

To here.

Parallel replication facts

✓Sharded by database

✓Good choice for slave lag problems

❖Bad choice for single database projects

Benchmarking parallel replication

sysbench

preparation (1)db0

before the test (2)

binary logs

MySQL slave

Tungsten slave

OFFLINE

STOPPED

replicator alpha

direct: alpha(slave)

starting the test

binary logs

MySQL slave

Tungsten slave

OFFLINE

STOPPED

replicator alpha

Concurrent sysbenchon 30 databasesrunning for 1 hour

TOTAL DATA: 130 GBRAM per server: 20GB

Slaves will have 1 hour lag37Tuesday, October 25, 11

measuring results

binary logs

MySQL slave

Tungsten slave

ONLINE

replicator alpha

Recording catch-up time

MySQL native replication

slave catch up in 04:29:30

Tungsten parallel replication

slave catch up in 00:55:40

Parallel replication made simpler

FROM HERE ....41Tuesday, October 25, 11

TO HERE42Tuesday, October 25, 11

parallel replicationdirect slave facts

✓No need to install Tungsten on the master

✓Tungsten runs only on the slave

✓Replication can revert to native slave with two commands (trepctl offline; start slave)

✓Native replication can continue on other slaves

❖Failover (either native or Tungsten) becomes a manual task

Multiple masters

multiple masters

• Tungsten Replicator recipe: use more services

Bi-directional replication

Bi-directional replication with slaves

Multiple sources

multiple sources

• Tungsten Replicator recipe is still valid: use more services

Multiple source replication

Multiple masters replication: 3 nodes

Multiple masters replication: 4 nodes

Conflict prevention

Conflict prevention facts

• Sharded by database

• Defined dynamically

• Applied either at the master or at the slave

• methods:

• make replication fail

• methods:

• drop silently

• methods:

• drop silently

• drop with warning55Tuesday, October 25, 11

host1host3

master

Tungsten Conflict prevention points

host1host3

master

host1host3

master

host1host3

master

Prevention methods: Fail on master

host1host3

master

Prevention methods: Fail on masterINSERT A x,y

INSERT Ax,z

host1host3

master

Prevention methods: Fail on masterINSERT A x,y

INSERT Ax,z

host1host3

master

Prevention methods: Fail on slave

host1host3

master

Prevention methods: Fail on slaveINSERT A x,y

INSERT Ax,z

host1host3

master

INSERT Ax,z

master

Prevention methods: Fail on slave(Multiple sources)

master

INSERT Ax,z

(Multiple sources)

master

INSERT Ax,z

(Multiple sources)

host1host3

master

Prevention methods: DROP on master

host1host3

master

Prevention methods: DROP on masterINSERT A x,y

host1host3

master

Prevention methods: DROP on masterINSERT A x,y

INSERT Ax,z

host1host3

master

Prevention methods: DROP on master

INSERT Ax,z

master

Prevention methods: DROP on slave(Multiple sources)

master

Prevention methods: DROP on slaveINSERT A x,y

INSERT Ax,z

(Multiple sources)

master

Prevention methods: DROP on slave

INSERT Ax,z

(Multiple sources)

INSERT A x,y

master

Prevention methods: DROP on slave

INSERT Ax,z

(Multiple sources)

Tungsten in practiceInstallation

Installation

• Check the requirements

• Get the binaries

• Expand the tarball

• Run ./tools/tungsten-installer

REQUIREMENTS

• Java JRE or JDK (Sun/Oracle or Open-jdk)

• Ruby 1.8 (only during installation)

• ssh access to the same user in all nodes

• MySQL user with all privileges

Installation types

• master / slave

• slave - direct

master/slave

binlog

master

host1host2

slave direct

binlogTHL

slavemaster

relay log

host1host2

relay log

sample installation

• 4 servers

• qa.r1.continuent.com r1

• MySQL 5.1

• tungsten-replicator-2.0.4

Installation (1)# starting at node 4, but any would do

for N in 1 2 3 4do ssh r$N mkdir tinstalldone

cd tinstalltar -xzf /path/to/tungsten-replicator-2.0.4.tar.gzcd tungsten-replicator-2.0.4

Installation (2)export TUNGSTEN_BASE=$HOME/tinstall

./tools/tungsten-installer \ --master-slave \ --master-host=r1 \ --datasource-user=tungsten \ --datasource-password=secret \ --service-name=dragon \ --home-directory=$TUNGSTEN_BASE \ --cluster-hosts=r1,r2,r3,r4 \ --start

1 2 3 4 5 6 7 8 9

Installation (2)export TUNGSTEN_BASE=$HOME/tinstall

./tools/tungsten-installer \ --master-slave \ # installation mode --master-host=r1 \ # who’s the master --datasource-user=tungsten \ # mysql username --datasource-password=secret \ # mysql password --service-name=dragon \ # name of the service --home-directory=$TUNGSTEN_BASE \ # where we install --cluster-hosts=r1,r2,r3,r4 \ # hosts in cluster --start # start replicator after installing

What does the installation do

1: Validate all servers

✔✗✔

Report all errors73Tuesday, October 25, 11

1 (again): Validate all servers

✔✔ ✔ ✔

2: install Tungsten in all servers

$HOME/ tinstall/ config/ releases/ relay/ logs/ tungsten/

host4host1

examplessh r2 chmod 444 $HOME/tinstall./tools/tungsten-installer \ --master-slave --master-host=r1 \ --datasource-user=tungsten \ --datasource-password=secret \ --service-name=dragon \ --home-directory=$HOME/tinstall \ --thl-directory=$HOME/tinstall/logs \ --relay-directory=$HOME/tinstall/relay \ --cluster-hosts=r1,r2,r3,r4 --start

ERROR >> qa.r2.continuent.com >> /home/tungsten/tinstall is not writeable

examplessh r2 chmod 755 $HOME/tinstall./tools/tungsten-installer \ --master-slave --master-host=r1 \ --datasource-user=tungsten \ --datasource-password=secret \ --service-name=dragon \ --home-directory=$HOME/tinstall \ --thl-directory=$HOME/tinstall/logs \ --relay-directory=$HOME/tinstall/relay \ --cluster-hosts=r1,r2,r3,r4 --start

# no errors

Installation - direct./tools/tungsten-installer \ --direct \ --master-host=r1 \ --slave-host=r4 \ --master-user=tungsten \ --slave-user=tungsten \ --master-password=secret \ --slave-password=secret \ --service-name=Castor \ --channels=1 \ --home-directory=$TUNGSTEN_BASE \ --start-and-report

Tungsten in practiceAdmin

Main components

• THL (Tungsten generated Transaction History Log)

• service database

• replicator

• trepctl

• thl

replicator

• It’s the service provider

• You launch it once when you start

• You may restart it when you change config

trepctl

• Tungsten Replicator ConTroLler

• It’s the driving seat for your replication

• You can start, update, and stop services

• You can get specific info

• Transaction History List

• Gives you access to the Tungsten relay logs

http://bit.ly/tr20_cookbook

Tungsten Replicator vs. Enterprise

Feature Replicator EnterpriseSimple admin with trepctl X XMySQL Row/Statement 5.0/5.1/5.5 X XMulti-master replication X XParallel apply (2.0) X XDistributed management with cctrl XAutomated failover and recovery XZero-downtime upgrade XSQL routing and load-balancing XFloating IP support X

Conclusion and Q&A

Worldwide560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009e-mail: sales@continuent.com

Contact Information

Continuent Web Site:http://www.continuent.com

Tungsten Projecthttp://tungsten-replicator.org

Getting started with Tungsten Replicator:http://datacharmer.blogspot.com

@datacharmer@continuent

mysql replication outside the box

Technology

dna replication

doag 2011: mysql replication

what's new in mysql 5.7 replication

mysql数据库的致性与可靠性bos.itdks.com/76c213d5118448c1969254dfde165db6.pdf ·...

consistency and replication - nanjing...

rabies virus replication outside the central...

mysql開発最新動向 oracle mysql cloud...

kiyonori kitasako solutions...

replication 설치 및 구성 vmware vsphere...1 vsphere...

mysql replication devconf 2012

mysql for the web -...

データウィンドウプログラマーズガイド ·...

mysql at netease -...

droonga...

mysq replication

itdks.su.bcebos.com · mysql vs hbase mysql - w t «...

models of dna replication4th semester c8t: molecular biology...

mysql56 replication

powerscript® リファレンスボリューム 1 ·...

art of mysql replication