Download - Mysql replication outside the box
![Page 1: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/1.jpg)
MySQL replication outside the box
Giuseppe MaxiaQA DirectorContinuent, Inc@datacharmer
This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License. 1Tuesday, October 25, 11
![Page 2: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/2.jpg)
about me -Giuseppe Maxia§ a.k.a. The Data Charmer§ QA Director, Continuent, Inc§ Long time hacking with database features§ Formerly, database consultant, designer, coder.§ A passion for QA§ An even greater passion for open source§ ... and community§ Passionate blogger§ http://datacharmer.blogspot.com
2Tuesday, October 25, 11
![Page 3: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/3.jpg)
Why Tungsten?
3Tuesday, October 25, 11
![Page 4: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/4.jpg)
BINARY LOG
REPLICATION
MySQLDBMS
trans
actio
n
trans
actio
n
trans
actio
n
trans
actio
n
trans
actio
ntra
nsac
tion
trans
actio
ntra
nsac
tion
trans
actio
ntra
nsac
tion
trans
actio
n
MySQLDBMS
transactiontransaction
transactiontransaction
transactiontransaction
transactiontransaction
transactiontransaction
transactiontransaction
MySQL replicationis single threaded
4Tuesday, October 25, 11
![Page 5: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/5.jpg)
MySQLDBMS
MySQLDBMS
MySQLDBMS
MySQLDBMS
master
slave slaveslave
MySQLDBMS
MySQLDBMS
MySQLDBMS
MySQLDBMS
master master master
slave
single source multi source (fan-in)
multiple sources?5Tuesday, October 25, 11
![Page 6: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/6.jpg)
MySQLDBMS
MySQLDBMS
MySQLDBMS
MySQLDBMS
mastermaster
master master
from this to this
multiple masters?
6Tuesday, October 25, 11
![Page 7: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/7.jpg)
MySQLDBMS
MySQLDBMS
MySQLDBMS
MySQLDBMS
mastermaster
master master
Avoiding conflicts?
INSERT RECORD
A
INSERT RECORD
A
MODIFY RECORD
B
MODIFY RECORD
B
7Tuesday, October 25, 11
![Page 8: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/8.jpg)
MySQLDBMS
MySQLDBMS
MySQLDBMS
master
slaveslave
MySQLDBMS
MySQLDBMS
MySQLDBMS
master
slaveslave
MySQLDBMS
MySQLDBMS
MySQLDBMS
master
slaveslave
Seamless failover?
8Tuesday, October 25, 11
![Page 9: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/9.jpg)
MySQLDBMS
MySQLDBMS
mysql master
postgresqlmysql
Replicating to something else?
oracle mongodb
9Tuesday, October 25, 11
![Page 10: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/10.jpg)
All these examples tell us:
Nice dream, but MySQL can’t do it
10Tuesday, October 25, 11
![Page 11: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/11.jpg)
Enter Tungsten Replicator
11Tuesday, October 25, 11
![Page 12: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/12.jpg)
http://code.google.com/p/tungsten-replicator
12Tuesday, October 25, 11
![Page 13: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/13.jpg)
http://code.google.com/p/tungsten-replicator
Open Source
12Tuesday, October 25, 11
![Page 14: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/14.jpg)
http://code.google.com/p/tungsten-replicator
100% GPL v2Open Source
12Tuesday, October 25, 11
![Page 15: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/15.jpg)
What can it do?
• Easy failover
• Multiple masters
• Multiple sources to a single slave
• Conflict prevention
• Parallel replication
• Replicate to Oracle and PostgreSQL database
13Tuesday, October 25, 11
![Page 16: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/16.jpg)
© Continuent 2010
MySQL to foreign services
BinLogs
Master DB
(Binlogs enabled) Slave DB
MySQL setup to run as
MySQL master
Data is applied to PostgreSQL or
Oracle or to NoSQL entities
14Tuesday, October 25, 11
![Page 17: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/17.jpg)
Tungsten Replicator in a nutshell
binlog THL
slavemaster
host1 host2
THL
trep_commit_seqnotrep_commit_seqnotrep_commit_seqnoorigin seqno eventid
trep_commit_seqnotrep_commit_seqnotrep_commit_seqnoorigin seqno eventid
global transaction ID
15Tuesday, October 25, 11
![Page 18: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/18.jpg)
Main components
• Transaction History Logs (THL)
• roughly corresponding to MySQL relay logs
• have a lot of metadata
• Service database
• contains metadata for latest transactions
• Metadata is committed together with data
• Makes slaves crash proof
16Tuesday, October 25, 11
![Page 19: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/19.jpg)
A failover scenario1: MySQL native replication
17Tuesday, October 25, 11
![Page 20: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/20.jpg)
© Continuent 2011
1. one Master, two slaves
/ Loading the “employees” test database
18Tuesday, October 25, 11
![Page 21: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/21.jpg)
© Continuent 2011
2. Master goes away
/ Stop replication
/ Slaves are updated at different levels
# 2select count(*) from titles 333,145
# 3
select count(*) from titles 443,308
19Tuesday, October 25, 11
![Page 22: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/22.jpg)
© Continuent 2011
3. Look into Slave #2 binary logs
/ find the last transaction
20Tuesday, October 25, 11
![Page 23: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/23.jpg)
© Continuent 2011
4. Look into Slave #3 binary logs
1. find the transaction that was last in slave #22. Recognize that last transaction in the log of slave #3
(This can actually take you a LOOOONG TIME)3. Get the position immediately after this transaction 4. (e.g. 134000 in file mysql-bin.000018)
21Tuesday, October 25, 11
![Page 24: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/24.jpg)
© Continuent 2011
5. promote Slave #3 to master
/ in slave #2
CHANGE MASTER TO master_host=‘slave_3_IP’,
master_user=‘slavename’,master_password=‘slavepassword’,master_log_file=‘mysql-bin.000018’,master_log_pos=134000;
22Tuesday, October 25, 11
![Page 25: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/25.jpg)
A failover scenario1: Tungsten Replicator
23Tuesday, October 25, 11
![Page 26: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/26.jpg)
© Continuent 2011
1. one master, two slaves
/ loading the ‘employees’ test database
24Tuesday, October 25, 11
![Page 27: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/27.jpg)
© Continuent 2011
2. Master goes away
/ Stop replication
/ Slaves are updated at different levels
# 2select count(*) from titles 333,145
# 3
select count(*) from titles 443,308
25Tuesday, October 25, 11
![Page 28: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/28.jpg)
© Continuent 2011
3. no need to find the last transaction
/ simply change roles
trepctl -host slave3 setrole -role mastertrepctl -host slave2 setrole -role slave \
-uri thl://slave3
trepctl -host slave3 online State: ONLINE
trepctl -host slave2 online State: GOING-ONLINE:SYNCHRONIZING
26Tuesday, October 25, 11
![Page 29: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/29.jpg)
© Continuent 2011
4. Check that the slave has synchronized
# new masterselect seqno from tungsten.trep_commit_seqno;78
# new slaveselect seqno from tungsten.trep_commit_seqno;64
27Tuesday, October 25, 11
![Page 30: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/30.jpg)
© Continuent 2011
4. Tell the replicator to hurry up
# new mastertrepctl -node slave3 flushMaster log is synchronized with database at log sequence number: 78
# new slavetrepctl host slave2 wait -applied 78ONLINE
select seqno from tungsten.trep_commit_seqno;78
28Tuesday, October 25, 11
![Page 31: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/31.jpg)
© Continuent 2011
4. ... and we’re done
# new masterselect count(*) from employees.titlescount(*)443308
# new slave: count(*)443308
29Tuesday, October 25, 11
![Page 32: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/32.jpg)
Parallel replication
30Tuesday, October 25, 11
![Page 33: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/33.jpg)
parallel replication
From here ...
31Tuesday, October 25, 11
![Page 34: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/34.jpg)
parallel replication
To here.
32Tuesday, October 25, 11
![Page 35: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/35.jpg)
Parallel replication facts
✓Sharded by database
✓Good choice for slave lag problems
❖Bad choice for single database projects
33Tuesday, October 25, 11
![Page 36: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/36.jpg)
Benchmarking parallel replication
34Tuesday, October 25, 11
![Page 37: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/37.jpg)
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
sysbench
db1
db2
db3
db4
db5
db6
db7
db8
db9
preparation (1)db0
35Tuesday, October 25, 11
![Page 38: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/38.jpg)
before the test (2)
binary logs
MySQL slave
Tungsten slave
OFFLINE
STOPPED
replicator alpha
direct: alpha(slave)
36Tuesday, October 25, 11
![Page 39: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/39.jpg)
starting the test
binary logs
MySQL slave
Tungsten slave
OFFLINE
STOPPED
replicator alpha
direct: alpha(slave)
Concurrent sysbenchon 30 databasesrunning for 1 hour
TOTAL DATA: 130 GBRAM per server: 20GB
Slaves will have 1 hour lag37Tuesday, October 25, 11
![Page 40: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/40.jpg)
measuring results
binary logs
MySQL slave
Tungsten slave
ONLINE
START
replicator alpha
direct: alpha(slave)
Recording catch-up time
38Tuesday, October 25, 11
![Page 41: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/41.jpg)
MySQL native replication
slave catch up in 04:29:30
39Tuesday, October 25, 11
![Page 42: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/42.jpg)
Tungsten parallel replication
slave catch up in 00:55:40
40Tuesday, October 25, 11
![Page 43: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/43.jpg)
Parallel replication made simpler
FROM HERE ....41Tuesday, October 25, 11
![Page 44: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/44.jpg)
Parallel replication made simpler
TO HERE42Tuesday, October 25, 11
![Page 45: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/45.jpg)
Parallel replication made simpler
43Tuesday, October 25, 11
![Page 46: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/46.jpg)
parallel replicationdirect slave facts
44Tuesday, October 25, 11
![Page 47: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/47.jpg)
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
44Tuesday, October 25, 11
![Page 48: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/48.jpg)
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
44Tuesday, October 25, 11
![Page 49: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/49.jpg)
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
✓Replication can revert to native slave with two commands (trepctl offline; start slave)
44Tuesday, October 25, 11
![Page 50: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/50.jpg)
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
✓Replication can revert to native slave with two commands (trepctl offline; start slave)
✓Native replication can continue on other slaves
44Tuesday, October 25, 11
![Page 51: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/51.jpg)
parallel replicationdirect slave facts
✓No need to install Tungsten on the master
✓Tungsten runs only on the slave
✓Replication can revert to native slave with two commands (trepctl offline; start slave)
✓Native replication can continue on other slaves
❖Failover (either native or Tungsten) becomes a manual task
44Tuesday, October 25, 11
![Page 52: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/52.jpg)
Multiple masters
45Tuesday, October 25, 11
![Page 53: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/53.jpg)
multiple masters
• Tungsten Replicator recipe: use more services
46Tuesday, October 25, 11
![Page 54: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/54.jpg)
Bi-directional replication
47Tuesday, October 25, 11
![Page 55: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/55.jpg)
Bi-directional replication with slaves
48Tuesday, October 25, 11
![Page 56: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/56.jpg)
Multiple sources
49Tuesday, October 25, 11
![Page 57: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/57.jpg)
multiple sources
• Tungsten Replicator recipe is still valid: use more services
50Tuesday, October 25, 11
![Page 58: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/58.jpg)
Multiple source replication
51Tuesday, October 25, 11
![Page 59: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/59.jpg)
Multiple masters replication: 3 nodes
52Tuesday, October 25, 11
![Page 60: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/60.jpg)
Multiple masters replication: 4 nodes
53Tuesday, October 25, 11
![Page 61: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/61.jpg)
Conflict prevention
54Tuesday, October 25, 11
![Page 62: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/62.jpg)
Conflict prevention facts
55Tuesday, October 25, 11
![Page 63: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/63.jpg)
Conflict prevention facts
• Sharded by database
55Tuesday, October 25, 11
![Page 64: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/64.jpg)
Conflict prevention facts
• Sharded by database
• Defined dynamically
55Tuesday, October 25, 11
![Page 65: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/65.jpg)
Conflict prevention facts
• Sharded by database
• Defined dynamically
• Applied either at the master or at the slave
55Tuesday, October 25, 11
![Page 66: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/66.jpg)
Conflict prevention facts
• Sharded by database
• Defined dynamically
• Applied either at the master or at the slave
• methods:
55Tuesday, October 25, 11
![Page 67: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/67.jpg)
Conflict prevention facts
• Sharded by database
• Defined dynamically
• Applied either at the master or at the slave
• methods:
• make replication fail
55Tuesday, October 25, 11
![Page 68: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/68.jpg)
Conflict prevention facts
• Sharded by database
• Defined dynamically
• Applied either at the master or at the slave
• methods:
• make replication fail
• drop silently
55Tuesday, October 25, 11
![Page 69: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/69.jpg)
Conflict prevention facts
• Sharded by database
• Defined dynamically
• Applied either at the master or at the slave
• methods:
• make replication fail
• drop silently
• drop with warning55Tuesday, October 25, 11
![Page 70: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/70.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Tungsten Conflict prevention points
56Tuesday, October 25, 11
![Page 71: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/71.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Tungsten Conflict prevention points
56Tuesday, October 25, 11
![Page 72: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/72.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Tungsten Conflict prevention points
56Tuesday, October 25, 11
![Page 73: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/73.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: Fail on master
57Tuesday, October 25, 11
![Page 74: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/74.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: Fail on masterINSERT A x,y
INSERT Ax,z
57Tuesday, October 25, 11
![Page 75: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/75.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: Fail on masterINSERT A x,y
INSERT Ax,z
57Tuesday, October 25, 11
![Page 76: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/76.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: Fail on slave
58Tuesday, October 25, 11
![Page 77: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/77.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: Fail on slaveINSERT A x,y
INSERT Ax,z
58Tuesday, October 25, 11
![Page 78: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/78.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: Fail on slaveINSERT A x,y
INSERT Ax,z
58Tuesday, October 25, 11
![Page 79: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/79.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: Fail on slave(Multiple sources)
59Tuesday, October 25, 11
![Page 80: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/80.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: Fail on slaveINSERT A x,y
INSERT Ax,z
(Multiple sources)
59Tuesday, October 25, 11
![Page 81: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/81.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: Fail on slaveINSERT A x,y
INSERT Ax,z
(Multiple sources)
59Tuesday, October 25, 11
![Page 82: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/82.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: DROP on master
60Tuesday, October 25, 11
![Page 83: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/83.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: DROP on masterINSERT A x,y
60Tuesday, October 25, 11
![Page 84: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/84.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: DROP on masterINSERT A x,y
INSERT Ax,z
60Tuesday, October 25, 11
![Page 85: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/85.jpg)
host1host3
master
host2
M
MS
SS
host4
S
Prevention methods: DROP on master
INSERT Ax,z
60Tuesday, October 25, 11
![Page 86: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/86.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: DROP on slave(Multiple sources)
61Tuesday, October 25, 11
![Page 87: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/87.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: DROP on slaveINSERT A x,y
INSERT Ax,z
(Multiple sources)
61Tuesday, October 25, 11
![Page 88: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/88.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: DROP on slave
INSERT Ax,z
(Multiple sources)
INSERT A x,y
62Tuesday, October 25, 11
![Page 89: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/89.jpg)
host1
master
host2
M
M
host3
S
Prevention methods: DROP on slave
INSERT Ax,z
(Multiple sources)
62Tuesday, October 25, 11
![Page 90: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/90.jpg)
Tungsten in practiceInstallation
63Tuesday, October 25, 11
![Page 91: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/91.jpg)
Installation
• Check the requirements
• Get the binaries
• Expand the tarball
• Run ./tools/tungsten-installer
64Tuesday, October 25, 11
![Page 92: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/92.jpg)
REQUIREMENTS
• Java JRE or JDK (Sun/Oracle or Open-jdk)
• Ruby 1.8 (only during installation)
• ssh access to the same user in all nodes
• MySQL user with all privileges
65Tuesday, October 25, 11
![Page 93: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/93.jpg)
Installation types
• master / slave
• slave - direct
66Tuesday, October 25, 11
![Page 94: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/94.jpg)
master/slave
binlog
THL
THL
slave
slave
master
host1host2
host3
THL
67Tuesday, October 25, 11
![Page 95: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/95.jpg)
slave direct
binlogTHL
slavemaster
relay log
host1host2
host3
THL
slave
relay log
68Tuesday, October 25, 11
![Page 96: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/96.jpg)
sample installation
• 4 servers
• qa.r1.continuent.com r1
• qa.r2.continuent.com r2
• qa.r3.continuent.com r3
• qa.r4.continuent.com r4
• MySQL 5.1
• tungsten-replicator-2.0.4
69Tuesday, October 25, 11
![Page 97: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/97.jpg)
Installation (1)# starting at node 4, but any would do
for N in 1 2 3 4do ssh r$N mkdir tinstalldone
cd tinstalltar -xzf /path/to/tungsten-replicator-2.0.4.tar.gzcd tungsten-replicator-2.0.4
70Tuesday, October 25, 11
![Page 98: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/98.jpg)
Installation (2)export TUNGSTEN_BASE=$HOME/tinstall
./tools/tungsten-installer \ --master-slave \ --master-host=r1 \ --datasource-user=tungsten \ --datasource-password=secret \ --service-name=dragon \ --home-directory=$TUNGSTEN_BASE \ --cluster-hosts=r1,r2,r3,r4 \ --start
1 2 3 4 5 6 7 8 9
71Tuesday, October 25, 11
![Page 99: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/99.jpg)
Installation (2)export TUNGSTEN_BASE=$HOME/tinstall
./tools/tungsten-installer \ --master-slave \ # installation mode --master-host=r1 \ # who’s the master --datasource-user=tungsten \ # mysql username --datasource-password=secret \ # mysql password --service-name=dragon \ # name of the service --home-directory=$TUNGSTEN_BASE \ # where we install --cluster-hosts=r1,r2,r3,r4 \ # hosts in cluster --start # start replicator after installing
72Tuesday, October 25, 11
![Page 100: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/100.jpg)
What does the installation do
host4
1: Validate all servers
✔✗✔
✔
host1
✔✗✔
✔
host2
✔✗✔
✔
host3
✔✗✔
✔
Report all errors73Tuesday, October 25, 11
![Page 101: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/101.jpg)
What does the installation do
host4
1 (again): Validate all servers
✔
✔
✔
host1
✔
✔
✔
host2
✔
✔
✔
host3
✔
✔
✔
✔✔ ✔ ✔
74Tuesday, October 25, 11
![Page 102: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/102.jpg)
What does the installation do
2: install Tungsten in all servers
host3
$HOME/ tinstall/ config/ releases/ relay/ logs/ tungsten/
host4host1
host2
75Tuesday, October 25, 11
![Page 103: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/103.jpg)
examplessh r2 chmod 444 $HOME/tinstall./tools/tungsten-installer \ --master-slave --master-host=r1 \ --datasource-user=tungsten \ --datasource-password=secret \ --service-name=dragon \ --home-directory=$HOME/tinstall \ --thl-directory=$HOME/tinstall/logs \ --relay-directory=$HOME/tinstall/relay \ --cluster-hosts=r1,r2,r3,r4 --start
ERROR >> qa.r2.continuent.com >> /home/tungsten/tinstall is not writeable
76Tuesday, October 25, 11
![Page 104: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/104.jpg)
examplessh r2 chmod 755 $HOME/tinstall./tools/tungsten-installer \ --master-slave --master-host=r1 \ --datasource-user=tungsten \ --datasource-password=secret \ --service-name=dragon \ --home-directory=$HOME/tinstall \ --thl-directory=$HOME/tinstall/logs \ --relay-directory=$HOME/tinstall/relay \ --cluster-hosts=r1,r2,r3,r4 --start
# no errors
77Tuesday, October 25, 11
![Page 105: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/105.jpg)
Installation - direct./tools/tungsten-installer \ --direct \ --master-host=r1 \ --slave-host=r4 \ --master-user=tungsten \ --slave-user=tungsten \ --master-password=secret \ --slave-password=secret \ --service-name=Castor \ --channels=1 \ --home-directory=$TUNGSTEN_BASE \ --start-and-report
78Tuesday, October 25, 11
![Page 106: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/106.jpg)
Tungsten in practiceAdmin
79Tuesday, October 25, 11
![Page 107: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/107.jpg)
Main components
• THL (Tungsten generated Transaction History Log)
• service database
80Tuesday, October 25, 11
![Page 108: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/108.jpg)
Tools
• replicator
• trepctl
• thl
81Tuesday, October 25, 11
![Page 109: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/109.jpg)
replicator
• It’s the service provider
• You launch it once when you start
• You may restart it when you change config
82Tuesday, October 25, 11
![Page 110: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/110.jpg)
trepctl
• Tungsten Replicator ConTroLler
• It’s the driving seat for your replication
• You can start, update, and stop services
• You can get specific info
83Tuesday, October 25, 11
![Page 111: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/111.jpg)
thl
• Transaction History List
• Gives you access to the Tungsten relay logs
84Tuesday, October 25, 11
![Page 112: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/112.jpg)
http://bit.ly/tr20_cookbook
85Tuesday, October 25, 11
![Page 113: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/113.jpg)
© Continuent 2011© Continuent 2011
Tungsten Replicator vs. Enterprise
Feature Replicator EnterpriseSimple admin with trepctl X XMySQL Row/Statement 5.0/5.1/5.5 X XMulti-master replication X XParallel apply (2.0) X XDistributed management with cctrl XAutomated failover and recovery XZero-downtime upgrade XSQL routing and load-balancing XFloating IP support X
86Tuesday, October 25, 11
![Page 114: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/114.jpg)
Conclusion and Q&A
87Tuesday, October 25, 11
![Page 115: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/115.jpg)
© Continuent 2011
Worldwide560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel (866) 998-3642 Fax (408) 668-1009e-mail: [email protected]
Contact Information
Continuent Web Site:http://www.continuent.com
Tungsten Projecthttp://tungsten-replicator.org
Getting started with Tungsten Replicator:http://datacharmer.blogspot.com
@datacharmer@continuent
88Tuesday, October 25, 11
![Page 116: Mysql replication outside the box](https://reader034.vdocuments.pub/reader034/viewer/2022051612/54bd42cd4a7959a8338b456d/html5/thumbnails/116.jpg)
89Tuesday, October 25, 11