data shard with - jcconf.tw · mysql fabric and connector/j 杜修文 [email protected] mysql...

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1

Data Shard with MySQL Fabric and Connector/J

杜修文 [email protected] Principal Sales ConsultantOracle LLC

Nov. 15, 2014

mailto:[email protected]

2

Insert Picture Here

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.| Oracle Webinar | December 17, 2013 |

An extensible and easy-to-use framework for managing a farm of MySQL server supporting high-availability and sharding

MySQL Fabric

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.| Oracle Webinar | December 17, 2013 |3

MySQL Fabric: What is it?

● “Farm” Management System● Distributed Framework

● Procedure Execution● Farm Management

● Extensible● Extensions are first-class● High-Availability Groups● “Semi-Automatic” Sharding

● Written in Python

● GAed in May 2014● Open Source

● You can participate● Suggest features● Report bugs● Contribute patches

● MySQL 5.6 is the focus


MySQL Fabric: Goals & Features● Connector API Extensions

● Support Transactions● Support full SQL

● Decision logic in connector● Reducing network load

● Load Balancing● Read-Write Split● Distribute transactions

● Global Updates● Global tables● Schema updates

● Shard Multiple Tables● Using same key

● Sharding Functions● Range● (Consistent) Hash

● Shard Operations● Using built-in executor● Shard move● Shard split


Transaction Handling


● Routing Transactions● Pre-declare properties of transactions● Detecting transaction boundaries● Push as much as possible to server

● Managing Session State● Move session state between servers

– Easy to use– Expensive and error prone

● Reset state after each transaction– Transactions start with default session state

Transaction Handling

What aboutcrashes?

Where do I storethe session state?

What SessionState Change?


Routing Transactions

Shard #2

Shard #1

Shard #3

Switch

StateStore Executor

QUERYKEY

KEY

QUERY

Contain decision logicfor routing queries

Contain informationabout location of shards

SHARD#


Deployment for Routing Transactions

Shard #2

Shard #1

Shard #3

Switch

StateStore Executor

Network hop!Performance?

Protocol?

Single Pointsof Failure!


Deployment for Routing Transactions

Shard #2

Shard #1

Shard #3

Switch

StateStore Executor

Deployed with application(e.g., inside connector)

API simple to add

Caches can beused to avoid

performance impact

10

Insert Picture Here


Introducing MySQL Fabric


Birds-eye View

High Availability Groups (Shards)

MySQL FabricNode

Application

XML-RPC

SQL

SQL

Connector

Connector

Connector

Operator


High-Level Components

● Fabric-aware Connectors● Python, PHP, and Java● Extended Connector API

● MySQL Servers● In High-Availability Groups● Managing the data

● MySQL Fabric Node● Maintain Meta-Information● Information Interfaces● Execute Procedures

High Availability Group

Application

MySQLFabricNode


MySQL Fabric: Prerequisites

● MySQL Servers (version 5.6.10 or later)● Server for meta-data backing store● Servers being managed

● Python 2.6 or 2.7● No support for 3.x yet

● MySQL Utilities 1.5.3● Available at http://labs.mysql.com/


MySQL Fabric: Configuration

● Backing Store● MySQL server● Persistent storage for state● Storage engine-agnostic

● Protocol● Address where node will be● Currently only XML-RPC

● Logging● Chatty: INFO (default)● Moderate: WARNING● URL for rotating log

[storage]address = fabricDB:3306user = fabricpassword =database = fabricconnection_timeout = 6

[protocol.xmlrpc]address = fabricHost:32274threads = 5

[logging]level = INFOurl = file:///var/log/fabric.log

15

Insert Picture Here


Architecture forHigh-Availability


High-Availability Concepts

● Redundancy● Duplicate critical

components

● Monitoring● Detecting failing

components● Monitor load

● Procedure● Activate backups● Distribute load


High-Availability Group Concept

● Group of servers● Hardware redundancy● Data redundancy

● Generic Concept● Implementation-independent● Self-managed or externally

managed

● Different Types● Primary-Backup (Master-Slave)● Shared or Replicated Storage● MySQL Cluster

Done!


High-Availability Group Concept

● Abstract Concept● Set of servers● Server attributes

● Connector Attributes● Connection information● Mode: read-only, read-write, ...● Weight: distribute load

● Management Attributes● State: state/role of the server State: Primary

Mode: Read-WriteHost: server-1.example.com

19

Insert Picture Here


Connecting to aMySQL Fabric Farm


Fabric-aware Connector API

● Connector API Extensions● Support Transactions● Support full SQL


● Load Balancing● Read-Write Split● Distribute transactions

● Fabric-aware Connectors● Connector/J● Connector/Python● Connector/PHP

● Fabric-aware Frameworks● Doctrine● Hibernate

● Focus on Connector/Python


Fabric-aware Connector API

import mysql.connector.fabric as connector

conn = connector.MySQLFabricConnection( fabric={"host": "fabric.example.com", "port" : 32274}, user='mats', database="employees")

● Establish a “virtual” connection● Real server connection established lazily

● Provide connection information for the Fabric node● Fabric node will provide information about real servers


Connector API: Executing a Transaction

● Provide group name● Property: group● Fabric will compute

candidate servers

conn.set_property(group='my_group', type=TYPE_READWRITE)cur = conn.cursor()cur.execute("INSERT INTO employees VALUES (%s,%s,%s)", (emp_no, first_name, last_name))cur.execute("INSERT INTO titles(emp_no,title,from_date)" “ VALUES (%s,%s,CURDATE())", (emp_no, 'Intern'));conn.commit()

● Provide transaction type● Property: type● Fabric will pick server in

right mode

Transactions work fine!

23

Insert Picture Here


Architecture forSharding


Benefits of Sharding

● Write scalability● Can handle more writes

● Large data set● Database too large● Does not fit on single server

● Improved performance● Smaller index size● Smaller working set● Improve performance

UID 10000-20000 UID 20001-40000


MySQL Fabric: Sharding Goals & Features

● Connector API Extensions● Support Transactions● Support full SQL


● Shard Multiple Tables● Using same key

● Global Updates● Global tables● Schema updates

● Sharding Functions● Range● (Consistent) Hash

● Shard Operations● Using built-in executor● Shard move● Shard split


Mapping the Sharding Key

● What is a sharding key?● Single column● Multi column

– Same table?– Different tables?

● How is the key transformed?● Hash● Range● User-defined

ComputeShard#

Key

Shard#

(X)(X,Y,...)

RANGEHASHSomething else


Sharded Tables

In desperate needof sharding!

Foreign keys


Multi-table Query with Sharded Tables

SELECT first_name, last_name, salaryFROM salaries JOIN employees USING (emp_no)WHERE emp_no = 21012 AND CURRENT_DATE BETWEEN from_date AND to_date;

● Referential Integrity Constraint● Example query joining salaries and employees● Same key, same shard: co-locate rows for same user

● JOIN normally based on equality● Using non-equality defeats purpose of foreign key


Global Tables

Does not needto be sharded


Multi-table Query with Global Tables

SELECT first_name, last_name, GROUP_CONCAT(dept_name) FROM employees JOIN dept_emp USING (emp_no) JOIN departments USING (dept_no)WHERE emp_no = 21012 GROUP BY emp_no;

● JOIN with departments table● Has no employee number, hence no sharding key● Table need to be present on all shards

● How do we update global tables?


Sharding Architecture

Shards

MySQL Fabric Node

Application

GlobalGroup

Global Updates

ShardUpdates

Replication

Support global updatefor off-line shards


Digression: Computing Shards

● Multiple Mappings● Which mapping to use?● Application don't care● … but know tables in transaction● Currently only one mapping

● Computing shard requires● Tables + sharding key

● Extended Connector API● Extra properties passed out-of-band

ComputeShard#

Key

Shard#

Map#

ComputeMap#

Tables


Demo


Demo Server Side – Startup Instances

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314 3408 3406


Demo Server Side – Configuration of Fabric /etc/mysql/fabric.cfg[DEFAULT]prefix = sysconfdir = /etclogdir = /var/log[storage]address = localhost:3506user = fabricpassword = welcome1database = fabricauth_plugin = mysql_native_passwordconnection_timeout = 6connection_attempts = 6connection_delay = 1

[servers]user = fabricpassword = welcome1[protocol.xmlrpc]address = localhost:32274threads = 5user = adminpassword =welcome1 disable_authentication = norealm = MySQL Fabricssl_ca = ssl_cert = ssl_key = [executor]executors = 5[logging]level = INFOurl = file:///var/log/fabric.log


Startup Fabric

• Fabric help command mysqlfabric help commands• Instantiate Fabricmysqlfabric manage setup• Bring up Fabricmysqlfabric manage start --deamonize


Demo – Configure by Fabric

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314 3408 3406


Fabric Group Operation Create 1 global group, 2 shard groupmysqlfabric group create gp-globalmysqlfabric group create gp1mysqlfabric group create gp2 Assign Servers to groupsmysqlfabric group add gp-global 127.0.0.1:3606mysqlfabric group add gp1 127.0.0.1:3306mysqlfabric group add gp1 127.0.0.1:3308mysqlfabric group add gp1 127.0.0.1:3314mysqlfabric group add gp2 127.0.0.1:3406mysqlfabric group add gp2 127.0.0.1:3408 Pick the master server by Fabricmysqlfabric group promote gp1mysqlfabric group promote gp2


Fabric Group Operations

• Let Fabric manage auto-failover for youmysqlfabric group activate gp-globalmysqlfabric group activate gp1mysqlfabric group activate gp2• Define Range shard on to global groupmysqlfabric sharding create_definition RANGE gp-global• Add shard table – employees to the shard created (shard map id = 1)

shard by id columnmysqlfabric sharding add_table 1 employees.employees id• Assign rows with id value 1~999 to group gp1, 1000 above to group gp2mysqlfabric sharding add_shard 1 gp1/1,gp2/1000 --state=ENABLED


Demo – Databases is Ready

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314-M 3408-M 3406


Demo Application Side– Need Connector/J 5.1.30 or later

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314-M 3408-M 3406

JavaConnector/J


Demo Application Side– Cache Fabric Configuration on creating DB Connection

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314-M 3408-M 3406

JavaConnector/JConnection

Object


Java Application- Global tables and DDLs are accessed by property fabricServerGroup

import com.mysql.fabric.jdbc.FabricMySQLConnection;…baseUrl = “jdbc:mysql:fabric:// + fabricHost + ”:32274/”;rawConnection = DriverManager.getConnection(baseUrl + database + "?fabricServerGroup=gp-global“,user, password);statement = rawConnection.createStatement();…statement.executeUpdate("create table employees.employees (emp_no int not null,first_name varchar(50),last_name varchar(50), department_id int,primary key (emp_no))");…statement.executeUpdate("insert into employees.departments values (1,'Sales'),(2,'Financial')");


Demo Environment – Dispatch Request Based On Cached Configuration with Connection

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314-M 3408-M 3406


Object – Shard Table

Write Sharded Table

Write Global Table


Shared tables are accessed by property fabricShardTable and fabericShardKey

rawConnection = DriverManager.getConnection( "jdbc:mysql:fabric://127.0.0.1:32274/mysql?fabricShardTable=employees.employees“, user, password);

…for (int i = 0; i < 4; ++i) { connection.setShardKey(ids[i].toString()); ps.setInt(1, ids[i]); ps.setString(2, firstNames[i]); ps.setString(3, lastNames[i]); ps.setString(4, departmentIds[i]); ps.executeUpdate();}


Demo Environment – Read/Write Split within Shard

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314-M 3408-M 3406


Object – Shard Table

Read Only Operations


Read Only Operations

FabricMySQLConnection newConn = (FabricMySQLConnection) rawConnection;

newConn.setReadOnly(true);ps = newConn.prepareStatement( "select emp_no, first_name, last_name, d.name, i.variable_value from employees.employees e, information_schema.global_variables i, employees.departments d where e.department_id = d.department_id and i.variable_name='port' and emp_no = ?");


Demo Environment – Failure Detected

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308 3314-M 3408-M 3406


Object

Write Global Table

Change Master to 3606

Change Master to 3308

X


Demo Environment – Failover within Shard

GP1

3306

GP-Global

3606

GP2

FabricRepository

3506

Fabric Node

32274

3308-M 3314-M 3408-M 3406


Object

Write Shard Table

Write Global Table

X

50

Insert Picture Here


Closing Remarks


Reading for the Interested

● MySQL Forum: Fabric, Sharding, HA, Utilitieshttp://forums.mysql.com/list.php?144

● A Brief Introduction to MySQL Fabrichttp://mysqlmusings.blogspot.com/2013/09/brief-introduction-to-mysql-fabric.html

● MySQL Fabric – Sharding – Introductionhttp://vnwrites.blogspot.com/2013/09/mysqlfabric-sharding-introduction.html

● Writing a Fault-tolerant Database Application using MySQL Fabrichttp://alfranio-distributed.blogspot.se/2013/09/writing-fault-tolerant-database.html

http://forums.mysql.com/list.php?144

http://mysqlmusings.blogspot.com/2013/09/brief-introduction-to-mysql-fabric.html

http://vnwrites.blogspot.com/2013/09/mysqlfabric-sharding-introduction.html

http://alfranio-distributed.blogspot.se/2013/09/writing-fault-tolerant-database.html

data shard with - jcconf.tw · mysql fabric and connector/j 杜修文 [email protected] mysql...

Documents