![Page 1: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/1.jpg)
1 1
Scaling PostgreSQL on Postgres-XL
February, 2015 Mason Sharp PGConf.ru
![Page 2: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/2.jpg)
2
Agenda
• Intro • Postgres-XL Background
• Architecture Overview
• Distributing Tables
• Configuring a Local Cluster
• Differences to PostgreSQL • Community
![Page 3: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/3.jpg)
3
whoami
• Co-organizer of NYC PUG • I like database clusters
• GridSQL • Postgres-XC • Postgres-XL
• Work: • TransLattice • StormDB • EnterpriseDB
![Page 4: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/4.jpg)
4
PGConf.US
• March 25–27, 2015
• New York City
• 44 Talks
• 4 Tracks!
![Page 5: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/5.jpg)
7
Postgres-XL
n Open Source
n Scale-out RDBMS – MPP for OLAP
– OLTP write and read scalability
– Custer-wide ACID properties
– Multi-tenant security
– Can also act as distributed key-value store
![Page 6: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/6.jpg)
8
Postgres-XL
n Hybrid Transactional and Analytical Processing (HTAP)
n Can replace multiple systems
![Page 7: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/7.jpg)
9
Postgres-XL
n Rich SQL support
– Correlated Subqueries
![Page 8: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/8.jpg)
10
![Page 9: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/9.jpg)
13
Postgres-XL Features
• Large subset of PostgreSQL SQL • Distributed Multi-version Concurrency Control
• Contrib-friendly
• Full Text Search
• JSON and XML Support
• Distributed Key-Value Store
![Page 10: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/10.jpg)
14
Postgres-XL Missing Features
• Built-in High Availability • Use external like Corosync/Pacemaker • Area for future improvement
• “Easy” to re-shard / add nodes • Some pieces there • Tables locked during redistribution • Can however “pre-shard” multiple nodes on
the same server or VM
• Some FK and UNIQUE constraints • Recursive CTEs (WITH)
![Page 11: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/11.jpg)
15
Performance
Transactional work loads
10 50 100 200 300 4000
2000
4000
6000
8000
10000
12000
14000
16000
18000
Dell DVD Benchmark
StormDBAmazon RDS
![Page 12: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/12.jpg)
16
OLTP Characteristics
n Coordinator Layer adds about 30% overhead to DBT-1 (TPC-W) workload – Single node (4 cores) gets 70% performance compared
to vanilla PostgreSQL
n If linear scaling 10 nodes should achieve 7x performance compared to PostgreSQL
n Actual result: 6 – 6.4 x
n More nodes, more open transactions, means larger snapshots and more visibility checking
![Page 13: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/13.jpg)
17
MPP Performance – DBT-3 (TPC-H)
Postgres-XL
PostgreSQL
Postgres-XL
![Page 14: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/14.jpg)
18
OLAP Characteristics
n Depends on schema n Star schema?
– Distribute fact tables across nodes – Replicate dimensions
n Linear/near-linear query performance for straight-forward queries
n Other schemes different results – Sometimes “better than linear” due to more caching
![Page 15: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/15.jpg)
19
Key-value Store Comparison
Postgres-XL
MongoDB
![Page 16: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/16.jpg)
21
Multiversion Concurrency
Control (MVCC)
![Page 17: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/17.jpg)
22 July 12th, 2012 Postgres-XC 22
MVCC
l Readers do not block writers l Writers do not block readers l Transaction Ids (XIDs)
l Every transaction gets an ID
l Snapshots contain a list of running XIDs
![Page 18: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/18.jpg)
23 July 12th, 2012 Postgres-XC 23
MVCC – Regular PostgreSQL
Example:
T1 Begin...
T2 Begin; INSERT...; Commit;
T3 Begin...
T4 Begin; SELECT
l T4's snapshot contains T1 and T3
l T2 already committed l T4 can see T2's commits, but not T1's nor T3's
Node
![Page 19: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/19.jpg)
24 July 12th, 2012 Postgres-XC 24
Example:
T1 Begin...
T2 Begin; INSERT...; Commit;
T3 Begin...
T4 Begin; SELECT
l Node 1: T2 Commit, T4 SELECT l Node 2: T4 SELECT, T2 Commit
l T4's SELECT statement returns inconsistent data l Includes data from Node1, but not Node2.
MVCC – Multiple Node Issues
Coord
N1 N2
![Page 20: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/20.jpg)
25 July 12th, 2012 Postgres-XC 25
Postgres-XL Solution: Ø Make XIDs and Snapshots cluster-wide
MVCC – Multiple Node Issues
![Page 21: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/21.jpg)
26
Postgres-XL Technology
• Users connect to any Coordinator
![Page 22: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/22.jpg)
27
Postgres-XL Technology
• Data is automatically spread across cluster
![Page 23: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/23.jpg)
28
Postgres-XL Technology
• Queries return back data from all nodes in parallel
![Page 24: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/24.jpg)
29
Postgres-XL Technology
• GTM maintains a consistent view of the data
![Page 25: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/25.jpg)
30
Standard PostgreSQL
Parser
Planner
Executor
MVCC and Transaction Handling
Storage
![Page 26: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/26.jpg)
31
Standard PostgreSQL
Parser
Planner
Executor
MVCC and Transaction Handling
Storage
GTM
MVCC and Transaction Handling • Transaction ID (XID) Generation • Snapshot Handling
![Page 27: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/27.jpg)
32
Standard PostgreSQL
Parser
Planner
Executor
MVCC and Transaction Handling
Storage
XL Coordinator Parser
Planner
Executor
“Metadata” Storage
XL Datanode Executor
Storage
![Page 28: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/28.jpg)
33
Postgres-XL Architecture
• Based on the Postgres-XC project • Coordinators • Connection point for applications • Parsing and planning of queries
• Data Nodes • Actual data storage • Local execution of queries
• Global Transaction Manager (GTM) • Provides a consistent view to transactions • GTM Proxy to increase performance
![Page 29: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/29.jpg)
34
Coordinators
• Handles network connectivity to the client • Parse and plan statements
• Perform final query processing of intermediate result sets
• Manages two-phase commit
• Stores global catalog information
![Page 30: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/30.jpg)
35
Data Nodes
• Stores tables and indexes • Only coordinators connects to data nodes • Executes queries from the coordinators • Communicates with peer data nodes for
distributed joins
![Page 31: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/31.jpg)
36
Global Transaction Manager (GTM)
• Handles necessary MVCC tasks • Transaction IDs • Snapshots
• Manages cluster wide values • Timestamps • Sequences
• GTM Standby can be configured
![Page 32: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/32.jpg)
37
Global Transaction Manager (GTM)
• Can this be a single point of failure?
![Page 33: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/33.jpg)
38
Global Transaction Manager (GTM)
• Can this be a single point of failure?
Yes.
![Page 34: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/34.jpg)
39
Global Transaction Manager (GTM)
• Can this be a single point of failure?
Yes.
Solution: GTM Standby
GTM GTM Standby
![Page 35: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/35.jpg)
40
Global Transaction Manager (GTM)
• Can GTM be a bottleneck?
![Page 36: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/36.jpg)
41
Global Transaction Manager (GTM)
• Can GTM be a bottleneck?
Yes.
![Page 37: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/37.jpg)
42
Global Transaction Manager (GTM)
• Can GTM be a bottleneck?
Yes.
Solution: GTM Proxy
GTM Proxy Coord Proc 1
Coord Proc 2
Coord Proc 3
DN Proc 1
DN Proc 2
GTM
![Page 38: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/38.jpg)
43
GTM Proxy
• Runs alongside Coordinator or Datanode • Backends use it instead of GTM
• Groups requests together
• Obtain range of transaction ids (XIDs)
• Obtain snapshot
GTM Proxy Coord Proc 1
Coord Proc 2
Coord Proc 3
DN Proc 1
DN Proc 2
GTM
![Page 39: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/39.jpg)
44
GTM Proxy
• Example: 10 process request a transaction id
• They each connect to local GTM Proxy
• GTM Proxy sends one request to GTM for 10 XIDs
• GTM locks procarray, allocates 10 XIDs
• GTM returns range
• GTM Proxy demuxes to processes
GTM Proxy Coord Proc 1
Coord Proc 2
Coord Proc 3
DN Proc 1
DN Proc 2
GTM
![Page 40: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/40.jpg)
46
Data Distribution – Replicated Tables
• Good for read only and read mainly tables
• Sometimes good for dimension tables in data warehousing
• If coordinator and datanode are colocated, local read occurs
• Bad for write-heavy tables
• Each row is replicated to all data nodes
• Statement based replication
• “Primary” avoids deadlocks
![Page 41: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/41.jpg)
47
Data Distribution – Distributed Tables
• Good for write-heavy tables
• Good for fact tables in data warehousing
• Each row is stored on one data node
• Available strategies • Hash • Round Robin • Modulo
![Page 42: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/42.jpg)
49
Availability
• No Single Point of Failure
• Global Transaction Manager Standby
• Coordinators are load balanced
• Streaming replication for data nodes
• But, currently manual…
![Page 43: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/43.jpg)
50
DDL
![Page 44: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/44.jpg)
51
Defining Tables
CREATE TABLE my_table (…)
DISTRIBUTE BY
HASH(col) | MODULO(col) | ROUNDROBIN | REPLICATION
[ TO NODE (nodename[,nodename…])]
![Page 45: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/45.jpg)
52
Defining Tables
CREATE TABLE state_code (
state_code CHAR(2),
state_name VARCHAR,
:
) DISTRIBUTE BY REPLICATION
An exact replica on each node
![Page 46: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/46.jpg)
53
Defining Tables
CREATE TABLE invoice (
invoice_id SERIAL,
cust_id INT,
:
) DISTRIBUTE BY HASH(invoice_id)
Distributed evenly amongst sharded nodes
![Page 47: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/47.jpg)
54
Defining Tables
CREATE TABLE invoice (
invoice_id SERIAL,
cust_id INT ….
) DISTRIBUTE BY HASH(invoice_id);
CREATE TABLE lineitem (
lineitem_id SERIAL,
invoice_id INT …
) DISTRIBUTE BY HASH(invoice_id);
Joins on invoice_id can be pushed down to the datanodes
![Page 48: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/48.jpg)
55
test2=# create table t1 (col1 int, col2 int) distribute by hash(col1); test2=# create table t2 (col3 int, col4 int) distribute by hash(col3);
![Page 49: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/49.jpg)
56
explain select * from t1 inner join t2 on col1 = col3; Remote Subquery Scan on all (datanode_1,datanode_2) -> Hash Join Hash Cond: -> Seq Scan on t1 -> Hash -> Seq Scan on t2
Join push-down. Looks much like regular PostgreSQL
![Page 50: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/50.jpg)
57
test2=# explain select * from t1 inner join t2 on col2 = col3; Remote Subquery Scan on all (datanode_1,datanode_2) -> Hash Join Hash Cond: -> Remote Subquery Scan on all (datanode_1,datanode_2) Distribute results by H: col2 -> Seq Scan on t1 -> Hash -> Seq Scan on t2
Will read t1.col2 once and put in shared queue for consumption for other datanodes for joining. Datanode-datanode communication and parallelism
![Page 51: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/51.jpg)
60
By the way
n PostgreSQL does not have query parallelism – yet
![Page 52: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/52.jpg)
61
By the way
n PostgreSQL does not have query parallelism – yet n Example: 16 core box, 1 user, 1 query
– Just 1 core will be used – 15 (or so) idle cores
![Page 53: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/53.jpg)
62
By the way
n PostgreSQL does not have query parallelism – yet n Example: 16 core box, 1 user, 1 query
– Just 1 core will be used – 15 (or so) idle cores
n You can use Postgres-XL even on a single server and configure multiple datanodes to achieve better performance thanks to parallelism
![Page 54: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/54.jpg)
63
![Page 55: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/55.jpg)
64
Configuration
![Page 56: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/56.jpg)
65
![Page 57: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/57.jpg)
66
Use pgxc_ctl!
(demo in a few minutes)
(please get from git HEAD for latest bug fixes)
![Page 58: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/58.jpg)
67
Otherwise, PostgreSQL-like steps apply:
initgtm initdb
![Page 59: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/59.jpg)
68
postgresql.conf
• Very similar to regular PostgreSQL • But there are extra parameters
• max_pool_size • min_pool_size • pool_maintenance _timeout • remote_query_cost • network_byte_cost • sequence_range • pooler_port • gtm_host, gtm_port • shared_queues • shared_queue_size
![Page 60: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/60.jpg)
69
Install
n Download RPMs – http://sourceforge.net/projects/postgres-xl/
files/Releases/Version_9.2rc/
Or
n configure; make; make install
![Page 61: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/61.jpg)
70
Setup Environment
• .bashrc • ssh used, so ad installation bin dir to
PATH
![Page 62: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/62.jpg)
71
Avoid password with pgxc_ctl
• ssh-keygen –t rsa (in ~/.ssh) • For local test, no need to copy key,
already there • cat ~/.ssh/id_rsa.pub >> ~/.ssh/
authorized_keys
![Page 63: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/63.jpg)
72
pgxc_ctl
• Initialize cluster • Start/stop • Deploy to node • Add coordinator • Add data node / slave • Add GTM slave
![Page 64: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/64.jpg)
73
pgxc_ctl.conf
• Let’s create a local test cluster! • One GTM • One Coordinator • Two Datanodes
Make sure all are using different ports • including pooler_port
![Page 65: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/65.jpg)
74
pgxc_ctl.conf
pgxcOwner=pgxl pgxcUser=$pgxcOwner
pgxcInstallDir=/usr/postgres-xl-9.2
![Page 66: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/66.jpg)
75
pgxc_ctl.conf
gtmMasterDir=/data/gtm_master gtmMasterServer=localhost
gtmSlave=n
gtmProxy=n
![Page 67: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/67.jpg)
76
pgxc_ctl.conf
coordMasterDir=/data/coord_master coordNames=(coord1)
coordPorts=(5432)
poolerPorts=(20010)
coordMasterServers=(localhost)
coordMasterDirs=($coordMasterDir/coord1)
![Page 68: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/68.jpg)
77
pgxc_ctl.conf
coordMaxWALSenders=($coordMaxWALsender)
coordSlave=n
coordSpecificExtraConfig=(none)
coordSpecificExtraPgHba=(none)
![Page 69: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/69.jpg)
78
pgxc_ctl.conf
datanodeNames=(dn1 dn2) datanodeMasterDir=/data/dn_master
datanodePorts=(5433 5434)
datanodePoolerPorts=(20011 20012)
datanodeMasterServers=(localhost localhost)
datanodeMasterDirs=($datanodeMasterDir/dn1 $datanodeMasterDir/dn2)
![Page 70: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/70.jpg)
79
pgxc_ctl.conf
datanodeMaxWALSenders=($datanodeMaxWalSender $datanodeMaxWalSender)
datanodeSlave=n
primaryDatanode=dn1
![Page 71: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/71.jpg)
80
pgxc_ctl
pgxc_ctl init all
Now have a working cluster
![Page 72: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/72.jpg)
82
PostgreSQL Differences
![Page 73: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/73.jpg)
83
pg_catalog
• pgxc_node • Coordinator and Datanode definition • Currently not GTM
• pgxc_class • Table replication or distribution info
![Page 74: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/74.jpg)
84
Additional Commands
• PAUSE CLUSTER / UNPAUSE CLUSTER • Wait for current transactions to complete • Prevent new commands until UNPAUSE
• EXECUTE DIRECT ON (nodename) ‘command’
• CLEAN CONNECTION • Cleans connection pool
• CREATE NODE / ALTER NODE
![Page 75: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/75.jpg)
85
Noteworthy
• SELECT pgxc_pool_reload() • pgxc_clean for 2PC
![Page 76: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/76.jpg)
86
Looking at Postgres-XL Code
n XC #ifdef PGXC n XL #ifdef XCP #ifndef XCP
![Page 77: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/77.jpg)
87
Community
![Page 78: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/78.jpg)
88
Website and Code
n postgres-xl.org n sourceforge.net/projects/postgres-xl
n Will add mirror for github.com
n Docs – files.postgres-xl.org/documentation/index.html
![Page 79: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/79.jpg)
90
Philosophy
n Stability and bug fixes over features n Performance-focused
n Open and inclusive – Welcome input for roadmap, priorities and design – Welcome others to become core members and
contributors
![Page 80: Scaling PostgreSQL on Postgres-XL - PgConf.Russia · Postgres-XL ! Rich SQL support ... Postgres-XL Technology • GTM maintains a consistent view of the data . 30 Standard PostgreSQL](https://reader031.vdocuments.pub/reader031/viewer/2022021622/5ba1d70909d3f2766b8d3e42/html5/thumbnails/80.jpg)
91
Roadmap
n New release n Improve Cluster Management
n Catch up to PostgreSQL Community -> 9.3, 9.4
n Make XL a more robust analytical platform – Distributed Foreign Data Wrappers
n GTM optional
n Native High Availability