postgresql 9.6 새 기능 소개

PostgreSQL 9.6 새 기능이지호([email protected])

강연자 소개

u 이름: 이지호, 호: 서치(슬기 서, 기다릴 치), 영문 중간 이름: persy

u 소속: 한국방송통신대학교 대학원 정보과학과, (주)위비즈넷, 사단법인 E.S.C

u 98년 알짜레드햇 6.0에서 PostgreSQL 6.4를 보고 아!!! 이거다!!

PostgreSQL은 어떤 데이터베이스에요?

u PostgreSQL은 1986년 캘리포니아 버클리에서 시작해 1988년 만들어진 BSD 기반의 오픈소스 데이터베이스

u 이후 여러번의 지속적인 개선을 통해 1996년 PostgreSQL 발표.

u PostgreSQL이 제공하는 특징

u 연산자, 복합 자료형, 집계 함수, 자료형 변환자, 확장 함수 등 제공

u 테이블 생성 및 조회시 상속 기능 사용 가능

u 함수(프로시저) 사용 가능(pl/pgsql이 공식이나 python, java, php, perl, tcl도 가능)

u 이름은 어떻게 부르나요? -> 여러분이 부르고 싶은대로 부르세요.

PostgreSQL 커뮤니티

u 한국 PostgreSQL 사용자 모임은

u http://www.facebook.com/groups/postgres.kr

u 2016년 10월 10일 기준 1,030명에 이르는 회원수를 자랑합니다.

u 모임은?

u 매달 넷째주 일요일에 애 안봐도 되고, 애인 안 만나도 되고, 다른 할 일도 없는 분들이모여 정기 기술 세미나를 오후 2시 ~ 오후 5시까지 하고,

u 주말의 필수품목인 ‘알콜’ Drinking을 하지 않는 정말 건전한 모임을 합니다.

Parallel execution of sequential scans, joins and aggregates


u max_parallel_workers_per_gather (integer) - 병렬 쿼리를 실행할 작업자 수. 기본값 0

u max_worker_processes (integer) - 백그라운드 프로세스의 최대 수. 기본값: 시스템 코어 수

u parallel_setup_cost (floating point) - Sets the planner's estimate of the cost of launching parallel worker processes. The default is 1000.

u parallel_tuple_cost (floating point) - Sets the planner's estimate of the cost of transferring one tuple from a parallel worker process to another process. The default is 0.1.

u min_parallel_relation_size (integer) - Sets the minimum size of relations to be considered for parallel scan. The default is 8 megabytes (8MB).


=# EXPLAIN SELECT count(*) FROM partest;QUERY PLAN

-----------------------------------------------------------------------Aggregate (cost=37059.38..37059.39 rows=1 width=8)

-> Seq Scan on partest (cost=0.00..31417.50 rows=2256750 width=0)


template1=# set max_parallel_workers_per_gather = 1;template1=# EXPLAIN SELECT count(*) FROM partest;

QUERY PLAN -----------------------------------------------------------------------Finalize Aggregate (cost=24556.00..24556.01 rows=1 width=8)

-> Gather (cost=24555.88..24555.99 rows=1 width=8) Workers Planned: 1-> Partial Aggregate (cost=23555.88..23555.89 rows=1 width=8)

-> Parallel Seq Scan on partest (cost=0.00..20614.71 rows=1176471 width=0)


template1=# set max_parallel_workers_per_gather = 3;template1=# EXPLAIN SELECT count(*) FROM partest;

QUERY PLAN -----------------------------------------------------------------------Finalize Aggregate (cost=20266.88..20266.89 rows=1 width=8)

-> Gather (cost=20266.67..20266.88 rows=1 width=8) Workers Planned: 2-> Partial Aggregate (cost=19266.67..19266.68 rows=1 width=8)

-> Parallel Seq Scan on partest (cost=0.00..17183.33 rows=833333 width=0)


template1=# set max_parallel_workers_per_gather = 5;template1=# explain analyze select min(col1) from partest;

QUERY PLAN -----------------------------------------------------------------------Finalize Aggregate (cost=127493.90..127493.91 rows=1 width=8) …

-> Gather (cost=127493.37..127493.88 rows=5 width=8) …Workers Planned: 5Workers Launched: 5-> Partial Aggregate (cost=126493.37..126493.38 rows=1 width=8) …

-> Parallel Seq Scan on partest (cost=0.00..121493.30 rows=…) …

Planning time: 0.110 msExecution time: 1422.139 ms


template1=# set max_parallel_workers_per_gather = 6;template1=# explain analyze select min(col1) from partest;

QUERY PLAN -----------------------------------------------------------------------Finalize Aggregate (cost=127493.90..127493.91 rows=1 width=8) …

-> Gather (cost=127493.37..127493.88 rows=5 width=8) …Workers Planned: 6Workers Launched: 6-> Partial Aggregate (cost=126493.37..126493.38 rows=1 width=8) …

-> Parallel Seq Scan on partest (cost=0.00..121493.30 rows=…) …

Planning time: 0.110 msExecution time: 1422.139 ms

Autovacuum no longer performs repetitive scanning of old data

u PostgreSQL 9.6 부터 성능 향상을 위해 영구보관용 데이터에 대해서auto vacuum 중지

u PostgreSQL 9.6 이전까지 모든 힙 페이지 스캔

u 9.6은 마지막 동결 이후 수정된 페이지만 스캔

u 이 결정 덕분에 쓰기가 드문 테이블 성능 향상 혜택

Synchronous replication now allows multiple standby servers for increased reliability

How replication works


synchronous_commit = off


synchronous_commit = local


synchronous_commit = on (default)


synchronous_commit = remote_write


synchronous_commit = remote_apply


u synchronous_standby_names = ‘standby_name [, …]’u synchronous_standby_names = ‘num (standby_name [, …])’

u ex) synchronous_standby_names = ‘2(*)’

u synchronous_standby_names is emptyu If this parameter is empty as shown it changes behaviour of

setting synchronous_commit to on, remote_write or remote_apply to behave same as local (ie, the COMMIT will only wait for flushing to local disk).

postgres_fdw now supports remote joins, sorts, UPDATEs, and DELETEs

FDW

Foreign Server

Foreign Server

Foreign Server

SQL 쿼리

Join, Sort, Updates, Deletes

Allow Limiting of Snapshot Age

old_snapshot_threshold 값은 서버를 시작할때만 설정 가능(DEFAULT: -1, 0, 1min - 60d)

Session 1 Session 2show old_snapshot_threshold;;create table snaptest(x int);;insert into snaptest values (1);;

begin work;;set transaction isolation level serializable;;select * from snaptest;;

update snaptest set x = 2;;select pg_sleep(100);;vacuum snaptest;;

select * from snaptest;;ERROR:snapshot too old;;

Allow Sessions with Long-Idle Transactions To Be Terminated

u 오래 기다리고 있는 세션에 대해서 연결을 기다리지 않고 세션 자동 종료. default: 0

=# SET idle_in_transaction_session_timeout = ’2s’;

=# BEGIN WORK;

-- sit idle for 3 seconds

=# SELECT 1;

FATAL: terminating connection due to idle-in-transaction timeout

server closed the connection unexpectedly

CREATE EXTENSION CASCADE Support

u PostgreSQL 9.6 이전에 EXTENSION 설치 시 종속성 부터 설치

=# create extension cube;

=# create extension earthdistance;

u PostgreSQL 9.6 부터는 EXTENSION의 control 파일에 종속성 정의

=# create extension earthdistance cascade;

NOTICE: installing required extension "cube"

CREATE EXTENSION

CREATE EXTENSION CASCADE Support

# blackhole extension

comment = 'Minimal Extension Template’

default_version = '1.0’

module_pathname = '$libdir/blackhole’

relocatable = true

requires = ’cube, jsonlog’

=# create extension blackhole cascade

SUPPORT \CROSSTABVIEW IN PSQL

u PostgreSQL 9.6 이전엔 crosstab 쿼리는 tablefunc 모듈 사용.

=# create table test2 as

select

('a,b,c,d,e,f'::text[])[1 + floor(random() * 6)] as type_1,

('m,n,o,p,q,r'::text[])[1 + floor(random() * 6)] as type_2,

random() * 1000 as some_float

from generate_series(1,20000);

SUPPORT \CROSSTABVIEW IN PSQL

u =# select type_1, type_2, count(*)

from test2 group by type_1, type_2 \crosstabview

type_1 | q | n | o | r | m | p--------+-----+-----+-----+-----+-----+-----f | 519 | 569 | 515 | 593 | 519 | 560e | 605 | 539 | 547 | 555 | 587 | 577c | 574 | 590 | 534 | 549 | 541 | 556d | 569 | 487 | 569 | 521 | 523 | 553b | 543 | 594 | 547 | 573 | 548 | 590a | 586 | 576 | 568 | 555 | 559 | 510

(6 rows)

type_1 | type_2 | some_float-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐1 | b | m |409.599955659362 | e | n |453.8348475471143 | f | o |113.6342091485864 | b | n |332.84047478815 | d | n |832.6567551121126 | b | o |525.8609829470527 | b | r |873.10529500246

Generic WAL facility

u PostgreSQL 9.6은외부개발자가 CUSTOM EXTENSION 개발을쉽게할수있도록일반화된 WAL 인터페이스제공

u GenericXLogStartu GenericXLogRegisterBufferu GenericXLogFinishu GenericXLogAbort

Allow COPY to copy the output of an INSERT/UPDATE/DELETE ... RETURNING query

=# COPY (INSERT INTO tab_data VALUES (1, 'aa'), (2, 'bb') RETURNING a, b) TO stdout;

1 aa

2 bb

=# COPY (UPDATE tab_data SET b = 'cc' WHERE a = 1 RETURNING *) TO stdout;

1 cc

=# COPY (DELETE FROM tab_data WHERE a = 2 RETURNING *) TO stdout;

2 bb

Prompt variable to show PId of backend

u psql 프롬프트에 Postmaster PID 표시

u 주의) pg_bouncer, pg_pool 등 사용시 PID 예측 불가

=# \set PROMPT1 '%n@%/ : %p $ '

depesz@depesz : 26853 $

u prompt에 사용 가능한 모든 옵션:

u https://www.postgresql.org/docs/current/static/app-psql.html#APP-PSQL-PROMPTING

Monitoring Features

u Add System Catalog

u Vacuum progress reporting

u pg_config System View

u System view to monitor WAL receiver status

u Modified System Catalog

u pg_stat_activity wait-type reporting

u Add System Function

u pg_control Values Exposed

u Notification queue monitoring

Monitoring Features -pg_stat_activity wait-type reporting

u PostgreSQL 9.6에서 pg_stat_activity catalog 컬럼 추가

u wait_event_type - The type of event for which the backend is waiting

u wait_event - Wait event name if backend is currently waiting

u The value of wait_event_type column

• LWLockNamed - Waiting by a particular lightweight lock

• LWLockTranche - Waiting by the lightweight lock for the group

• Lock - Waiting by weight lock (LOCK TABLE statement etc)

• BufferPin - PIN waiting for the buffer

u wait_event

• https://www.postgresql.org/docs/9.6/static/monitoring-stats.html

Monitoring Features -Vacuum progress reportingu This catalog provides information about the state of the progress of vacuum processing.

This catalog is readable by the general superuser.Column Name Data Type Description

pid integer Process ID of backend

datid oid OID of database

datname name Connected database name

relid oid OID of the table being vacuumed

phase text Current processing phase of vacuum

heap_blks_total bigint Total number of heap blocks in the table

heap_blks_scanned bigint Number of heap blocks scanned

heap_blks_vacuumed bigint Number of heap blocks vacuumed

index_vacuum_count bigint Number of completed index vacuum cycles

max_dead_tuples bigint Number of dead tuples that we can store before needing to perform an index vacuum cycle

num_dead_tuples bigint Number of dead tuples collected since the last index vacuum cycle

Monitoring Features -pg_config System View

u pg_config를 시스템 유저 권한으로 SQL로 조회

u Column: name, setting

postgres=# SELECT * FROM pg_config ; name | setting-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐BINDIR | /usr/local/pgsql/binDOCDIR | /usr/local/pgsql/share/docHTMLDIR | /usr/local/pgsql/share/docINCLUDEDIR | /usr/local/pgsql/includePKGINCLUDEDIR | /usr/local/pgsql/include

Monitoring Features -pg_control Values Exposed

Name Return Type Description

pg_control_checkpoint() record Returns information about current checkpoint state.

pg_control_system() record Returns information about current control file state.

pg_control_init() record Returns information about cluster initialization state.

pg_control_recovery() record Returns information about recovery state.

They also show information about write-ahead logging and checkpoint processing. This information is cluster-wide, and not specific to any one database

Monitoring Features -System view to monitor WAL receiver status This catalog provides information about the state of a slave instance's WAL receiver process in replication environment. This catalog is readable by the general user.

pid integer Process ID of the WAL receiver processstatus text Activity status of the WAL receiver process

receive_start_lsn pg_lsn First transaction log position used when WAL receiver is started

receive_start_tli integer First timeline number used when WAL receiver is started

received_lsn pg_lsn Last transaction log position already received and flushed to disk, the initial value

received_tli integer Timeline number of last transaction log position received and flushed to disk, the initial value

last_msg_send_time timestamp with time zone Send time of last message received from origin WAL sender

Monitoring Features -System view to monitor WAL receiver status (…) This catalog provides information about the state of a slave instance's WAL receiver process in replication environment. This catalog is readable by the general user.

pid integer Process ID of the WAL receiver process

last_msg_receipt_time timestamp with time zone Receipt time of last message received from origin WAL sender

latest_end_lsn pg_lsn Last transaction log position reported to origin WAL sender

latest_end_time timestamp with time zone Time of last transaction log position reported to origin WAL sender

slot_name text Replication slot name used by this WAL receiver

conninfo text Connection string used by this WAL receiver, with security-sensitive fields obfuscated.

Monitoring Features -Notification queue monitoring

u PostgreSQL 큐는 8GB까지의 제한 존재

u DBA 관리자가 큐에 남은 작업이 있는지 확인할 필요

=# select pg_notification_queue_usage();

u 이 함수가 반환하는 실수 양에 따라 남아있는 작업량 판단 가능

감사합니다.Q&A

postgresql 9.6 새 기능 소개

Technology