innodb mvcc architecture (by 권건우)

© Copyrights 2001~2017 EXEM CO.,LTD. All Rights Reserved.

InnoDB MVCC Architecture

엑셈 | 권건우 상무


02. MySQL/InnoDB MVCC Architecture

01. MySQL/InnoDB Undo Architecture

Table of Agenda

MySQL Deep Internal


MySQL Deep InternalMVCC Architecture 발전

1970

1978

1981

E.F. Codd

PHILIP A. BERNSTEIN AND NATHAN GOODMAN

- the data independence objective- the communicability objective - the set-processing objective

Our approach is quite different. We regard an object that is modifiable as asequence of immutable versions. each version is the state of the object afteran update is made to the object.

Multiversion Concurrency Control

1984 The first shipping, commercial database software product featuring MVCC

was Digital's VAX Rdb/ELN. The second was InterBase, both of which are still

active, commercial products.

Jim Starkey

1986 Oracle version 6 에서 Rollback Segment 도입

Bob Miner

1999 Mikheev Vadim B.PostgreSQL 6.5 에 MVCC 아키텍쳐를 도입함

David P. Reed

MySQL Deep Internal


MySQL Deep InternalMVCC Architecture의 두가지 접근법

• PostgreSQL, Firebird/Interbase, SQL Server 해당

• 데이터베이스에 다중 버전의 레코드를 저장

• 더 이상 필요하지 않을 때 모아둔 레코드를 버림

접근법 1

• Oracle, MySQL/InnoDB 해당

• 최신 버전의 데이터만 데이터베이스 내에 저장

• 언두를 이용하여 이전 버전의 데이터를 재구성

접근법 2


MySQL/InnoDB Undo Architecture

MySQL Deep Internal


MySQL Deep Internal

slot 0 4

slot 1 5

…

slot 1023

(1,5) Undo log

06d1 Rec001, A

T1.ibd space f9

Page 0

Page 1

Page 2

Page 3

Page 4 …

Page64

MySQL/InnoDB Undo Architecture

ibdata1 space 0

Page 0

Page 1

Page 2

Page 3

Page 4

Page 5

…

(0,5) TRX_SYS Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

undo001 space 1

Page 0

Page 1

Page 2

Page 3

Page 4

Page 5

… Page 8

ibtmp1

Rseg1

Rseg2

Rseg3

Rseg4

…Rseg

32

Page (f9,3)Rec001, A(TRX_ID 0x778c, Roll_PTR (5,06d1))

(1,3) Rollback Segment

System Tablespace

User Tablespace

Undo Tablespace

Temp Tablespace

MySQL Deep Internal


MySQL Deep Internal

slot 0 4

slot 1

…

slot 1023

(1,4) Undo log

0110 Rec001, A


T1.ibd space f9

Page 0

Page 1

Page 2

Page 3

Page 4 …

Page64

Page (f9,3)Rec001, A(TRX_ID 0x778b, Roll_PTR (4,0110))

MySQL/InnoDB Undo Architecture (select 시)

0x7f0208608000Data Page (f9,3)Rec001, A(TRX_ID 0x778b, Roll_PTR (4,0110))

Insert A 후, Select * from t1;

TRX_SYSmax_trx_id = 0x778crw_trx_list->count = 0mvcc->m_views->count = 1

Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

undo001 space 1

Page 0

Page 1

Page 2

Page 3

Page 4

Page 5

… Page 8

Buffer Pool

User Tablespace

System Tablespace

Undo Tablespace

MySQL Deep Internal


MySQL Deep Internal

slot 0 4

slot 1 5

…

slot 1023

(1,5) Undo log

06d1 Rec001, A


T1.ibd space f9

Page 0

Page 1

Page 2

Page 3

Page 4 …

Page64

Page (f9,3)Rec001, B(TRX_ID 0x778c, Roll_PTR (5,06d1))

MySQL/InnoDB Undo Architecture (update 시)

0x7f0208608000Data Page (f9,3)Rec001, B(TRX_ID 0x778c, Roll_PTR (5,06d1))

A를 B로 update (Update t1 Set c2 = B;)

TRX_SYSmax_trx_id = 0x778drw_trx_list->count = 0mvcc->m_views->count = 1

Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

undo001 space 1

Page 0

Page 1

Page 2

Page 3

Page 4

Page 5

…

0x7f0208608000Data Page (f9,3)Rec001, A(TRX_ID 0x778b, Roll_PTR (4,0110))

TRX_ID : 0x778caddr=0x1b74a750page_no=5offset=0x6d1

Buffer Pool

User Tablespace

System Tablespace

Undo Tablespace


MySQL/InnoDB MVCC Architecture

1) MVCC

2) Readview

MySQL Deep Internal


MySQL Deep Internal

slot 0 4

slot 1 5

…

slot 1023(1,5) Undo log


MySQL/InnoDB MVCC Architecture


Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

undo001 space 1

Page 0

Page 1

Page 2

Page 3

Page 4

Page 5

… Page 8

System Tablespace

Undo Tablespace

Readview Readview Readview Readview

TRX_STRUCTURE TRX_STRUCTURE TRX_STRUCTURE TRX_STRUCTURE

MySQL Deep Internal


MySQL Deep InternalMySQL/InnoDB MVCC Architecture

Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

System Tablespace

slot 0 4

slot 1

…

slot 1023

(1,4) Undo log(1,3) Rollback Segment

Undo Tablespace

Undo Page (1,4)Rec001, A(TRX_ID 0x778b)

1. 초기상태

Insert into t1 values(‘Rec001’,’A’);

Commit ;

Session 0

T1.ibd space f9

Page 0 …

Page 3

Page 4 …

Page64


Buffer Pool

User Tablespace


MySQL Deep Internal



Session 0

Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

System Tablespace

T1.ibd space f9

Page 0 …

Page 3

Page 4 …

Page64


Buffer Pool

User Tablespace

slot 0 4

slot 1

…

slot 1023


Undo Tablespace


Select * from t1;Rec001, A

Readview : 0x35dd108

Session 1

Readview : 0x35dd108m_low_limit_no = 0x778cm_low_limit_id = 0x778cm_up_limit_id = 0x778cm_ids->m_size = 0

0x7f0208608000Data Page (f9,3)Rec001, A(TRX_ID 0x778b,ROLL_PTR (4,110)

2. 세션1 Select

MySQL Deep Internal



Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

System Tablespace

T1.ibd space f9

Page 0 …

Page 3

Page 4 …

Page64

Page (f9,3)Rec001, B(TRX_ID 0x778c)

Buffer Pool

User Tablespace

slot 0 4

slot 1 5

…

slot 1023


Undo Tablespace



TRX ID 0x778caddr=0x1b74a750

0x7f02083b4000Undo Page (1,5)Rec001, A(TRX_ID 0x778b)

0x7f0208608000Data Page (f9,3)Rec001, B(TRX_ID 0x778c,ROLL_PTR (5,6d1))

Undo Page (1,5)(0x6d1)Rec001, A (TRX_ID 0x778b)

(1,5) Undo log

Update t1 Set c2 = B;

Session 0



Session 1

3. 세션0 Update

MySQL Deep Internal



Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

System Tablespace

T1.ibd space f9

Page 0 …

Page 3

Page 4 …

Page64

Page (f9,3)Rec001, B(TRX_ID 0x778c)

Buffer Pool

User Tablespace

slot 0 4

slot 1 5

…

slot 1023


Undo Tablespace

TRX_SYSmax_trx_id = 0x778erw_trx_list->count = 0mvcc->m_views->count = 2m_views->start = 0x35dd068

Readview : 0x35dd068m_low_limit_no = 0x778em_low_limit_id = 0x778em_up_limit_id = 0x778em_ids->m_size = 0

0x7f02083b4000Undo Page (1,5)Rec001, A(TRX_ID 0x778b)


Undo Page (1,5)(0x6d1)Rec001, A (TRX_ID 0x778b)

(1,5) Undo log

Session 0



Session 1

4. 세션2 Select

Select * from t1;Rec001, B


Session 2


MySQL Deep Internal



Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

System Tablespace

T1.ibd space f9

Page 0 …

Page 3

Page 4 …

Page64

Page (f9,3)Rec001, C(TRX_ID 0x778e)

Buffer Pool

User Tablespace

slot 0 4

slot 1 5

…

slot 1023


Undo Tablespace

TRX_SYSmax_trx_id = 0x778erw_trx_list->count = 0mvcc->m_views->count = 2m_views->start = 0x35dd068


(1,5) Undo log

Update t1 set c2 = C;

Session 0



Session 1

5. 세션0 Update



Session 2


TRX ID 0x778eaddr=0x1b74a750

0x7f02083b4000Undo Page (1,5)(0x6d1)Rec001, A(TRX_ID 0x778b)

0x7f02083b4000Undo Page (1,5)(0x7be)Rec001, B(TRX_ID 0x778cROLL_PTR (5,6d1))


Undo Page (1,5)0x6d1Rec001, A (TRX_ID 0x778b)

Undo Page (1,5)(0x7be)Rec001, B (TRX_ID 0x778cROLL_PTR (5,6d1))

MySQL Deep Internal



Rseg 0 (0,6)

Rseg 1

…

Rseg 33 (1,3)

Rseg 127

ibdata1 space 0

Page 0

Page 1

…Page

5…

(0,5) TRX_SYS

System Tablespace

T1.ibd space f9

Page 0 …

Page 3

Page 4 …

Page64

Page (f9,3)Rec001, C(TRX_ID 0x778e)

Buffer Pool

User Tablespace

slot 0 4

slot 1 5

…

slot 1023


Undo Tablespace

TRX_SYSmax_trx_id = 0x7790rw_trx_list->count = 0mvcc->m_views->count = 3m_views->start = 0x35ddfc8

Readview : 0x35dcfc8m_low_limit_no = 0x7790m_low_limit_id = 0x7790m_up_limit_id = 0x7790m_ids->m_size = 0

(1,5) Undo log

Session 0



Session 1

6. 세션3 Select



Session 2


0x7f02083b4000Undo Page (1,5)(0x6d1)Rec001, A(TRX_ID 0x778b)

0x7f02083b4000Undo Page (1,5)(0x7be)Rec001, B(TRX_ID 0x778cROLL_PTR (5,6d1))

0x7f0208608000Data Page (f9,3)Rec001, C(TRX_ID 0x778e,ROLL_PTR (5,7be))

Undo Page (1,5)0x6d1Rec001, A (TRX_ID 0x778b)

Undo Page (1,5)(0x7be)Rec001, B (TRX_ID 0x778cROLL_PTR (5,6d1))

Select * from t1;Rec001, C

Readview : 0x35ddfc8

Session 2


MySQL Deep Internal


MySQL Deep InternalReadview

정의 및 목적

• 트랜잭션이 발생하면, COMMIT 시점이 아닌 트랜잭션 시작 시점의 TRX_ID 가 저장됨

• 쿼리 시작시점의 액티브 트랜잭션들의 집합 (COMMIT/ROLLBACK 하지 않은 상태)

• MVCC 구현에 반드시 필요

Component Description

low_limit_no- 해당 값을 Purge thread가 참조하여 불필요한 Undo log들을 처리하는 purge 작업에 사용된다.- low_limit_no = Readview 생성시 trx_sys의 max_trx_id값

low_limit_id- low_limit_id보다 크거나 같은 TRX_ID (trx_id >= low_limit_id) 는 해당 readview에서 조회 안 됨- low_limit_id = Readview 생성시 trx_sys의 max_trx_id값

up_limit_id- up_limit_id(trx_id < up_limit_id) 보다 작은 트랜잭션

TRX_ID 는 해당 readview에서 모두 조회됨- up_limit_id = readview 생성시, active 트랜잭션 중 가장 작은 TRX_ID

trx_ids[] - 모든 액티브 트랜잭션 id로 구성된 그룹

view_list - Readview list (doubly linked list)

MySQL Deep Internal



New TRX Structure

mutex

max_trx_id = 0x98fd

rw_trx_listcount = 3start = 0x4fe4fae0

mvcc->m_viewscount = 3start = 0x25b9f08

…

trx_sys Trx linked list

trx id = 0x98fc

update_undo :page_no = 6offset = 0x86c

trx_listprev = 0x00next = 0x4fe4fe70

…

trx id = 0x98fb

update_undo :page_no = 7offset = 0x370

trx_listprev = 0x4fe4fae0next = 0x4fe4f750

…

trx id = 0x98f8

update_undo :page_no = 5offset = 0x2c4

trx_listprev = 0x4fe4fe70next = 0x00

…

trx0 0x4fe4fae0 trx1 0x4fe4fe70 trx2 0x4fe4f750

New Readview

Readview linked list

m_low_limit_no = 0x98fd

m_low_limit_id = 0x98fd

m_up_limit_id = 0x98f8

m_view_listprev = 0x0next = 0x25b9fa8

m_ids->m_size = 3

m_ids[0] = 0x98f8m_ids[1] = 0x98fbm_ids[2] = 0x98fc

readview0 0x25b9f08

m_low_limit_no = 0x98fc

m_low_limit_id = 0x98fc


m_view_listprev = 0x25b9f08next = 0x25ba048

m_ids->m_size = 2

m_ids[0] = 0x98f8m_ids[1] = 0x98fb

readview1 0x25b9fa8

m_low_limit_no = 0x98fb

m_low_limit_id = 0x98fb


m_view_listprev = 0x25b9fa8next = 0x0

m_ids->m_size = 1

m_ids[0] = 0x98f8

readview2 0x25ba048

MySQL Deep Internal



1. t1 Table 생성

create table t1 (id int primary key, comment

char(50), filler char(2)) engine = innodb;

2. Secondary index 생성

create index test_idx on test(comment); T1

.ibd

sp

ace

17

3

Page0

Page1

Page2

Page3

Page4

Page5

…

UN

DO

00

1 s

pac

e 1

Page0

Page1

Page2

Page3

Page4

Page5

…

Secondary Index

source : InnoDBTransaction/Lock/Multi-version analysis, 网易杭研-何登成

Clustering Index

Page 4

MySQL Deep Internal



3. TRX_ID가 0x96f6 일때, t1에 1건 Insert

insert into t.test values (65, 'aaa',' ') ;

T1

.ibd

sp

ace

17

3

Page0

Page1

Page2

Page3

Page4

Page5

…

Deleted: 0

65

DB_TRX_ID (0x96f6)

DB_ROLL_PTR (4,110)

aaa

Deleted: 0

aaa

PK: 65

UN

DO

00

1 s

pac

e 1

Page0

Page1

Page2

Page3

Page4

Page5

…

Insert Undo

aaa

PK: 65


Secondary Index

Clustering Index

Page 4

MySQL Deep Internal



4. Primary key 업데이트

update t.test set id = 67 where id = 65 ;

T1

.ibd

sp

ace

17

3

Page0

Page1

Page2

Page3

Page4

Page5

…

Deleted: 1

65

DB_TRX_ID (0x96fc)

DB_ROLL_PTR (7,110)

aaa

Deleted: 1

aaa

PK: 65

UN

DO

00

1 s

pac

e 1

Page0

Page1

Page2

Page3

Page4

…

Page7

Update Undo

aaa

PK: 65


• 새로운 레코드 (67, aaa)가 추가됨

(Insert 발생하는 것처럼 Insert Undo에 저장)

• 이전 레코드 (65, aaa)는 삭제 표시됨 (삭제 flag, trx_id, roll_ptr를

설정하고, before-image를 Update Undo에 저장)

• Secondary Index 페이지에서도 레코드 (aaa, 65) 는 삭제 설정하고

레코드 (aaa,67)가 추가됨

Deleted: 0

67

DB_TRX_ID (0x96fc)

DB_ROLL_PTR (4,11d)

aaa

Deleted: 0

aaa

PK: 67

Insert Undo

aaa

PK: 67

Secondary Index

Clustering Index

Page 4 Page 7

MySQL Deep Internal



5. Secondary key 업데이트

update t.test set comment = 'ccc' where id = 67 ;

T1

.ibd

sp

ace

17

3

Page0

Page1

Page2

Page3

Page4

Page5

…

Deleted: 1

65

DB_TRX_ID (0x96fc)

DB_ROLL_PTR (7,110)

aaa

Deleted: 1

aaa

PK: 65

UN

DO

00

1 s

pac

e 1

Page0

Page1

Page2

Page3

Page4

…

Page7

Update Undo

aaa

PK: 65


• 레코드 (67,aaa)를 레코드 (67,ccc)로 변경하고, before-image를

Update Undo에 저장

• Secondary Index 페이지에서도 레코드 (aaa,67)는 삭제 설정되고,

레코드 (ccc,67)가 추가됨.

Deleted: 0

67

DB_TRX_ID (0x9700)

DB_ROLL_PTR (7,224)

ccc

Deleted: 1

aaa

PK: 67

Update Undo

aaa

PK: 67

Deleted: 0

ccc

PK: 67

Secondary Index

Clustering Index

Page 7

MySQL Deep Internal



Session 0 (DML)

Session 1 (DML)

Session 2 (DML)

Session 3 (Read)

Session 4 (Read)

Session 5 (Read)

UPDATE(67, ‘ccc’)

INSERT (65, ‘aaa’)

UPDATE(67, ‘aaa’)

0x96f5

0x96f6

0x96fb 0x96fc 0x97000x96ff 0x97060x96fd 0x97090x9705

READVIEW 0 0x25c0ee8UP:0x96f5 ~ LOW:0x96fdTrx:0x96f5, 0x96fb, 0x96fcSelect * from test (65,aaa)

READVIEW 1 0x25c0e48UP:0x96fb ~ LOW:0x9705Trx:0x96fb, 0x96ff, 0x9700Select * from test (67,aaa) READVIEW 2 0x25c0da8

UP:0x9706~ LOW:0x9709Trx:0x9706Select * from test (67,ccc)

TX 0x96f5

TX 0x96fb

TX 0x96fc

TX 0x9700

TX 0x96ff

TX 0x9706

MySQL Deep Internal


MySQL Deep Internal


Readview

select * from t1;

case up_limit_id low_limit_id trx_ids [ ] query result

Readview0 Primary Key 변경 0x96f5 0x96fd 0x96f5, 0x96fb, 0x96fc (65, aaa)

Readview1 Secondary Key 변경 0x96fb 0x9705 0x96fb, 0x96ff, 0x9700 (67, aaa)

Readview2해당 테이블에

트랜잭션이 없을 경우0x9706 0x9709 0x9706 (67, ccc)


NAVER http://cafe.naver.com/playexem

ITPUB (中) http://blog.itpub.net/31135309/

Wordpress https://playexem.wordpress.com/

Slideshare http://www.slideshare.net/playexem

교육 문의 [email protected]

EXEM Research & Contents Team

Youtube https://www.youtube.com/channel/UC5wKR_-A0eL_Pn_EMzoauJg

Tudou (中) http://www.tudou.com/home/maxgauge/

http://cafe.naver.com/playexem

http://blog.itpub.net/31135309/

https://playexem.wordpress.com/

http://www.slideshare.net/playexem

mailto:[email protected]

https://www.youtube.com/channel/UC5wKR_-A0eL_Pn_EMzoauJg

http://www.tudou.com/home/maxgauge/


감사합니다

innodb mvcc architecture (by 권건우)

Technology