IN-MEMORYDATABASE SYSTEMAND LOW LATENCYFB lee.hyeongchae

About me국산 DBMSs 이너비트 NHN ( CUBRID ) 텔코웨어 알티베이스 티베로 리얼타임테크 아키스

국산 DBs 선재소프트 유엔젤

MobileLiteEmbedded In-Memory DBMS

CUBRIDObject Oriented DBMS

TelcobaseIn-Memroy DBMS

AltibaseIn-Memroy DBMS

Agenda

In-Memory DBMS !!Altibase, TimesTen, ExtremeDB, KDB+, C-ISAM …

Why ?!Ultra & Extreme low latency AND Exture+

How ?!NUMA, SSD, Infiniband, Compiler, MQ, Tick, CEP …

Best ?!In-Memory Computing ?!

Best Of Best !!High Performance Computing !!

IN-MEMORYDATABASE SYSTEM

In-Memory Architecture

In-Memory Database System

High Performance

Low Latency No Jitter

Disk-Based RDBMS vs Oracle TimesTen

( Simple Architecture )

Modern DBMS

≒In-Memory

DBMS

In-Memory Database System

1G

2G3G 4G

Altibase HDB & XDB

장점 국내개발 및 기술지원 증권사 레퍼런스 MVCC 지원

단점 성능한계 & 메모리 이슈 고급 엔지니어 부재

장점 오라클 & 레퍼런스 성능 DA ( Direct Attach )

단점 DA ( Direct Attach ) 한계 MVCC 미지원 Durability 이슈 엔지니어 및 기술지원 부제

Oracle TimesTen & Co-herence

McObject ExtremeDB & FE장점 성능 해외 통신 & 증권 레퍼런스 STAC Member Join

단점 임베디드 전문 회사 국내 기술지원 FE ( Financial Edition )

검증

KX Systems KDB+

장점 WORLD BEST

QL ( Q Language )

STAC-M3 Leader

단점 가격 국내 기술지원

IBM Informix C-ISAM

장점 OLD BEST

성능 & 가격 Zero Configuration &

Admin

단점 File DB

ACID 미지원 유지보수 및 기술지원

WHY ?!LOW LATENCY

LowLatency

UltraLow La-tencyExtreme

Low La-tency

second

㎳㎳ ~ ㎲

㎲

2 digit ㎲ ~㎱

Ultra & Extreme Low Lantecy

Exture+ RFP

Exture+ RFP

Exture+ RFP

Exture vs Exture+

Sync vs Async

Pub-Sub Architecture

Exture+ Dev ?!

Exture+ Dev ?!

Exture+ Dev ?!

Exture+ Dev ?!

HOW ?!TO-BE ARCHITECTURE

NUMA Architecture

NUMA Architecture

Using McObject’s 64-bit eXtremeDB-64, the application creates a 1.17 Terabyte, 15.54 billion row database on a 160-core Linux based SGI® Altix® 4700 server.

NUMA Architecture

It supports up to 512 * sockets or 1024 cores under one instance ofLinux and as much as 128TB of globally addressable memory.

NUMA Architecture

CPU-Socket-Isolation via PhysicalNIC / PhysicalCPU pairing.Multiple CPU sockets holding multiple CPUs should be used like multiple machines. Avoid inter-CPU communication

NUMA Architecture

8-socket Nehalem-EX: memory bandwidth matrix

8-socket Nehalem-EX: architecture

InfiniBand & 10GbE

In computer networking, Server Message Block (SMB), also known as Common Internet File System (CIFS, /ˈsɪfs/) operates as an application-layer network protocol[1] mainly used for providing shared access to files, printers, serial ports, and miscellaneous communications between nodes on a network.

InfiniBand & 10GbE

Isolation

!!

Bypass !

!

SSD

SSD is very very diffi-cult.

SSD (DRAM/NVRAM/flash/SLC/MLC) is Blah Blah !!

Compiler & *.[so|a] Li-brary

Message QueueZeroMQ Crazy fast

Brokerless architecture

In-process library

Lower latencies

Very simple to use

No persistence – requiring higher layers to manage persistence

RabbitMQ VMware vFabric

AMQP compliant

Written in erlang

Small footprint and seemingly fewer lines of code in comparison to other AMQP compliant queue managers

OpenMAMA is high perfor-mance Middleware Agnostic Messaging API

Tick & Time series data-baseeXtremeDB Financial Edition meets the specialized requirements of handling market data with several powerful features:

Flexible data layout. eXtremeDB Financial Edition implements columnar data layout for fields of type ‘sequence’.  Sequences can be combined to form a time series, ideal for working with tick streams, historical quotes and other sequential data. The technology supports database designs that combine row-based and column-based layouts, to best leverage L1/L2 cache speed.

Traditional DBMSs bring rows of data into L1/L2 cache for processing. But financial data – such as trades and quotes – are better handled by a column-based layout that avoids flooding the cache with unwanted data.

Vector-based statistical function library. Vector-based statistical functions provide high efficiency by executing over all or part of one or more sequences and supporting assembly lines of operations on sequences, for statistical/quantitative analysis

eXtremeDB Financial Edition provides a rich library of vector-basedstatistical functions that execute over sequence to accelerate management of time series data.

 Handles real-time and historical data. eXtremeDB Financial Edition’s in-memory storage is ideal for real-time data, while developers can easily specify persistent tables for historical data with a simple notation in the database schema.  eXtremeDB programming skills are fully interchangeable between in-memory and persistent database designs (developers needn’t learn two database system products for real-time and historical data).

Complex Event Process-ing

우리는 대용량 고속 데이터에 대한 통찰력을 가지기 위한 복합 이벤트 처리 기술(CEP), 인 - 메모리 기반 분석 기술 , 대용량 데이터 베이스 및 저장 기술 등 고급 기술 세트를 제공합니다 . 복합 이벤트 분석 (Complex Event Analytics) 솔루션은 실시간 대용량 정보 소스 (Source) 로부터 통찰과 함께 즉시 의사 결정을 지원하는 혁신적인 솔루션입니다 .

BESTIN-MEMORY COMPUTING

In-Memory Computing

In-Memory Comput-ing

!=No-Disk Comput-

ing

BEST OF BESTHIGH PERFORMANCE COMPUTING

TOP500 - 1 st

Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in the PFLOPS (petaFLOPS) range, with low power consumption.

High Performance Com-puting

HPI ( Hasso Plattner Institut )

Q?!AFB lee.hyeongchaehyeongchae@G+