in-memory database system and low latency
Post on 10-May-2015
1.789 Views
Preview:
DESCRIPTION
TRANSCRIPT
IN-MEMORYDATABASE SYSTEMAND LOW LATENCYFB lee.hyeongchae
About me국산 DBMSs 이너비트 NHN ( CUBRID ) 텔코웨어 알티베이스 티베로 리얼타임테크 아키스
국산 DBs 선재소프트 유엔젤
MobileLiteEmbedded In-Memory DBMS
CUBRIDObject Oriented DBMS
TelcobaseIn-Memroy DBMS
AltibaseIn-Memroy DBMS
Agenda
In-Memory DBMS !!Altibase, TimesTen, ExtremeDB, KDB+, C-ISAM …
Why ?!Ultra & Extreme low latency AND Exture+
How ?!NUMA, SSD, Infiniband, Compiler, MQ, Tick, CEP …
Best ?!In-Memory Computing ?!
Best Of Best !!High Performance Computing !!
IN-MEMORYDATABASE SYSTEM
In-Memory Architecture
In-Memory Database System
High Performance
Low Latency No Jitter
Disk-Based RDBMS vs Oracle TimesTen
( Simple Architecture )
Modern DBMS
≒In-Memory
DBMS
In-Memory Database System
1G
2G3G 4G
Altibase HDB & XDB
장점 국내개발 및 기술지원 증권사 레퍼런스 MVCC 지원
단점 성능한계 & 메모리 이슈 고급 엔지니어 부재
장점 오라클 & 레퍼런스 성능 DA ( Direct Attach )
단점 DA ( Direct Attach ) 한계 MVCC 미지원 Durability 이슈 엔지니어 및 기술지원 부제
Oracle TimesTen & Co-herence
McObject ExtremeDB & FE장점 성능 해외 통신 & 증권 레퍼런스 STAC Member Join
단점 임베디드 전문 회사 국내 기술지원 FE ( Financial Edition )
검증
KX Systems KDB+
장점 WORLD BEST
QL ( Q Language )
STAC-M3 Leader
단점 가격 국내 기술지원
IBM Informix C-ISAM
장점 OLD BEST
성능 & 가격 Zero Configuration &
Admin
단점 File DB
ACID 미지원 유지보수 및 기술지원
WHY ?!LOW LATENCY
LowLatency
UltraLow La-tencyExtreme
Low La-tency
second
㎳㎳ ~ ㎲
㎲
2 digit ㎲ ~㎱
Ultra & Extreme Low Lantecy
Exture+ RFP
Exture+ RFP
Exture+ RFP
Exture vs Exture+
Sync vs Async
Pub-Sub Architecture
Exture+ Dev ?!
Exture+ Dev ?!
Exture+ Dev ?!
Exture+ Dev ?!
HOW ?!TO-BE ARCHITECTURE
NUMA Architecture
NUMA Architecture
Using McObject’s 64-bit eXtremeDB-64, the application creates a 1.17 Terabyte, 15.54 billion row database on a 160-core Linux based SGI® Altix® 4700 server.
NUMA Architecture
It supports up to 512 * sockets or 1024 cores under one instance ofLinux and as much as 128TB of globally addressable memory.
NUMA Architecture
CPU-Socket-Isolation via PhysicalNIC / PhysicalCPU pairing.Multiple CPU sockets holding multiple CPUs should be used like multiple machines. Avoid inter-CPU communication
NUMA Architecture
8-socket Nehalem-EX: memory bandwidth matrix
8-socket Nehalem-EX: architecture
InfiniBand & 10GbE
In computer networking, Server Message Block (SMB), also known as Common Internet File System (CIFS, /ˈsɪfs/) operates as an application-layer network protocol[1] mainly used for providing shared access to files, printers, serial ports, and miscellaneous communications between nodes on a network.
InfiniBand & 10GbE
Isolation
!!
Bypass !
!
SSD
SSD is very very diffi-cult.
SSD (DRAM/NVRAM/flash/SLC/MLC) is Blah Blah !!
Compiler & *.[so|a] Li-brary
Message QueueZeroMQ Crazy fast
Brokerless architecture
In-process library
Lower latencies
Very simple to use
No persistence – requiring higher layers to manage persistence
RabbitMQ VMware vFabric
AMQP compliant
Written in erlang
Small footprint and seemingly fewer lines of code in comparison to other AMQP compliant queue managers
OpenMAMA is high perfor-mance Middleware Agnostic Messaging API
Tick & Time series data-baseeXtremeDB Financial Edition meets the specialized requirements of handling market data with several powerful features:
Flexible data layout. eXtremeDB Financial Edition implements columnar data layout for fields of type ‘sequence’. Sequences can be combined to form a time series, ideal for working with tick streams, historical quotes and other sequential data. The technology supports database designs that combine row-based and column-based layouts, to best leverage L1/L2 cache speed.
Traditional DBMSs bring rows of data into L1/L2 cache for processing. But financial data – such as trades and quotes – are better handled by a column-based layout that avoids flooding the cache with unwanted data.
Vector-based statistical function library. Vector-based statistical functions provide high efficiency by executing over all or part of one or more sequences and supporting assembly lines of operations on sequences, for statistical/quantitative analysis
eXtremeDB Financial Edition provides a rich library of vector-basedstatistical functions that execute over sequence to accelerate management of time series data.
Handles real-time and historical data. eXtremeDB Financial Edition’s in-memory storage is ideal for real-time data, while developers can easily specify persistent tables for historical data with a simple notation in the database schema. eXtremeDB programming skills are fully interchangeable between in-memory and persistent database designs (developers needn’t learn two database system products for real-time and historical data).
Complex Event Process-ing
우리는 대용량 고속 데이터에 대한 통찰력을 가지기 위한 복합 이벤트 처리 기술(CEP), 인 - 메모리 기반 분석 기술 , 대용량 데이터 베이스 및 저장 기술 등 고급 기술 세트를 제공합니다 . 복합 이벤트 분석 (Complex Event Analytics) 솔루션은 실시간 대용량 정보 소스 (Source) 로부터 통찰과 함께 즉시 의사 결정을 지원하는 혁신적인 솔루션입니다 .
BESTIN-MEMORY COMPUTING
In-Memory Computing
In-Memory Comput-ing
!=No-Disk Comput-
ing
BEST OF BESTHIGH PERFORMANCE COMPUTING
TOP500 - 1 st
Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in the PFLOPS (petaFLOPS) range, with low power consumption.
High Performance Com-puting
HPI ( Hasso Plattner Institut )
Q?!AFB lee.hyeongchaehyeongchae@G+
top related