con7248 top 5 things to know about oracle in-memory database

Oracle Database In-‐Memory Top Five Tips Maria Colgan Master Product Manager For In-‐Memory

Francois Bermond Schneider Electric

Sudhi Vajayakumar Principal DBA Yahoo

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |


Safe Harbor Statement

The following is intended to outline our general product direcSon. It is intended for informaSon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcSonality, and should not be relied upon in making purchasing decisions. The development, release, and Sming of any features or funcSonality described for Oracle’s products remains at the sole discreSon of Oracle.

3


Program Agenda

In-‐Memory – A Store Not A Cache

Don’t Forget Your Old Friend

Meet In-‐Memory’s Best Friend

In-‐Memory’s Frienemy

Everything Is BeXer with Friends

1

2

3

4

5


Oracle Database In-‐Memory

•  BOTH row and column formats for same table •  TransacSonally consistent

•  AnalyScs automaScally uses new vector-‐opSmized in-‐memory column format

•  OLTP uses proven row format

•  No applicaSon changes required

6

Normal Buffer Cache

New In-‐Memory Format

SALES SALES Row

Format Column Format

SALES


In-‐Memory Row Format: Slower for AnalyScs

SELECT COL4 FROM MYTABLE;

7

X X X X X

RESULT

Row Format

Buffer Cache

Needs to skip over unneeded data


In-‐Memory Columnar Format: Faster for AnalyScs

SELECT COL4 FROM MYTABLE;

8

RESULT

Column Format

IM Column Store

RESULT

X X X X Scans only the data required by the query


Configuring : In-‐Memory Column Store

• New Component of SGA

• Controlled by INMEMORY_SIZE parameter

• Minimum size of 100MB

•  SGA_TARGET must be large enough to accommodate

•  StaSc Pool

System Global Area SGA

Buffer Cache

Shared Pool

Log Buffer

Large Pool

In-‐Memory Area

Other

SELECT * FROM V$SGA;

NAME VALUE -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐-‐-‐ Fixed Size 2927176 Variable Size 570426808 Database Buffers 4634022912 Redo Buffers 13848576 In-‐Memory Area 1024483648


•  What is a cache?

In-‐Memory A Store – Not A Cache

A

B

C

Buffer Cache

A B C E A •  A pool of memory

•  Data automaScally brought into memory based on access

•  Data automaScally aged out

•  Good example: Oracle Database Buffer Cache

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11

In-‐Memory A Store – Not A Cache

SALES

Pure In-‐Memory Columnar

• What is a store?

• A staSc pool of memory

• You decide what objects are populated in memory

Alter table SALES INMEMORY

• Objects don’t age out • Objects automaScally kept transacSonally consistent


In-‐Memory Area: ComposiSon •  Contains two subpools for:

–  In Memory Compression Units (IMCUs)

–  Snapshot Metadata Units (SMUs)

•  IMCUs contain column formaXed data

•  SMUs contain metadata and transacSonal informaSon

•  Space required for SMU pool is a small percentage (typically <10%) of In-‐Memory Area

•  Current pool sizes and status visible in V$INMEMORY_AREA view

SMU SMU

SMU SMU

SMU SMU

SMU SMU

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU IMCU

In Memory Area

Column Format Data

Metadata

12


Program Agenda


Don’t Forget Your Old Friend – The OpSmizer




1

2

3

4

5


In-‐Memory & The OpSmizer

• Decision to use the In-‐Memory Column Store is made by the OpSmizer

• Decision based on tradiSonal table staSsScs & new In-‐Memory staSsScs

•  In-‐Memory staSsScs automaScally supplied to the OpSmizer at parse

•  In-‐Memory staSsScs include – What columns from the table are populated – How much of the table has been populated

– What data pruning can be achieved based on where clause predicates

– 

How can you tell what the opSmizer decided?


In-‐Memory & The OpSmizer Plan Display

15

SELECT count(*) FROM sales WHERE quantity_sold > 30 or (quantity_sold < 10 and prod_id = 50); -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | Id | Operation | Name | Pstart| Pstop | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | 0 | SELECT STATEMENT | | | | | 1 | SORT AGGREGATE | | | | | 2 | PARTITION RANGE ALL | | 1 | 16 | |* 3 | TABLE ACCESS INMEMORY FULL| SALES | 1 | 16 | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Predicate Information (identified by operation id): -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 3 -‐ inmemory(("QUANTITY_SOLD">30 OR "QUANTITY_SOLD"<10) AND ("QUANTITY_SOLD">30 OR "PROD_ID"=50 AND "QUANTITY_SOLD"<10)) filter("QUANTITY_SOLD">30 OR "PROD_ID"=50 AND "QUANTITY_SOLD"<10)


Tracing: SALES is parSally inmemory

16

*************************************** BASE STATISTICAL INFORMATION *********************** Table Stats:: Table: SALES Alias: SALES (Using composite stats) #Rows: 960 SSZ: 0 LGR: 0 #Blks: 12 AvgRowLen: 30.00 NEB: 0 ChainCnt: 0.00 SPC: 0 RFL: 0 RNF: 0 CBK: 0 CHR: 0 KQDFLG: 1 #IMCUs: 7 IMCRowCnt: 560 IMCJournalRowCnt: 14 #IMCBlocks: 7 IMCQuoSent: 0.583333 Index Stats:: Index: SALES_CHANNEL_BIX Col#: 4 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 5 LB/K: 2.00 DB/K: 12.00 CLUF: 60.00 NRW: 60.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00 KQDFLG:

1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_CUST_BIX Col#: 2 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 630 LB/K: 1.00 DB/K: 1.00 CLUF: 927.00 NRW: 927.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00

KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_PROD_BIX Col#: 1 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 766 LB/K: 1.00 DB/K: 1.00 CLUF: 938.00 NRW: 938.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00

KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_PROMO_BIX Col#: 5 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 116 LB/K: 1.00 DB/K: 1.00 CLUF: 133.00 NRW: 133.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00

KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_TIME_BIX Col#: 3 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 620 LB/K: 1.00 DB/K: 1.00 CLUF: 620.00 NRW: 620.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00

KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE IMC Implied predicates generated: Table SALES [SALES]: (null)"SALES"."QUANTITY_SOLD">30 OR "SALES"."QUANTITY_SOLD"<10 …

Table: SALES Alias: SALES (Using composite stats) #Rows: 960 SSZ: 0 LGR: 0 #Blks: 12 AvgRowLen: 30.00 NEB: 0 ChainCnt: 0.00 SPC: 0 RFL: 0 RNF: 0 CBK: 0 CHR: 0 KQDFLG: 1 #IMCUs: 7 IMCRowCnt: 560 #IMCBlocks: 7 IMCQuotient: 0.583333

select count(*) from sales where quantity_sold > 30 or (quantity_sold < 10 and prod_id = 5000);


In-‐Memory & The OpSmizer

• As in all cases an accurate set of OpSmizer staSsScs is required

• Use DBMS_STATS.GATHER_*_STATS procedures to gather staSsScs – Analyze command has long since been deprecated – Use default values as much as possible especially AUTO_SAMPLE_SIZE

• Use histograms to make the OpSmizer aware of any data skews – New types of histograms in 12c provide more detailed informaSon

• Use extended staSsScs to make the OpSmizer aware of correlaSon – Column group staSsScs used for both single table cardinality esSmates, joins & aggregaSon


Program Agenda



Meet In-‐Memory’s Best Friend -‐ ParSSoning



1

2

3

4

5


In-‐Memory’s BFF -‐ ParSSoning

Large Table Difficult to Manage

ParSSons Divide and Conquer Easier to Manage Improve Performance

Transparent to applicaFons

SALES JANUARY 2014

MARCH 2014

FEBRUARY 2014

APRIL 2014


In-‐Memory’s BFF -‐ ParSSoning

• Not all ParSSons need to have INMEMORY aXribute

• ParSSons can have different compression levels

• Allows for easy ILM strategy as data can be added or removed from memory by parSSon

CREATE TABLE ORDERS …… PARTITION BY RANGE …… (PARTITION p1 …… NO INMEMORY PARTITION p2 …… INMEMORY MEMCOMPRESS FOR DML, PARTITION p3 …… INMEMORY MEMCOMPRESS FOR QUERY, : PARTITION p200 …… INMEMORY MEMCOMPRESS FOR CAPACITY );


IMCU

IMCU

IMCU

IMCU

In-‐Memory

In-‐Memory’s BFF – ParSSoning Enables Table Expansion

21

P1: ORDER_STATUS=‘OPEN’

SELECT * FROM SALES where ORDER_DATE between‘2013-‐01-‐01’ and ‘2014-‐01-‐01’;

P2: ORDER_STATUS=‘CLOSED’

P1: Best access path is full table scan

SALES

P2: Best access path is index scan

Local index on ORDER_DATE

IDX_SALES: ORDER_DATE


In-‐Memory’s BFF – ParSSoning Enables Table Expansion -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | Id | Operation | Name | Pstart| Pstop | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | 0 | SELECT STATEMENT | | | | | 1 | SORT AGGREGATE | | | | | 2 | VIEW | VW_TE_2 | | | | 3 | UNION-‐ALL | | | | | 4 | PARTITION RANGE SINGLE | | 1 | 1 | |* 5 | TABLE ACCESS INMEMORY FULL| SALES | 1 | 1 | | 6 | PARTITION RANGE SINGLE | | 2 | 2 | |* 7 | INDEX RANGE SCAN | IDX_SALES | 2 | 2 | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 5 -‐ inmemory("ORDER_DATE“>=TO_DATE(' 2013-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' 2014-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss')) filter("ORDER_DATE“>=TO_DATE(' 2013-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' 2014-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss')) 7 -‐ access("ORDER_DATE“>=TO_DATE(' 2013-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' 2014-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss'))


In-‐Memory and ParSSon Exchange Loads

DBA

1. Create external table for flat files

3. Set INMEMORY aXribute on

4. Populate TMP_SALES in column store

2. Use CTAS command to create non-‐parSSoned table TMP_SALES & gather opSmizer staSsSc

5. Alter table Sales exchange parSSon May_15_2014 with table tmp_sales

Sales table populated in the In-‐Memory column store

A�er exchange data immediately available for In-‐Memory scan


Program Agenda




In-‐Memory’s Frienemy -‐ Indexes


1

2

3

4

5


TradiSonal Mixed Workload OLTP & AnalySc Reports

•  Most Indexes in a mixed workload environment are only used for analytic queries

•  Inserting one row into a table requires updating 10-20 analytic indexes: Slow!

•  Indexes only speed up predictable queries & reports

Table

1 – 3 OLTP

Indexes

10 – 20 AnalySc Indexes

25


In-‐Memory Column Store Replaces AnalySc Indexes

•  Fast analytics on any columns •  Better for unpredictable analytics

•  Less tuning & administration

• Column Store not persistent so update cost is much lower •  OLTP & batch run faster

Table

1 – 3 OLTP

Indexes In-‐Memory Column Store

26

But how do you determine which indexes to drop?


QuesSons To Determine Which Indexes To Drop

• Is the index needed for referenSal integrity? – Primary Key Index – Foreign Key Index

• How selecSve is the index? – Number of rows returned

• Is the index used by transacSonal workload or analyScal queries?


Update TransacSons Benchmark

• Dataset -‐ Sales Accounts •  Main table has 1 Primary Key + 21 secondary indexes

• Comparison of In-‐Memory versus

-‐ All indexes

-‐ No secondary indexes

28


Performance benchmark – Update TransacSons From 5x to 9x faster


Added Benefit – ReducSon in Storage Capacity Over 70% reducFon in storage capacity due to analyFc index removal

0 50 100 150 200 250 300 350

Objects Undo Redo

Size in GBs

SECONDARY INDEXES

TABLES & PK INDEXES


0

20

40

60

80

100

Buffer Cache

In-‐Memory

AnalySc Query Benchmark In-‐Memory Versus Buffer Cache From 7x to 128x faster

Elapse Sme (Seconds)

Millions of rows returned 2000 300 30 5 0.5


Program Agenda





Everything Is BeXer with Friends – In-‐Memory And RAC

1

2

3

4

5


In-‐Memory and RAC

33

•  Each node in RAC has its own IM column store

•  Objects are automaScally distributed across column store

•  Scale-‐out policy is at segment level (table, (sub)parSSon)

•  In-‐Memory queries parallelized across nodes to access local column data


In-‐Memory and RAC

•  Policy is user-‐specifiable

•  Controlled by DISTRIBUTE subclause

•  Distribute by rowid range •  Distribute by parSSon •  Distribute AUTO

34

ALTER TABLE sales INMEMORY DISTRIBUTE BY PARTITION;

ALTER TABLE COSTS INMEMORY DISTRIBUTE AUTO;


Oracle Database In-‐Memory: Unique Fault Tolerance

35

Only Available on Engineered Systems

•  Similar to storage mirroring

•  Duplicate in-‐memory columns on another node

•  Enabled per table/parSSon •  ApplicaSon transparent

•  DownSme eliminated by using duplicate a�er failure


Oracle Database In-‐Memory: Unique Fault Tolerance

36

ALTER TABLE sales INMEMORY DUPLICATE;

ALTER TABLE COSTS INMEMORY DISTRIBUTE AUTO DUPLICATE ALL;

•  Policy is user-‐specifiable

•  Controlled by DUPLICATE subclause

•  DUPLICATE •  DUPLICATE ALL

•  Can improve performance by enabling co-‐located joins


Oracle Database In-‐Memory Top Five Tips

• Remember it’s a store -‐ you control when data goes in & out

• Help the opSmizer help you: gather representaSve set of staSsScs

• Use parSSoning: provides fine grained control on what data is populated

• Remove unnecessary indexes: improves DML performance & reduce index confusion

• Use RAC to scale out: not only increases capacity but performance


Date Title LocaFon Speaker Tuesday Sept 30th 5:00 – 5:45

MulStenant Meets In-‐Memory Moscone South Room 102

Maria Colgan & Patrick Wheeler

Wednesday Oct 1st 14:00 – 14:45

Oracle Database In-‐Memory: Under the Hood

Moscone South Room 104

Tirthankar Lahiri – Vice President, Oracle

Thursday Oct 2nd 09:30 – 10:15

Top Five Things to Know About Oracle Database In-‐Memory


Maria Colgan with Schneider Electric & Yahoo

Thursday Oct 2nd 12:00 – 12:45

Oracle OpSmizer Meets Oracle Database In-‐Memory


Dinesh Das & Bud Endress

Monday 16:15 Tuesday 18:45 Wednesday 16:15 Thursday 13:00

Oracle Database In-‐Memory Boot Camp: Everything You Need to Get Started

Hotel Nikko Room Peninsula

Maria Colgan & Andy Rivenes

Oracle Database In-‐Memory Schedule for Oracle Open World

con7248 top 5 things to know about oracle in-memory database

Documents