con7248 top 5 things to know about oracle in-memory database
Post on 03-Dec-2015
7 Views
Preview:
DESCRIPTION
TRANSCRIPT
Oracle Database In-‐Memory Top Five Tips Maria Colgan Master Product Manager For In-‐Memory
Francois Bermond Schneider Electric
Sudhi Vajayakumar Principal DBA Yahoo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direcSon. It is intended for informaSon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcSonality, and should not be relied upon in making purchasing decisions. The development, release, and Sming of any features or funcSonality described for Oracle’s products remains at the sole discreSon of Oracle.
3
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
In-‐Memory – A Store Not A Cache
Don’t Forget Your Old Friend
Meet In-‐Memory’s Best Friend
In-‐Memory’s Frienemy
Everything Is BeXer with Friends
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
In-‐Memory – A Store Not A Cache
Don’t Forget Your Old Friend
Meet In-‐Memory’s Best Friend
In-‐Memory’s Frienemy
Everything Is BeXer with Friends
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory
• BOTH row and column formats for same table • TransacSonally consistent
• AnalyScs automaScally uses new vector-‐opSmized in-‐memory column format
• OLTP uses proven row format
• No applicaSon changes required
6
Normal Buffer Cache
New In-‐Memory Format
SALES SALES Row
Format Column Format
SALES
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory Row Format: Slower for AnalyScs
SELECT COL4 FROM MYTABLE;
7
X X X X X
RESULT
Row Format
Buffer Cache
Needs to skip over unneeded data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory Columnar Format: Faster for AnalyScs
SELECT COL4 FROM MYTABLE;
8
RESULT
Column Format
IM Column Store
RESULT
X X X X Scans only the data required by the query
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Configuring : In-‐Memory Column Store
• New Component of SGA
• Controlled by INMEMORY_SIZE parameter
• Minimum size of 100MB
• SGA_TARGET must be large enough to accommodate
• StaSc Pool
System Global Area SGA
Buffer Cache
Shared Pool
Log Buffer
Large Pool
In-‐Memory Area
Other
SELECT * FROM V$SGA;
NAME VALUE -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ -‐-‐-‐-‐-‐-‐-‐-‐-‐ Fixed Size 2927176 Variable Size 570426808 Database Buffers 4634022912 Redo Buffers 13848576 In-‐Memory Area 1024483648
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• What is a cache?
In-‐Memory A Store – Not A Cache
A
B
C
Buffer Cache
A B C E A • A pool of memory
• Data automaScally brought into memory based on access
• Data automaScally aged out
• Good example: Oracle Database Buffer Cache
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11
In-‐Memory A Store – Not A Cache
SALES
Pure In-‐Memory Columnar
• What is a store?
• A staSc pool of memory
• You decide what objects are populated in memory
Alter table SALES INMEMORY
• Objects don’t age out • Objects automaScally kept transacSonally consistent
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory Area: ComposiSon • Contains two subpools for:
– In Memory Compression Units (IMCUs)
– Snapshot Metadata Units (SMUs)
• IMCUs contain column formaXed data
• SMUs contain metadata and transacSonal informaSon
• Space required for SMU pool is a small percentage (typically <10%) of In-‐Memory Area
• Current pool sizes and status visible in V$INMEMORY_AREA view
SMU SMU
SMU SMU
SMU SMU
SMU SMU
IMCU IMCU
IMCU IMCU
IMCU IMCU
IMCU IMCU
In Memory Area
Column Format Data
Metadata
12
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
In-‐Memory – A Store Not A Cache
Don’t Forget Your Old Friend – The OpSmizer
Meet In-‐Memory’s Best Friend
In-‐Memory’s Frienemy
Everything Is BeXer with Friends
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory & The OpSmizer
• Decision to use the In-‐Memory Column Store is made by the OpSmizer
• Decision based on tradiSonal table staSsScs & new In-‐Memory staSsScs
• In-‐Memory staSsScs automaScally supplied to the OpSmizer at parse
• In-‐Memory staSsScs include – What columns from the table are populated – How much of the table has been populated
– What data pruning can be achieved based on where clause predicates
–
How can you tell what the opSmizer decided?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory & The OpSmizer Plan Display
15
SELECT count(*) FROM sales WHERE quantity_sold > 30 or (quantity_sold < 10 and prod_id = 50); -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | Id | Operation | Name | Pstart| Pstop | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | 0 | SELECT STATEMENT | | | | | 1 | SORT AGGREGATE | | | | | 2 | PARTITION RANGE ALL | | 1 | 16 | |* 3 | TABLE ACCESS INMEMORY FULL| SALES | 1 | 16 | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Predicate Information (identified by operation id): -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 3 -‐ inmemory(("QUANTITY_SOLD">30 OR "QUANTITY_SOLD"<10) AND ("QUANTITY_SOLD">30 OR "PROD_ID"=50 AND "QUANTITY_SOLD"<10)) filter("QUANTITY_SOLD">30 OR "PROD_ID"=50 AND "QUANTITY_SOLD"<10)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Tracing: SALES is parSally inmemory
16
*************************************** BASE STATISTICAL INFORMATION *********************** Table Stats:: Table: SALES Alias: SALES (Using composite stats) #Rows: 960 SSZ: 0 LGR: 0 #Blks: 12 AvgRowLen: 30.00 NEB: 0 ChainCnt: 0.00 SPC: 0 RFL: 0 RNF: 0 CBK: 0 CHR: 0 KQDFLG: 1 #IMCUs: 7 IMCRowCnt: 560 IMCJournalRowCnt: 14 #IMCBlocks: 7 IMCQuoSent: 0.583333 Index Stats:: Index: SALES_CHANNEL_BIX Col#: 4 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 5 LB/K: 2.00 DB/K: 12.00 CLUF: 60.00 NRW: 60.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00 KQDFLG:
1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_CUST_BIX Col#: 2 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 630 LB/K: 1.00 DB/K: 1.00 CLUF: 927.00 NRW: 927.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00
KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_PROD_BIX Col#: 1 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 766 LB/K: 1.00 DB/K: 1.00 CLUF: 938.00 NRW: 938.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00
KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_PROMO_BIX Col#: 5 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 116 LB/K: 1.00 DB/K: 1.00 CLUF: 133.00 NRW: 133.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00
KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE Index: SALES_TIME_BIX Col#: 3 USING COMPOSITE STATS LVLS: 0 #LB: 12 #DK: 620 LB/K: 1.00 DB/K: 1.00 CLUF: 620.00 NRW: 620.00 SSZ: 0.00 LGR: 0.00 CBK: 0.00 GQL: 0.00 CHR: 0.00
KQDFLG: 1 BSZ: 1 KKEISFLG: 1 ALL PARTITIONS USABLE IMC Implied predicates generated: Table SALES [SALES]: (null)"SALES"."QUANTITY_SOLD">30 OR "SALES"."QUANTITY_SOLD"<10 …
Table: SALES Alias: SALES (Using composite stats) #Rows: 960 SSZ: 0 LGR: 0 #Blks: 12 AvgRowLen: 30.00 NEB: 0 ChainCnt: 0.00 SPC: 0 RFL: 0 RNF: 0 CBK: 0 CHR: 0 KQDFLG: 1 #IMCUs: 7 IMCRowCnt: 560 #IMCBlocks: 7 IMCQuotient: 0.583333
select count(*) from sales where quantity_sold > 30 or (quantity_sold < 10 and prod_id = 5000);
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory & The OpSmizer
• As in all cases an accurate set of OpSmizer staSsScs is required
• Use DBMS_STATS.GATHER_*_STATS procedures to gather staSsScs – Analyze command has long since been deprecated – Use default values as much as possible especially AUTO_SAMPLE_SIZE
• Use histograms to make the OpSmizer aware of any data skews – New types of histograms in 12c provide more detailed informaSon
• Use extended staSsScs to make the OpSmizer aware of correlaSon – Column group staSsScs used for both single table cardinality esSmates, joins & aggregaSon
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
In-‐Memory – A Store Not A Cache
Don’t Forget Your Old Friend
Meet In-‐Memory’s Best Friend -‐ ParSSoning
In-‐Memory’s Frienemy
Everything Is BeXer with Friends
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory’s BFF -‐ ParSSoning
Large Table Difficult to Manage
ParSSons Divide and Conquer Easier to Manage Improve Performance
Transparent to applicaFons
SALES JANUARY 2014
MARCH 2014
FEBRUARY 2014
APRIL 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory’s BFF -‐ ParSSoning
• Not all ParSSons need to have INMEMORY aXribute
• ParSSons can have different compression levels
• Allows for easy ILM strategy as data can be added or removed from memory by parSSon
CREATE TABLE ORDERS …… PARTITION BY RANGE …… (PARTITION p1 …… NO INMEMORY PARTITION p2 …… INMEMORY MEMCOMPRESS FOR DML, PARTITION p3 …… INMEMORY MEMCOMPRESS FOR QUERY, : PARTITION p200 …… INMEMORY MEMCOMPRESS FOR CAPACITY );
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
IMCU
IMCU
IMCU
IMCU
In-‐Memory
In-‐Memory’s BFF – ParSSoning Enables Table Expansion
21
P1: ORDER_STATUS=‘OPEN’
SELECT * FROM SALES where ORDER_DATE between‘2013-‐01-‐01’ and ‘2014-‐01-‐01’;
P2: ORDER_STATUS=‘CLOSED’
P1: Best access path is full table scan
SALES
P2: Best access path is index scan
Local index on ORDER_DATE
IDX_SALES: ORDER_DATE
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory’s BFF – ParSSoning Enables Table Expansion -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | Id | Operation | Name | Pstart| Pstop | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | 0 | SELECT STATEMENT | | | | | 1 | SORT AGGREGATE | | | | | 2 | VIEW | VW_TE_2 | | | | 3 | UNION-‐ALL | | | | | 4 | PARTITION RANGE SINGLE | | 1 | 1 | |* 5 | TABLE ACCESS INMEMORY FULL| SALES | 1 | 1 | | 6 | PARTITION RANGE SINGLE | | 2 | 2 | |* 7 | INDEX RANGE SCAN | IDX_SALES | 2 | 2 | -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ 5 -‐ inmemory("ORDER_DATE“>=TO_DATE(' 2013-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' 2014-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss')) filter("ORDER_DATE“>=TO_DATE(' 2013-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' 2014-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss')) 7 -‐ access("ORDER_DATE“>=TO_DATE(' 2013-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss') AND "ORDER_DATE"<=TO_DATE(' 2014-‐01-‐01 00:00:00', 'syyyy-‐mm-‐dd hh24:mi:ss'))
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory and ParSSon Exchange Loads
DBA
1. Create external table for flat files
3. Set INMEMORY aXribute on
4. Populate TMP_SALES in column store
2. Use CTAS command to create non-‐parSSoned table TMP_SALES & gather opSmizer staSsSc
5. Alter table Sales exchange parSSon May_15_2014 with table tmp_sales
Sales table populated in the In-‐Memory column store
A�er exchange data immediately available for In-‐Memory scan
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
In-‐Memory – A Store Not A Cache
Don’t Forget Your Old Friend
Meet In-‐Memory’s Best Friend
In-‐Memory’s Frienemy -‐ Indexes
Everything Is BeXer with Friends
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
TradiSonal Mixed Workload OLTP & AnalySc Reports
• Most Indexes in a mixed workload environment are only used for analytic queries
• Inserting one row into a table requires updating 10-20 analytic indexes: Slow!
• Indexes only speed up predictable queries & reports
Table
1 – 3 OLTP
Indexes
10 – 20 AnalySc Indexes
25
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory Column Store Replaces AnalySc Indexes
• Fast analytics on any columns • Better for unpredictable analytics
• Less tuning & administration
• Column Store not persistent so update cost is much lower • OLTP & batch run faster
Table
1 – 3 OLTP
Indexes In-‐Memory Column Store
26
But how do you determine which indexes to drop?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
QuesSons To Determine Which Indexes To Drop
• Is the index needed for referenSal integrity? – Primary Key Index – Foreign Key Index
• How selecSve is the index? – Number of rows returned
• Is the index used by transacSonal workload or analyScal queries?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Update TransacSons Benchmark
• Dataset -‐ Sales Accounts • Main table has 1 Primary Key + 21 secondary indexes
• Comparison of In-‐Memory versus
-‐ All indexes
-‐ No secondary indexes
28
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Performance benchmark – Update TransacSons From 5x to 9x faster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Added Benefit – ReducSon in Storage Capacity Over 70% reducFon in storage capacity due to analyFc index removal
0 50 100 150 200 250 300 350
Objects Undo Redo
Size in GBs
SECONDARY INDEXES
TABLES & PK INDEXES
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
0
20
40
60
80
100
Buffer Cache
In-‐Memory
AnalySc Query Benchmark In-‐Memory Versus Buffer Cache From 7x to 128x faster
Elapse Sme (Seconds)
Millions of rows returned 2000 300 30 5 0.5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
In-‐Memory – A Store Not A Cache
Don’t Forget Your Old Friend
Meet In-‐Memory’s Best Friend
In-‐Memory’s Frienemy
Everything Is BeXer with Friends – In-‐Memory And RAC
1
2
3
4
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory and RAC
33
• Each node in RAC has its own IM column store
• Objects are automaScally distributed across column store
• Scale-‐out policy is at segment level (table, (sub)parSSon)
• In-‐Memory queries parallelized across nodes to access local column data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-‐Memory and RAC
• Policy is user-‐specifiable
• Controlled by DISTRIBUTE subclause
• Distribute by rowid range • Distribute by parSSon • Distribute AUTO
34
ALTER TABLE sales INMEMORY DISTRIBUTE BY PARTITION;
ALTER TABLE COSTS INMEMORY DISTRIBUTE AUTO;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory: Unique Fault Tolerance
35
Only Available on Engineered Systems
• Similar to storage mirroring
• Duplicate in-‐memory columns on another node
• Enabled per table/parSSon • ApplicaSon transparent
• DownSme eliminated by using duplicate a�er failure
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory: Unique Fault Tolerance
36
ALTER TABLE sales INMEMORY DUPLICATE;
ALTER TABLE COSTS INMEMORY DISTRIBUTE AUTO DUPLICATE ALL;
• Policy is user-‐specifiable
• Controlled by DUPLICATE subclause
• DUPLICATE • DUPLICATE ALL
• Can improve performance by enabling co-‐located joins
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-‐Memory Top Five Tips
• Remember it’s a store -‐ you control when data goes in & out
• Help the opSmizer help you: gather representaSve set of staSsScs
• Use parSSoning: provides fine grained control on what data is populated
• Remove unnecessary indexes: improves DML performance & reduce index confusion
• Use RAC to scale out: not only increases capacity but performance
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Date Title LocaFon Speaker Tuesday Sept 30th 5:00 – 5:45
MulStenant Meets In-‐Memory Moscone South Room 102
Maria Colgan & Patrick Wheeler
Wednesday Oct 1st 14:00 – 14:45
Oracle Database In-‐Memory: Under the Hood
Moscone South Room 104
Tirthankar Lahiri – Vice President, Oracle
Thursday Oct 2nd 09:30 – 10:15
Top Five Things to Know About Oracle Database In-‐Memory
Moscone South Room 104
Maria Colgan with Schneider Electric & Yahoo
Thursday Oct 2nd 12:00 – 12:45
Oracle OpSmizer Meets Oracle Database In-‐Memory
Moscone South Room 102
Dinesh Das & Bud Endress
Monday 16:15 Tuesday 18:45 Wednesday 16:15 Thursday 13:00
Oracle Database In-‐Memory Boot Camp: Everything You Need to Get Started
Hotel Nikko Room Peninsula
Maria Colgan & Andy Rivenes
Oracle Database In-‐Memory Schedule for Oracle Open World
top related