Oracle Database 12c In-Memory
Saurabh K. GuptaDatabase Product Management
Who am I?
• Principal Technologist, Database Product Management• Author of “Oracle Advanced PL/SQL
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Author of “Oracle Advanced PL/SQL Developer Professional Guide”, Technical Reviewer, blogger• Experience in Oracle Databases, design, development and administration, High Availability solutions, Exadata
3
Accelerate OLTPReal Time
Analytics
No Changes to
Applications
Exploit latest
generation
hardware
Oracle Database In-Memory Goals
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
CPU100x 2x
Become a Real-Time Enterprise
• Data-Driven
• Rapidly make decisions based on real-time data
• Agile
AGILEDATA-
DRIVEN
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Agile
• Respond quickly to change
• Efficient
• Continually improve processes and profitability
5
EFFICIENT
Row Format Databases vs. Column Format Databases
Row
� Transactions run faster on row format
– Example: Insert or query a sales order
– Fast processing few rows, many columns
SALES
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Column
� Analytics run faster on column format
– Example : Report on sales totals by region
– Fast accessing few columns, many rows
SALES
6
Row Format Databases vs. Column Format Databases
Row
� Insert a new sales order on row format
– One row insert = FAST
SALES
INSERT
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 7
Column
� Insert a new sales order in Column Format
– Many column inserts = S L O W
StoresSALES
INSERT
INSERTINSERT
INSERT
Until Now Must Choose One Format and Suffer Tradeoffs
Breakthrough: Dual Format Database
• BOTH row and column formats for same table
• Simultaneously active and transactionally consistent
Memory Memory
SALES SALES
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
transactionally consistent
• Analytics & reporting use new in-memory Column format
• OLTP uses proven row format
8
SALES SALES
RowFormat
ColumnFormat
OLTP is Slowed Down by Analytic Indexes
Insert rate decreases as
number of indexes
increases
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Column Store Replaces Analytic Indexes
• Fast analytics on any columns
• Better for unpredictable analytics
• Less tuning & administration
Table
1 – 3 OLTP
IndexesIn-Memory
Column Store
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Less tuning & administration
• Column Store not persistent so update cost is much lower
• OLTP & batch run faster
10
Oracle In-Memory Requires Zero Application Changes
Full Functionality - No restrictions on SQL
Easy to Implement - No migration of data
Fully Compatible - All existing applications run unchanged
Fully Multitenant - Oracle In-Memory is Cloud Ready
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Fully Multitenant - Oracle In-Memory is Cloud Ready
Uniquely Achieves All In-Memory Benefits With No Application Changes
11
� Configuring the In-Memory Column Store
� Populating the In-Memory Column Store
� Querying the In-Memory Column Store
Program Agenda
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 12
� Querying the In-Memory Column Store
� Monitoring the In-Memory Column Store
System Global Area SGA
Buffer Cache Shared Pool Redo Buffer
Configuring : In-Memory Column Store
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Buffer Cache
Large PoolOther shared
Memory
Components
In-Memory
Area
Configuring : In-Memory Column Store
• Controlled by INMEMORY_SIZEparameter
•Minimum size of 100MB
• SGA_TARGET must be large
SELECT * FROM V$SGA;
NAME VALUE------------------ ---------
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• SGA_TARGET must be largeenough to accommodate
• Static Pool
------------------ ---------Fixed Size 2927176Variable Size 570426808Database Buffers 4634022912Redo Buffers 13848576In-Memory Area 1024483648
� Configuring the In-Memory Column Store
� Populating the In-Memory Column Store
� Querying the In-Memory Column Store
Program Agenda
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 15
� Querying the In-Memory Column Store
� Monitoring the In-Memory Column Store
Oracle In-Memory Columnar Technology
• Pure in-memory column format
• Not persistent, and no logging
• Quick to change data: fast OLTP
• 2x to 20x compression
Pure In-Memory Columnar
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 16
• 2x to 20x compression
• Enabled at table or partition level
• Available on all hardware platforms
SALES
Populating : In-Memory Column Store
• Populate is the term used to bring data into the In-Memory column store
• Populate is used instead of load because load is commonly used to mean inserting new data into the database
• Populate doesn’t bring new data into the database, it brings existing data into
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Populate doesn’t bring new data into the database, it brings existing data into memory and formats it in an optimized columnar format
• Population is completed by a new set of background processes
–ORA_W001_orcl
–Number of processes controlled by INMEMORY_MAX_POPULATE_SERVERS
17
Populating : In-Memory Column Store
• New INMEMORY ATTRIBUTE
• Following segment types are
eligible
• Tables
• Partitions
ALTER TABLE sales INMEMORY;
ALTER TABLE sales NO INMEMORY;
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Partitions
• Subpartition
• Materialized views
• Following segment types not
eligible
• IOTs
• Hash clusters
• Out of line LOBs
CREATE TABLE customers …… PARTITION BY LIST(PARTITION p1 …… INMEMORY,(PARTITION p2 …… NO INMEMORY); Pure OLTP
Features
Populating : In-Memory Column Store
• Possible to populate only
certain columns from a table or
partition
• Order in which objects are
populated controlled by
ALTER TABLE sales INMEMORYNO INMEMORY (PROD_ID);
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
populated controlled by
PRIORITY subclause
• Critical, high, medium, low
• Default – none (populate on
first access)
• Does not control the speed of
population
CREATE TABLE orders (c1 number,c2 varchar(20),c3 number)
INMEMORY PRIORITY CRITICALNO INMEMORY (c1);
Populating : In-Memory Column Store
• Objects compressed during
population
• New compression techniques
• Focused on scan performance
ALTER MATERIALIZED VIEW mv1 INMEMORY MEMCOMPRESS FOR QUERY;
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Controlled by MEMCOMPRESS
subclause
• Multiple levels of compression
• Possible to use a different level for
different partitions in a table
CREATE TABLE trades (Name varchar(20),Desc varchar(200))
INMEMORYMEMCOMPRESS FOR DML(desc);
Populating : In-Memory Column Store
• Different levels
• FOR DML
Use on tables or partitions with
very active DML activity
• FOR QUERY
CREATE TABLE ORDERS …… PARTITION BY RANGE ……(PARTITION p1 ……INMEMORY NO MEMCOMPRESSPARTITION p2 ……
INMEMORY MEMCOMPRESS FOR DML,
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• FOR QUERY
Default mode for most tables
• FOR CAPACITY
For less frequently accessed
segments
• Easy to switch levels as part of
ILM strategy
INMEMORY MEMCOMPRESS FOR DML,PARTITION p3 ……
INMEMORY MEMCOMPRESS FOR QUERY,:
PARTITION p200 …… INMEMORY MEMCOMPRESS FOR CAPACITY);
Identifying : Tables With INMEMORY Attribute
• New INMEMORY column in
*_TABLES dictionary tables
• INMEMORY is a segment
attribute
SELECT table_name, inmemoryFROM USER_TABLES;
TABLE_NAME INMEMORY• USER_TABLESdoesn’t display
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
------------ --------CHANNELS DISABLEDCOSTSCUSTOMERS DISABLEDPRODUCTS ENABLEDSALESTIMES DISABLED
• USER_TABLESdoesn’t display
segment attributes for logical
objects
• Both COSTS & SALES are
partitioned => logical objects
• INMEMORY attribute also
reported in *_TAB_PARTITIONS
Identifying : Tables With INMEMORY Attribute
• New view v$IM_SEGMENTS
• Indicate:
• Objects populated in memory
• Current population status
SELECT segment_name name, population_status statusFROM v$IM_SEGMENTS;
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Can also be used to determine
compression ratio achieved
NAME STATUS------------ --------PRODUCTS COMPLETEDSALES STARTED
Oracle Compression Advisor And In-Memory
• Easy way to determine
memory requirements
• Use DBMS_COMPRESSION
• Applies MEMCOMPRESS to
sample set of data from a table
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
sample set of data from a table
• Returns estimated
compression ratio
Oracle In-Memory Advisor
• New In-Memory Advisor
• Analyzes existing DB workload
via AWR & ASH repositories
• Provides list of objects that
would benefit most from being
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
would benefit most from being
populated into IM column
store
� Configuring the In-Memory Column Store
� Populating the In-Memory Column Store
� Querying the In-Memory Column Store
Program Agenda
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 26
� Querying the In-Memory Column Store
� Monitoring the In-Memory Column Store
Why is an In-Memory scan faster than the buffer cache?
SELECT COL4 FROM MYTABLE;
XX
Buffer Cache
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 27
XXXX
RESULT
Row Format
Why is an In-Memory scan faster than the buffer cache?
SELECT COL4 FROM MYTABLE;
IM Column Store
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 28
RESULT
Column Format
RESULT
XXXX
Oracle In-Memory Column Store Storage Index
• Each column is the made up of multiple column units
• Min / max value is recorded for each column unit in a storage index
Memory
Min 1
Max 3
Min 4
Example: Find sales from stores with a store_id of 8 or higher
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
for each column unit in a storage index
• Storage index provides partition pruning like performance for ALL queries
29
SALES
ColumnFormat
Min 4
Max 7
Min 8
Max 12
Min 13
Max 15
Orders of Magnitude Faster Analytic Data Scans
CPU
PR
OM
O_
ID
Example:Find all salesWith PROMO_ID 9999
Memory• Each CPU core scans local in-
memory columns
• Scans use super fast SIMD vector instructions
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 30
LoadmultiplePROMO_ID
values
VectorCompare all valuesan 1 cycle
CPU9999
99999999
9999VE
CTO
RR
EG
IST
ER
vector instructions
• Originally designed for graphics & science
• Billions of rows/sec scan rate per CPU core
• Row format is millions/sec
Identifying : INMEMORY Table Scan
• Optimizer fully aware
• Cost model adapted to consider INMEMORY scan
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• New access method TABLE ACCESS IN MEMORY FULL
• Can be disabled via new parameter
• INMEMORY_QUERY
Identifying : INMEMORY Table Scan
• NEW Session level statistics
• Best way to determine if In-Memory was use
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
• Best way to measure the benefits of In-Memory scan
Joining and Combining Data Also Dramatically Faster
• Converts joins of data in multiple tables into fast column scans
Example:Find all orders placed on Christmas eve
LINEORDERDATE_DIM
Datekey is
24122013
Da
te
Da
teK
ey
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
scans
• Joins tables 10x faster
33
Da
teK
ey
Type=d.d_date='December 24, 2013'
Da
te
Da
teK
ey
Am
ou
nt
Sum
Identifying : INMEMORY Joins• Bloom filters enable joins to be
converted into fast column scans
• Tried and true technology originally released in 10g
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
originally released in 10g
• Same technique used to offload joins on Exadata
Multiple bloom filter
• Multi-table joins are common in analytic queries
• Possible to apply multiple bloom filters to the fact table due to sophisticated
Select d.d_year, s.s_nation, sum(lo_revenue - lo_supplycost)
From LINEORDER l, DATE_DIM d, PART p, SUPPLIER s
Where l.lo_orderdate = d.d_datekeyAnd l.lo_partkey = p.p_partkey
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
bloom filters to the fact table due to sophisticated Optimizer
• Simultaneously joining to all of the dimension tables while scanning the fact table
And l.lo_partkey = p.p_partkeyAnd l.lo_suppkey = s.s_suppkeyAnd p.p_mfgr= 'MFGR#12’And s.s_region = 'AMERICA’Group by d.d_year, s.s_nationOrder by d.d_year, s.s_nation;
Multiple Bloom Filter
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
In-Memory Aggregation
• Dynamically creates in-memory report outline
• Then report outline filled-in during fast fact scan
Example: Report sales of footwear in outlet stores
SalesProducts
In-Memory Report Outline
Footwear
$
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
during fast fact scan
• Reports run much faster without predefined cubes
37
Stores
Ou
tle
tsFootwear
OutletsSales
$$$
$$$$
Identifying : INMEMORY Aggregation
Hash Group By
� Monitoring the In-Memory Column Store� Monitoring the In-Memory Column Store
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Vector Group By
� Monitoring the In-Memory Column Store� Monitoring the In-Memory Column Store
� Configuring the In-Memory Column Store
� Populating the In-Memory Column Store
� Querying the In-Memory Column Store
Program Agenda
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 39
� Querying the In-Memory Column Store
� Monitoring the In-Memory Column Store
New In-Memory Central Screen in EM
In-Memory Area (GB) :4.00
In-Memory Area Used (GB) :1.79
Object map integrated with
12c heatmap feature
New In-Memory Central Screen in EM
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
In-Memory Area Used (GB) :1.79
New In-Memory Central Screen in EM
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
New In-Memory Central Screen in EM
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
New In-Memory Entries in CPU Wait Class
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
New In-Memory Entries in CPU Wait Class
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
In-Memory Query in SQL Monitor
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |