Date post: | 16-Apr-2017 |
Category: |
Technology |
Upload: | oracleimc-innovation-modernization-center |
View: | 568 times |
Download: | 0 times |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory Powering the Real-Time Enterprise
Rhine Singh Migration consultant Database & Options Oracle AxizWorkgroup Partner Hub South Africa
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
3
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Why did Oracle introduce Database In-Memory ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory option Goals
5
Real-Time Analytics Trivial to Implement
No Application Changes Not Limited by Memory
100X
Risk-Free
Proven Scale-Out, Availability, Security
Accelerate Mixed Workload
Analytics Transactions
Run analytics on Operational
Systems
Enable Real-Time Business Decisions
Real-Time Analytics
100X
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
What is an analytic query?
Which products give us our highest
margins?
Who are the top 10 sales reps in the north
west region this month?
If I get a 20% discount on widget A, how
much will our margins improve?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Row Format Databases vs. Column Format Databases
Rows Stored Contiguously
Transactions run faster on row format
– Example: Query or Insert a sales order – Fast processing few rows, many columns
Columns Stored
Contiguously
Analytics run faster on column format
– Example : Report on sales totals by region – Fast accessing few columns, many rows
SALES
SALES
7
Until Now Must Choose One Format and Suffer Tradeoffs
Query
Qu
ery
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How has Oracle dealt with this ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Breakthrough: Dual Format Database
• BOTH row and column formats for same table
• Simultaneously active and transactionally consistent
• Analytics & reporting use new in-memory Column format
• OLTP uses proven row format
9
Normal Buffer Cache
New In-Memory Format
SALES SALES
Row Format
Column Format
SALES
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How does it work ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Columnar Technology
• Pure in-memory column format • Not persistent, and no logging
• Quick to change data: fast OLTP
• Enabled at table or partition • Only active data in-memory
• 2x to 20x compression typical
• Available on all hardware platforms
11
SALES
Pure In-Memory Columnar
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Dual-Format Database
ID PROD STORE QTY AMOUNT
983 MILK SF 5 20
111 BREAD LA 20 45
695 APPLE LA 10 5
12
SALES
ID 983 111 695
PROD MILK BREAD APPLE
STORE SF LA LA
QTY 5 20 10
AMOUNT 20 45 5
Column Store
Rotate 90°
SALES
Row Store
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Dual-Format Database Logical Representation
13
1:983,MILK,SF,5,20-2:111,BREAD,LA,20,45- 3:695,APPLE,LA,10,5
983:1,111:2,695:3-MILK:1,BREAD:2, APPLE:3-SF:1,LA:2:3-20:1,45:2,5:3
ROW Representation
Column Representation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Populating : In-Memory Column Store
• New INMEMORY ATTRIBUTE
• Following segment types are eligible
• Tables
• Partitions
• Subpartition
• Materialized views
• Following types not eligible
• IOTs
• Hash clusters
• Out of line LOBs
CREATE TABLE customers ……
PARTITION BY LIST
(PARTITION p1 …… INMEMORY,
(PARTITION p2 …… NO INMEMORY);
ALTER TABLE sales INMEMORY;
ALTER TABLE sales NO INMEMORY;
Pure OLTP Features
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Users And Schemas Used In This Demo
RTL schema is a stripped-down and
modified version of Oracle Retail
Data Model.
LABUSER schema is a copy of the
Sales History (SH) schema, which
is a part of the example schemas supplied by the Oracle Database.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Verify IN-Memory Area Configuration
Connect to Database • CONNECT / AS SYSDBA
Check if the In-Memory parameter is enabled • SHOW PARAMETER INMEMORY
Current value for SGA – Not set for In-Memory yet • SELECT NAME, VALUE FROM V$SGA;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Initialization Parameters Related To IM Column Store
• INMEMORY_SIZE
• INMEMORY_FORCE
• INMEMORY_QUERY
• INMEMORY_CLAUSE_DEFAULT
• INMEMORY_MAX_POPULATE_SERVERS
• NMEMORY_OPTIMIZER_AWARE
• INMEMORY_TRICKLE_POPULATE_SERVERS_PERCENT
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
The In-Memory Clause
• MEMCOMPRESS – To Enable the object for the IM column store
• DUPLICATE – In a RAC environment each database node has its own IM Column Store.
• DISTRIBUTE - This attributes also pertains to a RAC environment. When an objectis too big to fit into the IM Column
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-Memory Column Store Compression
• IMCUs compressed during population
• Controlled by MEMCOMPRESS sub-clause
• Ascending order of compression levels: • FOR DML (for heavily updated tables, slower for queries)
• FOR QUERY LOW (default: light compression and fastest)
• FOR QUERY HIGH (slightly more compression)
• FOR CAPACITY LOW (balances capacity and performance)
• FOR CAPACITY HIGH (maximizes capacity)
• Allows in-memory storage tiering
19
DML
QUERY LOW
QUERY HIGH
CAPACITY LOW
CAPACITY HIGH H
ott
er
Par
titi
on
s C
old
er
Par
titi
on
s
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Enable The In-Memory Column Store
Need to Alter the system in order to set the In-Memory size • ALTER SYSTEM SET INMEMORY_SIZE=1G SCOPE=SPFILE;
View the details of the new In-Memory parameter • SHOW PARAMETER SGA_TARGET
Need to Shut down & start-up the DB after the In-Memory area has been set • SHUTDOWN IMMEDIATE • STARTUP
Once restarted In-Memory parameter new value becomes visible • SHOW PARAMETER INMEMORY_SIZE
View the change in the SGA area • SELECT NAME, VALUE FROM V$SGA;
Query the amount of available memory allocated to each pool • SELECT POOL, ALLOC_BYTES, USED_BYTES FROM V$INMEMORY_AREA;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Enabling A New Object CONNECT LABUSER/labuser
Creating a new table and enabling it for In-Memory CREATE TABLE PARTS ( PART_ID NUMBER, NAME VARCHAR2(100), MFR VARCHAR2(50), CATEGORY VARCHAR2(30), BRAND VARCHAR2(50)) INMEMORY; Checking the In-Memory attributes for table ‘PARTS’ SELECT INMEMORY, INMEMORY_PRIORITY, INMEMORY_COMPRESSION, INMEMORY_DISTRIBUTE, INMEMORY_DUPLICATE FROM USER_TABLES WHERE TABLE_NAME = 'PARTS';
Disable In-Memory on an object ALTER TABLE PARTS NO INMEMORY ;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory Advisor
• Analytic processing performance improvement factors based on – Elimination of user I/O waits, cluster transfer waits, buffer cache latch waits, etc.
– Cost: Estimated memory size with various compression options • Certain query processing advantages related to specific compression types.
• Decompression cost heuristics per specific compression types.
– Benefit: Estimated DB time reduction of processing analytic workload
• Report & SQL plan selectivity, number of columns in the result set, etc
– Vary IM Column Store size to receive specific loading plan
– Generate DDL script with all the tables/partitions/sub-partitions recommended
– Top SQL benefits from given configuration
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
– A name for the SPA Task
– The STS you’ve created containing the SQL to be tested with Database In-Memory
– The parameter you wish to test – ‘inmemory_query’
– The two values of the parameter – ‘DISABLE’ and ‘ENABLE’.
– The metric to be used for comparison between the two trials.
For testing Database In-Memory
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Advisor Reports
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Advisor Reports
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Why is an In-Memory scan faster than the buffer cache?
SELECT COL4 FROM MYTABLE;
26
X X X X X
RESULT
Row Format
Buffer Cache
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Why is an In-Memory scan faster than the buffer cache?
SELECT COL4 FROM MYTABLE;
27
RESULT
Column Format
IM Column Store
RESULT
X X X X X
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Column Store Storage Index
• Each column is made up of multiple column units
• Min / max value is recorded for each column unit in a storage index
• Storage index provides partition pruning like performance for ALL queries
28
Memory
SALES Column Format
Min 1 Max 3
Min 4 Max 7
Min 8 Max 12
Min 7 Max 15
Example: Find all sales from stores with a store_id of 8
?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Orders of Magnitude Faster Analytic Data Scans
• Each CPU core scans local in-memory columns
• Scans use super fast SIMD vector instructions
• Originally designed for graphics & science
• Billions of rows/sec scan rate per CPU core
• Row format is millions/sec
29
Vec
tor
Reg
iste
r Load multiple region values
Vector Compare all values an 1 cycle
CPU
Memory
REG
ION
CA
CA CA
CA
Example: Find sales in region of CAlifornia
> 100x Faster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Joining and Combining Data Also Dramatically Faster
• Converts joins of data in multiple tables into fast column scans
• Joins tables 10x faster
30
Example: Find total sales in outlet stores
Sales Stores
Sto
re ID
StoreID in 15, 38, 64
Type=‘Outlet’
Typ
e
Sum
Sto
re ID
Am
ou
nt
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Generates Reports Instantly
• Dynamically creates in-memory report outline
• Then report outline filled-in during fast fact scan
• Reports run much faster
• Without predefined cubes
• Also offloads report filtering to Exadata Storage servers
31
Example: Report sales of footwear in outlet stores
Sales
Stores
Products
In-Memory Report Outline
Footwear
Ou
tlet
s
$ $$
$ $$$
Footwear
Sales Outlets
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How does it impact OLTP environments ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Complex OLTP is Slowed by Analytic Indexes
• Most Indexes in complex OLTP (e.g. ERP) databases are only used for analytic queries
• Inserting one row into a table requires updating 10-20 analytic indexes: Slow!
• Indexes only speed up predictable queries & reports
Table
1 – 3 OLTP
Indexes
10 – 20 Analytic Indexes
33
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
OLTP is Slowed Down by Analytic Indexes
Insert rate decreases as number of indexes
increases
# of Fully Cached Indexes (Disk Indexes are much slower)
34
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Column Store Replaces Analytic Indexes
• Fast analytics on any columns
• Better for unpredictable analytics
• Less tuning & administration
• Column Store not persistent so update cost is much lower
• OLTP & batch run faster
Table
1 – 3 OLTP
Indexes In-Memory
Column Store
35
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How much of a difference does it make when running a simple aggregation query
Step1: We will connect as the RTL user Enable In-Memory on all tables in the RTL schema • ALTER TABLE DWB_RTL_SLS_RETRN_LINE_ITEM INMEMORY; • ALTER TABLE DWB_RTL_TNDR_LI INMEMORY; • ALTER TABLE DWB_RTL_TRX INMEMORY; • ALTER TABLE DWR_ORG_BSNS_UNIT INMEMORY; • ALTER TABLE DWR_SKU_ITEM INMEMORY;
Step 2: Populate the RTL tables. Execute a simple COUNT(*) query on each of the tables in RTL schema to trigger their population, as shown below. • SELECT COUNT(*) FROM DWB_RTL_SLS_RETRN_LINE_ITEM; • SELECT COUNT(*) FROM DWB_RTL_TNDR_LI; • SELECT COUNT(*) FROM DWB_RTL_TRX; • SELECT COUNT(*) FROM DWR_ORG_BSNS_UNIT; • SELECT COUNT(*) FROM DWR_SKU_ITEM;
Step 3: Running A Simple Aggregation Query SET TIMING ON; SELECT MAX(EXTENDED_AMT) FROM DWB_RTL_SLS_RETRN_LINE_ITEM;
Step 4: View the time laps difference. Execute the query against the buffer cache ALTER SESSION SET INMEMORY_QUERY=DISABLE; SELECT MAX(EXTENDED_AMT) FROM DWB_RTL_SLS_RETRN_LINE_ITEM;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How much of a difference does it make when using a filter
Step 1: SET TIMING ON SELECT SUM(EXTENDED_AMT) FROM RTL.DWB_RTL_SLS_RETRN_LINE_ITEM WHERE ACTN_CD = 'Return'; SET TIMING OFF
Step 2: Now, execute the same query against the buffer cache ALTER SESSION SET INMEMORY_QUERY=DISABLE; SET TIMING ON SELECT SUM(EXTENDED_AMT) FROM RTL.DWB_RTL_SLS_RETRN_LINE_ITEM WHERE ACTN_CD = 'Return'; SET TIMING OFF ALTER SESSION SET INMEMORY_QUERY=ENABLE;
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How can I scale this solution ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Scale-Out In-Memory Database to Any Size
• Scale-Out across servers to grow memory and CPUs
• In-Memory queries parallelized across servers to access local column data
• Direct-to-wire InfiniBand protocol speeds messaging on Engineered Systems
39
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-Memory Speed + Capacity of Low Cost Disk
DISK
PCI FLASH
DRAM
Cold Data
Hottest Data
Active Data
• Size not limited by memory
• Data transparently accessed across tiers
• Each tier has specialized algorithms & compression
• Simultaneously Achieve:
• Speed of DRAM
• I/Os of Flash
• Cost of Disk
40
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Scale-Up for Maximum In-Memory Performance
M6-32
Big Memory Machine
32 TB DRAM
32 Socket
3 Terabyte/sec Bandwidth
• Scale-Up on large SMPs
• Algorithms NUMA optimized
• SMP scaling removes overhead of distributing queries across servers
• Memory interconnect far faster than any network
41
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory: Industrial Strength Availability
RAC
ASM
RMAN
Data Guard &
GoldenGate • Pure In-Memory format does not
change Oracle’s storage format, logging, backup, recovery, etc.
• All Oracle’s proven availability technologies work transparently
• Protection from all failures
• Node, site, corruption, human error, etc.
42
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory: Unique Fault Tolerance
• Similar to storage mirroring
• Duplicate in-memory columns on another node
• Enabled per table/partition
• E.g. only recent data
• Application transparent
• Downtime eliminated by using duplicate after failure
43
Only Available on Engineered Systems
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Example: Duplicate Strategy For a Star Schema
• Fact tables are distributed by partition
• Dimension tables are duplicated (DUPLICATE ALL)
• Co-locates joins between the distributed fact table partitions and the dimension tables
44
D
D
D
D
F D
D
D
D
F D
D
D
D
F
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How easy is it to get started ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory: Simple to Implement
1. Configure Memory Capacity
• inmemory_size = XXX GB
2. Configure tables or partitions to be in memory
• alter table | partition … inmemory;
3. Later drop analytic indexes to speed up OLTP
46
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Requires Zero Application Changes
Full Functionality - ZERO restrictions on SQL
Easy to Implement - No migration of data
Fully Compatible - All existing applications run unchanged
Fully Multitenant - Oracle In-Memory is Cloud Ready
Uniquely Achieves All In-Memory Benefits With No Application Changes
47
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
How are customers using Oracle Database In-Memory ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-Memory Use Cases OLTP
• Real-time reporting directly on OLTP source data
• Removes need for separate ODS
• Speeds data extraction
Data Warehouse
• Staging/ETL/Temp not a candidate
• Write once, read once
• All or a subset of Foundation Layer
• For time sensitive analytics
• Potential to replace Access Layer
49
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Getting The Most From In-Memory
• Fast cars speed up travel, not meetings
• In-Memory speeds up analytic data access, not: – Network round trips, logon/logoff
– Parsing, PL/SQL, complex functions
– Data processing (as opposed to access) • Complex joins or aggregations where not much data is filtered before processing
– Load and select once – Staging tables, ETL, temp tables
Understand Where it Helps
50
Know your bottleneck!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Applications Examples
CHANGE
Oracle Application Module Improvement Elapsed Time
Demantra - Promotion Planning 102x Faster 18.3 min to 11 second
Oracle In-Memory Transportation Mgmt 1031x Faster 17 min to < 1 seconds
Oracle E-business Suite - In-Memory Cost Management
628x Faster
58 hours to 6 minutes
Demantra - Consumption Driven Planning 13.5x Faster 12.7 hours to 56 min
JD Edwards - Supply Chain Planning and Analytics
76x Faster 28 minutes to 3 min
Siebel Marketing - List Import 141x Faster 115 mins to 49 seconds
51
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Getting The Most From In-Memory
• Avoid stop and go traffic – Process data in sets of rows in the Database
– Not one row at a time in the application
• Plan ahead, take shortest route – Help the optimizer help you: Gather representative statistics using DBMS_STATS
• Use all your cylinders – Enable parallel execution
– In-Memory removes storage bottlenecks allowing parallelism to increase
The Driver Matters
52
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory Key Differentiators
#1 OLTP Scales with Unique Dual Format
#2 Both Scale Up & Scale Out supported
#3 Maximum Availability is Built In
#4 Database Size Isn’t Limited by Memory
#5 All Applications are Transparently Compatible
#6 No new Personnel Needed
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Where can I get more information ?
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Join the Conversation
55
https://twitter.com/db_inmemory
https://blogs.oracle.com/in-memory/
Related White Papers • Oracle Database In-Memory White Paper • Oracle Database In-Memory Aggregation Paper • When to use Oracle Database In-Memory • Oracle Database In-Memory Advisor
Related Database In-Memory Free Webcasts • Oracle Database In-Memory meets Data Warehousing
Related Videos • In-Memory YouTube Channel • Database Industry Experts Discuss Oracle Database In-Memory (11:10) • Software on Silicon
Any Additional Questions • Oracle Database In-Memory Blog
https://www.facebook.com/OracleDatabase
http://www.oracle.com/goto/dbim.html
Additional Resources
Oracle Partners Sales guided learning path is intended for sales representatives at partner organizations who specialize in selling and positioning Oracle Database In-Memory to their own customer base.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Q&A
56
Rhine Singh Oracle AxizWorkgroup ISV Migration Center Consultant [email protected] ISV Migration Center blog: http://blogs.oracle.com/imc ISV Migration Center email: [email protected]
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Oracle.com Partner Hub oracle.com/partners/goto/hub-ecemea
• Migration Center Team Blog blogs.oracle.com/imc
feeds.feedburner.com/oracleIMC
• Partner Webcast Recordings youtube.com/OracleIMCteam
• Partner Webcast Presentations slideshare.net:/Oracle_IMC_team
Oracle Partner Hub ISV Migration Center • twitter.com/OracleIMC
• plus.google.com/+OracleIMC
• facebook.com/OracleIMC
• linkedin.com/groups/Oracle-Partner-Hub-Migration-Center-4535240
57