Safe Harbor Statement - Amazon S3Days... · actually almost boring. It just worked - just turn it...

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Oracle Confidential – Internal 2


Oracle Database12cEffectively using the Oracle Database In-Memory Option

Sean StaceyDirector PTS, Server TechnologiesDatabase Product Management


Accelerate Mixed

Workload OLTP

Real Time Analytics No Changes to Applications

Trivial to Implement

Oracle Database In-Memory Goals

100x

2x

4


2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

2018

2019

2020

2021

2022

2023

2024

2025

Oracle 11.1 (GA: Aug 2007)

Oracle 11.2 (GA: Sep 2009)

Oracle 12.1 (GA: Jun 2013)

Release Roadmap

Sustaining Support

AUG 2015 AUG 2012

JAN 2018 JAN 2015

JUN 2021 JUN 2018

Waived Extended Extended Support Premier Support

1

Today

Oracle Database 12c Release 2 Targeted for Release 1HCY2016

(My Oracle Support Note 742060.1 )

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=742060.1




Accelerate Mixed Workload OLTP

Real Time Analytics No Changes to

Applications

Trivial to Implement

Oracle Database In-Memory Goals

100x 2x

2


Until Now Must Choose One Format and Suffer Tradeoffs

Row Format Databases vs. Column Format Databases

Row

Transactions run faster on row format

– Example: Insert or query a sales order – Fast processing few rows, many columns

Column

Analytics run faster on column format

– Example : Report on sales totals by region – Fast accessing few columns, many rows

SALES

SALES

3


Breakthrough: Dual Format Database

• BOTH row and column formats for same table

• Simultaneously active and transactionally consistent

• Analytics & reporting use new in-memory Column format

• OLTP uses proven row format

4

Memory Memory

SALES SALES

Row Format

Column Format


Real-time analytics will create new use cases

• Analytics & reporting against new and external data types

5

New In-Memory Format

Column Format

“Now we can run time-sensitive analytical queries directly

against our OLTP database. This is something we wouldn’t

have dreamt of earlier.”

Arup Nanda Enterprise Architect

Starwood Hotels and Resorts

“Oracle Database In-Memory is a game changer for OLTP, DW,

and mixed workloads. It dramatically improves the

performance of all types of analytical queries.”

“Oracle Database In-Memory is a game changer for OLTP, DW,

and mixed workloads. It dramatically improves the

performance of all types of analytical queries.”

Liviu Horn AVP Database Management McKesson Health Solutions

Liviu Horn AVP Database Management McKesson Health Solutions


Oracle In-Memory Applications Examples

CHANGE

Oracle Application Module Improvement Elapsed Time

Demantra - Promotion Planning 102x Faster 18.3 min to 11 second

Oracle In-Memory Transportation Mgmt 1031x Faster 17 min to <1 seconds

Oracle E-business Suite - In-Memory Cost Management

628x Faster

58 hours to 6 minutes

Demantra - Consumption Driven Planning 13.5x Faster 12.7 hours to 56 min

JD Edwards - Supply Chain Planning and Analytics

76x Faster 28 minutes to 3 min

Siebel Marketing - List Import 141x Faster 115 mins to 49 seconds

8


Getting to Real-Time

9

Source: Richard Hackethorn’s Component’s of Action Time

Data

base S

peed

Best Practices Engineered Systems Database In-Memory

Database Technology


AGENDA

10

High Level Overview Introduction to In-Memory Configuring, Enabling and Populating In-Memory The Optimizer and In-Memory Using In-Memory

1

4

5

3

2 2

3

4

5


AGENDA

11


2

4

5

3

1


Oracle In-Memory Columnar Technology

• Pure in-memory column format

• Not persistent, and no logging

• Quick to change data: fast OLTP

• 2x to 20x compression

• Enabled at table or partition level

• Available on all hardware platforms

12

SALES

Pure In-Memory Columnar


Orders of Magnitude Faster Analytic Data Scans

• Each CPU core scans local in-memory columns

• Scans use super fast SIMD vector instructions

• Originally designed for graphics & science

• Billions of rows/sec scan rate per CPU core

• Row format is millions/sec

13

Vec

tor

Reg

iste

r Load multiple region values

Vector Compare all values an 1 cycle

CPU

Memory

REG

ION

CA

CA

CA

CA

Example: Find all sales in region of CA

> 100x Faster


Joining and Combining Data Also Dramatically Faster

• Converts joins of data in multiple tables into fast column scans

• Joins tables 10x faster

14

Example: Find total sales in outlet stores

Sales Stores

Sto

re ID

Am

ou

nt

Type=Outlet

StoreID in 15, 38, 64

Sto

re ID

Typ

e

Sum


Generates Reports Instantly

• Dynamically creates in-memory report outline

• Then report outline filled-in during fast fact scan

• Reports run much faster without predefined cubes

15

Example: Report sales of footwear in outlet stores

Sales

Stores

Products

In-Memory Report Outline

Footwear

Ou

tlet

s $ $$

$ $$$

Footwear

Sales Outlets


Complex OLTP is Slowed by Analytic Indexes

• Most Indexes in complex OLTP (e.g. ERP) databases are only used for analytic queries

• Inserting one row into a table requires updating 10-20 analytic indexes: Slow!

• Indexes only speed up predictable queries & reports

Table

1 – 3 OLTP

Indexes

10 – 20 Analytic Indexes

16


Complex OLTP is Slowed by Analytic Indexes

17

Insert rate decreases as number of indexes

increases


Column Store Replaces Analytic Indexes

• Fast analytics on any columns

• Better for unpredictable analytics

• Less tuning & administration

• Column Store not persistent so update cost is much lower

• OLTP & batch run faster

Table

1 – 3 OLTP

Indexes In-Memory

Column Store

18


AGENDA

19


3

4

5

1

2


System Global Area SGA

Buffer Cache Shared Pool Redo Buffer

Large Pool Other shared Memory Components

In-Memory Area

Configuring : In-Memory Area


Configuring : In-Memory Column Store

• Column formatted data is stored in In-Memory Column Store

• Is a new component of SGA, called In-Memory Area

• Does not replace the buffer cache

• Controlled by initialization parameter, INMEMORY_SIZE

– Default 0 (Disabled)

– Minimum 100 MB

• In-Memory Area is a Static Pool – Is not controlled or impacted by Automatic Memory Management (AMM)

21



• SGA_TARGET must be large enough to accommodate the IM Column Store • Or MEMORY_TARGET for AMM

• Check if enabled using V$SGA or V$PARAMETER

SELECT * FROM V$SGA;

NAME VALUE

------------------ -----------

Fixed Size 2927176

Variable Size 570426808

Database Buffers 4634022912

Redo Buffers 13848576

In-Memory Area 10244836480


Configuring : In-Memory Area

• Subdivided into 2 pools – IMCU Pool (aka 1 MB) and SMU Pool (aka 64 KB)

– IMCU pool (In-Memory Compression Units): Sized in 1 MB contiguous chunks, stores actual column formatted data

– SMU pool (Snapshot Metadata Units): Sized in 64 KB chunks, stores metadata and transactional information: Transaction Journal

– Contiguous chunks are called IM Extents

– Majority of the space is consumed by IMCU Pool

23

SMU Pool

23

SMU

SMU

SMU

SMU

SMU

SMU

SMU SMU

In Memory Area

IMCU Pool

SMU Pool

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU IMCU

IMCU

IMCU

IMCU

IMCU

…



• V$INMEMORY_AREA: Displays current pool sizes their status

• Useful to detect if IM Column Store is running out of capacity

SQL> SELECT * from V$INMEMORY_AREA;

POOL ALLOC_BYTES USED_BYTES POPULATE_STATUS

---------- ----------- ---------- ---------------

1MB POOL 1307574272 135266304 DONE

64KB POOL 318767104 524288 DONE


Configuring : In-Memory Initialization Parameters

25

INMEMORY_SIZE – Enables and sets the size of the In-Memory column store static pool. The minimum setting is 100M. INMEMORY_CLAUSE_DEFAULT – Use this parameter to set default IM clauses for new tables and materialized views. The default is an empty string.

OPTIMIZER_INMEMORY_AWARE – Set this parameter to make the optimizer cost model enhancements for IM to be enabled or disabled. INMEMORY_QUERY – Set this parameter to enable or disable access to IM. i.e: You can disable access even if the IM column store is populated. The default is ENABLE.


Configuring : In-Memory Initialization Parameters

26

INMEMORY_MAX_POPULATE_SERVERS – Set this parameter for the Maximum number of background processes to populate the IM column store. Set this based on CPU cores. INMEMORY_TRICKLE_REPOPULATE_SERVERS_PERCENT – Set this parameter to limit the background processes for trickle repopulation. It is a percentage of INMEMORY_MAX_POPULATE_SERVERS.

INMEMORY_FORCE – Allows you to specify whether objects specified as INMEMORY are populated into the In-Memory Column Store or not.


Enabling : IM Column Store – INMEMORY Clause

• New INMEMORY clause

• Following segment types are eligible

• Tables

• Partitions

• Subpartition

• Materialized views

• Following segment types not eligible

• IOTs

• Hash clusters

• Out of line LOBs, LONGs

ALTER TABLE sales INMEMORY;

ALTER TABLE sales NO INMEMORY;

CREATE TABLE customers ……

PARTITION BY LIST

(PARTITION p1 …… NO INMEMORY,

(PARTITION p2 …… INMEMORY);

Pure OLTP Features


About the INMEMORY Clause

In-Memory Sub-Clauses

• The in-memory clause has multiple sub-clauses

– PRIORITY: Controls the population priority of the object

– MEMCOMPRESS: Specifies Compression levels

– DUPLICATE: For RAC, to duplicate the IM Store content on other nodes

– DISTRIBUTE: For RAC, to distribute IM Store content, in chunks, to other nodes

– Example:

28

ALTER TABLE sales INMEMORY PRIORITY HIGH;

ALTER TABLE sales INMEMORY MEMCOMPRESS FOR QUERY HIGH PRIORITY MEDIUM;


In-Memory Enablement Object Metadata

• Enablement status can be checked in *_TABLES,

*_SEGMENTS,

*_TAB_PARTITIONS,

*_TAB_SUBPARTITIONS and

DBA_TABLESPACES views

• Tracked in 5 new columns (attributes): • INMEMORY: Object level enablement - ENABLED or DISABLED

• INMEMORY_PRIORITY: Stores the PRIORITY

• INMEMORY_COMPRESSSION: Stores the Compression attributes as defined

• INMEMORY_DUPLICATE: Stores the DUPLICATE clause specification

• INMEMORY_DISTRIBUTE: Stores the DISTRIBUTE clause specification

SELECT TABLE_NAME,

INMEMORY,

INMEMORY_PRIORITY,

INMEMORY_COMPRESSION,

INMEMORY_DISTRIBUTE,

INMEMORY_DUPLICATE

FROM USER_TABLES


Enabling : IM Column Store – Selective Columns

• Possible to enable only certain columns from a table or partition

ALTER TABLE sales INMEMORY

NO INMEMORY (PROD_ID);

ALTER TABLE sales INMEMORY

INMEMORY (PROD_ID, PROD_NAME)

NO INMEMORY (PROD_CATEGORY)


Enabling : Columns With the INMEMORY Attribute

• V$IM_COLUMN_LEVEL captures column-level enablement status

– Also their compression attributes

• The view contains rows only when the in-memory column clause is specified

• Valid Values:

– NO INMEMORY: Column is not part of IMCU

– DEFAULT (Default compression i.e. Query LOW)

– Valid Compression Types: NONE, DML, QUERY [LOW|HIGH], CAPACITY [LOW|HIGH]

SELECT table_name, column_name, inmemory_compression from v$im_column_level;

TABLE_NAME COLUMN_NAME INMEMORY_COMPRESSION

------------------- ----------------------- ----------------------

SALES PROD_ID NO INMEMORY

SALES CUST_ID DEFAULT

SALES TIME_ID DEFAULT


Enabling : Tablespaces for the IM Column Store

• The in-memory clause can be specified as a DEFAULT for the Tablespace

– Using CREATE or the ALTER TABLESPACE statements

• If a tablespace is enabled:

– All tables and materialized views created in the tablespace are enabled for the IM column store by default

– Objects inherit the tablespace in-memory clause (at creation time)

• Object level In-Memory clause overrides the tablespace in-memory clause

32

• ALTER TABLESPACE … DEFAULT INMEMORY

• ALTER TABLESPACE … DEFAULT INMEMORY MEMCOMPRESS QUERY HIGH

• ALTER TABLESPACE … DEFAULT INMEMORY PRIORITY LOW

Example DDL


Populating : In-Memory Column Store

• Populate is the term used to bring data into the In-Memory column store

• Populate is used instead of load because load is commonly used to mean inserting new data into the database

• Populate brings existing row store data into the IM Column Store

– Reformats it in the columnar format

33

• Columns with unsupported data types (LONGs, Out-of-line LOBS) are skipped in population – Rest of the columns are still populated

• Small segments (less than 64k) are skipped

• Segments owned by SYS, SYSTEM are skipped

• Two basic kinds of populate • Initial Populate: Initial creation of IMCU • Repopulate: Recreate an existing IMCU after it has been modified

Note: An object needs to be enabled for INMEMORY prior to its population


Initial Population : In-Memory Column Store

• Initial Population (aka Pre-Population) of an object happens as follows: – Manually:

• Using DBMS_INMEMORY.POPULATE (or REPOPULATE with FORCE)

• When querying the object (via. table scan)

– Automatically: • Using the PRIORITY clause on the object

– At CREATE or ALTER time

• Using INMEMORY_FORCE parameter – Force populates all eligible objects

• Initial Population is also needed when an object acquires new Extents not yet mapped to IMCUs – Via Bulk loads or DML that causes the segment to grow i.e. add new Extents

34



In Memory Area

35

SALES Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

….

….

IMCU Pool SMU Pool

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

Extent

….

….

SMU

SMU SMU

SMU SMU

SMU

SALE

S

IMCU IMCU

IMCU IMCU

IMCU

IMCU

SALE

S

Population

SMU SMU

SMU

IMCU IMCU

IMCU IMCU

Population

SMU



Manual Population

• Querying the Object – SELECT /*+ FULL (s) */ COUNT(*) FROM SALES s;

• Using DBMS_INMEMORY.POPULATE – EXEC DBMS_INMEMORY.POPULATE('LABUSER','SALES');

• Using DBMS_INMEMORY.REPOPULATE with FORCE=>TRUE – EXEC DBMS_INMEMORY.REPOPULATE('LABUSER','SALES', FORCE=>TRUE);

36


Initial Population : In-Memory Column Store Automatic Population – PRIORITY Clause

• Objects can be set to automatically populate after the database is opened, or after the object is created, or altered, or loaded with data – This is possible via the PRIORITY clause

• Objects get populated in a prioritized list, the order of which is controlled by the priority level

• All objects at a given priority level must be fully populated before a lower priority level can commence – CRITICAL > HIGH > MEDIUM > LOW

• The population order can be superseded if an object without a PRIORITY is scanned, thereby triggering its population into IM column store – Also by executing DBMS_INMEMORY.POPULATE on demand

37


Initial Population : IM Column Store Automatic Population

• PRIORITY subclause controls the order of population

• Not the speed

• CRITICAL > HIGH > MEDIUM > LOW

• Default – NONE (populate on first access)

• Priority levels only work on tables and partitions – NOT columns

CREATE TABLE orders

(c1 number,

c2 varchar(20),

c3 number)

INMEMORY PRIORITY CRITICAL

NO INMEMORY (c1);

ALTER TABLESPACE TS_DATA DEFAULT INMEMORY

PRIORTY LOW

select segment_name, populate_status, inmemory_priority,

inmemory_size, bytes_not_populated from v$im_segments;

SEGMENT_NAME POPULATE_STATUS INMEMORY_PRIORITY INMEMORY_SIZE BYTES_NOT_POPULATED

------------ --------------- ----------------- ------------- -------------------

ACCOUNTS STARTED HIGH 196606 2434886912

SALES COMPLETED CRITICAL 135790592 0


Populating : Background Processes

• New IMCU Population is triggered by IMCO or Table Access

– When IMCO wakes up, it checks the PRIORITY Queues

• For newly created INMEMORY objects

• For existing INMEMORY objects having new Extents not yet mapped to IMCUs

• PRIORITY clause is required

• Worker process populates IMCUs

– A set of background processes referred to as worker processes (ora_w001_orcl)

– Each worker process allocates its own IMCU and maps a set of extents to the IMCU

– Number of worker processes is controlled by INMEMORY_MAX_POPULATE_SERVERS

– The database is fully active / accessible while population occurs

– Population is a streaming mechanism, simultaneously columnizing and compressing the data

– Data is not sorted or ordered during population - Read in the same order as it resides in disk blocks

39


In-Memory Column Store - Six Compression Levels


Compressing : In-Memory Column Store

• Objects compressed during population

• New compression techniques

• Focused on scan performance

• Controlled by MEMCOMPRESS subclause

• Multiple levels of compression

• Possible to use a different level for different partitions in a table and different columns in a table

• Easy to switch levels for ILM

ALTER MATERIALIZED VIEW mv1 INMEMORY

MEMCOMPRESS FOR QUERY LOW;

CREATE TABLE trades

(Name varchar(20),

Desc varchar(200))

INMEMORY

MEMCOMPRESS FOR CAPACITY HIGH(desc);


Column Level Compression

• By default, columns inherit the table compression clause

• Compression techniques can be overridden at the column level

– Using the selective column level compression clause

• Overridden column level attributes are stored in V$IM_COLUMN_LEVEL

– Along with their participation status

– This view returns no rows when ALL columns default to the table compression level

42


Oracle Compression Advisor And In-Memory

• Enhanced to include In-Memory compression

• Easy way to determine memory requirements

• Use DBMS_COMPRESSION

• Applies MEMCOMPRESS to sample set of data from a table

• Returns estimated compression ratio


Oracle Compression Advisor and In-Memory

• DBMS_COMPRESSION.GET_COMPRESSION_RATIO

– Comptype: valid values: • COMP_INMEMORY_NOCOMPRESS, COMP_INMEMORY_DML,

• COMP_INMEMORY_CAPACITY_HIGH, COMP_INMEMORY_CAPACITY_LOW

• COMP_INMEMORY_QUERY_HIGH, COMP_INMEMORY_QUERY_LOW

– Subset_numrows: • COMP_RATIO_MINROWS, COMP_RATIO_ALLROWS

44


AGENDA

45


5

3

2 2

3 3

2

1

4


Query Processing – Multiple Optimizations gets applied

• Column Projections

• Vector Processing

• Bloom Filter Joins

• Predicate Push Down

• Predicate Elimination

• Dictionary Pruning

• In-Memory Aggregation

46

How would you measure the benefits?


In-Memory & The Optimizer

• Decision to use the In-Memory Column Store is made by the Optimizer – Optimizer is fully aware of the IM Column Store

• Decision based on traditional table statistics & new In-Memory statistics

• In-Memory statistics automatically supplied to the Optimizer at parse

• In-Memory statistics include – What columns from the table are populated

– How much of the table has been populated

– What data pruning can be achieved based on where clause predicates

How can you tell what the optimizer decided?


In-Memory and The Optimizer Plan Display

• New access method TABLE ACCESS IN MEMORY FULL

– Cost model adapted to consider In-Memory scan

• TABLE ACCESS INMEMORY FULL indicates a POSSIBLE use of column store

– May still not use the store in some cases

– For example, at scan time, if the IM Statistics indicate that INMEMORY access will be costly

48


In-Memory Column Store Statistics

• Database Statistics are a better measure of In-Memory Column Store benefits

• New Session and System level statistics

• Best way to determine if the IM Column Store was used

• Best way to measure the benefits of In-Memory scan

• Display all stats using V$STATNAME

– IM column store related statistics begin with “IM”

– There are currently over 200 new statistics

49

SELECT DISPLAY_NAME

FROM V$STATNAME

WHERE DISPLAY_NAME LIKE ‘IM %’;


In-Memory Scans: Under the Hood

• Wealth of information in AWR reports

• V$SYSSTAT monitors all aspects of in-memory scans

How many rows were scanned

How many rows were avoided by dictionary pruning

How many predicates were optimized

How many IMCUs were min-max pruned

Etc ..

IM scan bytes in-memory IM scan bytes uncompressedIM scan CUs columns accessedIM scan CUs columns decompressedIM scan CUs columns theoretical maxIM scan rowsIM scan rows range excludedIM scan rows excludedIM scan rows optimizedIM scan rows projectedIM scan CUs predicates receivedIM scan CUs predicates appliedIM scan CUs predicates optimizedIM scan CUs prunedIM scan segments ….

IM scan bytes in-memory IM scan bytes uncompressed IM scan CUs columns accessed IM scan CUs columns decompressed IM scan CUs columns theoretical max IM scan rows IM scan rows range excluded IM scan rows excluded IM scan rows optimized IM scan rows projected IM scan CUs predicates received IM scan CUs predicates applied IM scan CUs predicates optimized IM scan CUs pruned IM scan segments minmax eligible ….

50


IM Column Store Statistics

• IM scan bytes uncompressed: Uncompressed size in bytes of all CU involved a scan before optimizations

• IM scan bytes in-memory: Compressed size in bytes of all CUs in scan before optimizations

• IM scan CUs columns accessed: Count of column units accessed by the scan

• IM scan CUs columns theoretical max: Count of columns that would be accessed if each scan looked at all columns in the IMCU

51


Identifying : INMEMORY Joins • Bloom filters enable joins to be

converted into fast column scans

• Tried and true technology originally released in 10g

• Same technique used to offload joins on Exadata


Multiple bloom filter

• Multi-table joins are common in analytic queries

• Possible to apply multiple bloom filters to the fact table due to sophisticated Optimizer

• Simultaneously joining to all of the dimension tables while scanning the fact table

Select d.d_year, s.s_nation,

sum(lo_revenue - lo_supplycost)

From LINEORDER l, DATE_DIM d,

PART p, SUPPLIER s

Where l.lo_orderdate = d.d_datekey

And l.lo_partkey = p.p_partkey

And l.lo_suppkey = s.s_suppkey

And p.p_mfgr= 'MFGR#12’

And s.s_region = 'AMERICA’

Group by d.d_year, s.s_nation

Order by d.d_year, s.s_nation;


Multiple Bloom Filter


Swap Join Input Optimization

Left deep tree

PART

HASH JOIN

LINEORDER

HASH JOIN

SUPPLIER

HASH JOIN

DATEDIM

1 2

3

4

1

2

3


Swap Join Input Optimization

Becomes a right deep tree

PART

HASH JOIN

LINEORDER

HASH JOIN

SUPPLIER

HASH JOIN

DATEDIM

2

3

1

2

3

1

4

Why is this significant? Applying multiple filter on initial scan of LINEORDER reduces the number of rows sent upstream in the plan


Multiple Bloom Filter


In-Memory Aggregation

• A feature of Oracle Database In-Memory

• New Vector Group-By enables extremely efficient in-memory array based aggregation

• Simultaneously accumulates aggregate values into in-memory arrays during fact table scan

• Optimizes star, snowflake and similar queries

• Introduces new internal operators

• Transparent to applications

Oracle Confidential – Internal/Restricted/Highly Restricted 58

Average Elapsed Time, Seconds per Query


In-Memory Aggregation Processing

High-level Processing steps

– Transform joins into scans on the fact table

• Fast in-memory scans and array lookups

– Optimize aggregation using in-memory arrays

• Data structures optimized for in-memory

– Late joins to dimension attribute data

• Minimizes data movement in the execution plan


Example: Report sales by Quarter and Region

Sales

Customers

Date In-Memory

Accumulator (PGA)

Quarters

Reg

ion

s $ $$

$ $$$

Quarters

Sales Regions


Sales

1 2 3 4

1 2 3 4 5 6

1

2

3

4

Aggregation : In-Memory Aggregation Details

Quarters

Regions

1 CUST Date_Dim

Key vectors created on each join column

Customers Key Vector

Date_Dim Key Vector

2 Sales

4 In-Memory Accumulator

3 5 2 10 4 6 20 1 7 08

0 1 3 2 1 0 0 0 1 3 …

0 3 1 5 0 4 3 …

10

20

2 7 05

Customers

DATE

15


When to Expect a Vector Transformation Plan

Query Characteristics

• Join columns between tables “mostly” unique or numeric keys

• Tables populated into the IM column store

• One or more small tables joined to a large table

– Small tables usually 1/10th the size of the large table (or smaller)

– Equijoins

• One or more measures aggregated from the large table – SUM, MIN, MAX, AVERAGE, MEDIAN, etc.

• Statement contains a GROUP BY clause

61


When to Expect a Vector Transformation Plan

Optimizer Costing

• KEY VECTOR CREATE operator adds to ‘up-front’ cost of vector transformation plan

– Cost proportional to the size of dimension tables

– Biased against vector transformation for low cost queries

• KEY VECTOR USE lower cost than HASH JOIN – Biased towards vector transformation for join operations

– More joins, more benefit

– More the number of aggregated rows, more the benefit

62


Vector Transformation Plan Unlikely if…

• Joins are performed between two or more very large tables

• Dimension tables contain more than 2 billion rows

• The system does not have sufficient memory resources

63


Vector Transformation Plan Hints

• VECTOR_TRANSFORM

– Forces a vector transformation on the query block if a transformation is possible

• NO_VECTOR_TRANSFORM

– Prevents a vector transformation plan

SELECT /*+ vector_transform */

d1.year_name,

d1. quarter_name,

d3.region_name,

SUM(f.sales)

FROM time_dim d1,

customer_dim d3,

sales_fact f

WHERE d1.day_id = f.day_id

AND d3.customer_id = f.customer_id

AND d1. year_name ='CY2012'

GROUP BY d1. year_name,

d1.quarter_name,

d3.region_name;



AGENDA

65


4

5

3

2 2

3

5

3

2

1

“In terms of how easy the in-memory option was to use, it was

actually almost boring. It just worked - just turn it on, select

the tables, nothing else to do.”

“In terms of how easy the in-memory option was to use, it was

actually almost boring. It just worked - just turn it on, select

the tables, nothing else to do.”

Mark Rittman Chief Technical Officer Rittman Mead

Mark Rittman Chief Technical Officer Rittman Mead


Keys to Real-Time Data Processing

• Process data in database not application

• Process sets of rows using SQL

• Row by row processing is slow and cannot be parallelized

• Enable in-memory column format

• Enable parallel SQL

• Memory removes storage bottlenecks enabling highly parallel SQL

67


Using In-Memory - Lessons Learned

- parallel query influence - parallel_degree_policy = AUTO (not ADAPTIVE) - Should use AutoDOP when running in RAC - https://blogs.oracle.com/In-Memory/entry/oracle_database_in_memory_on

- patch installation?

- not nulls

- how to determine if it is being used

- Remove HINTS and NON-DEFAULT PARAMETERS

- In Memory aggregation

- indexes - INVISIBLE and UNUSEABLE

- segments < 64K do not get loaded

- turn off adaptive optimizer : optimizer_adaptive_features=off

- Parallel_threads_cpu

- “robbing from peter to pay paul”- don't reduce the size of the buffer_cache to accommodate the IMCS

68

By now we should have discussed all of these lessons learned along the way, if not, this slide is a placeholder or pensieve…


Oracle In-Memory Requires Zero Application Changes

Full Functionality - No restrictions on SQL

Easy to Implement - No migration of data

Fully Compatible - All existing applications run unchanged

Fully Multitenant - Oracle In-Memory is Cloud Ready

Uniquely Achieves All In-Memory Benefits With No Application Changes

69


Oracle In-Memory Requires Zero Application Changes

Full Functionality - No restrictions on SQL

Easy to Implement - No migration of data

Fully Compatible - All existing applications run unchanged

Fully Multitenant - Oracle In-Memory is Cloud Ready

Uniquely Achieves All In-Memory Benefits With No Application Changes

1


Oracle Database In-Memory Differentiators

Orders of magnitude faster performance for analytics

OLTP becomes faster

Complete application transparency

Leverage complete Oracle Security and HA/DR technology

Scale up or scale out

Complements Oracle Exadata

Orders of magnitude faster performance for analytics

OLTP becomes faster

Complete application transparency

Leverage complete Oracle Security and HA/DR technology

Scale up or scale out

Complements Oracle Exadata

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 3

Date post:	29-Apr-2018
Category:	Documents
Upload:	trinhduong
View:	216 times
Download:	1 times

Safe Harbor Statement - Amazon S3Days... · actually almost boring. It just worked - just turn it...

Documents