Date post: | 12-Jul-2015 |
Category: |
Technology |
Upload: | christian-antognini |
View: | 555 times |
Download: | 1 times |
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
2014 © Trivadis
Oracle Database In-Memory and
the Query Optimizer UKOUG Tech14, Liverpool (GB)
Christian Antognini
11 December 2014 Oracle Database In-Memory and the Query Optimizer
1
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
2
@ChrisAntognini
■ Senior principal consultant, trainer and partner
at Trivadis in Zurich (CH)
■ http://antognini.ch
■ Focus: get the most out of Oracle Database
■ Logical and physical database design
■ Query optimizer
■ Application performance management
■ Author of Troubleshooting Oracle Performance (Apress, 2008/2014)
■ OakTable Network, Oracle ACE Director
2014 © Trivadis
1. In-Memory Column Store
2. In-Memory Aggregation
11 December 2014 Oracle Database In-Memory and the Query Optimizer
3
AGENDA
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
In-Memory Column Store
4
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
5
Architecture Overview
2014 © Trivadis
■ Data is populated into the IMCS in chunks called IM Column Unit (IMCU)
11 December 2014 Oracle Database In-Memory and the Query Optimizer
6
IM Column Unit
… C1 C2 C3 C1 C2 C3 C1 C2 C3
2014 © Trivadis
■ They are similar to Exadata storage indexes
■ Do not store information about NULL values
■ They are automatically created and maintained for each column inside the IMCS
■ They are only stored in memory
■ They keep track of the minimum and maximum values in each IMCU
■ Their aim is to avoid accessing IMCUs that do not contain relevant data
11 December 2014 Oracle Database In-Memory and the Query Optimizer
7
IM Storage Indexes
2014 © Trivadis
■ The cost model has been enhanced to be IM aware
■ It requires IM statistics
■ The total cost is a combination of both non-IM costs and IM costs
■ Enhancements are disabled if OPTIMIZER_INMEMORY_AWARE = FALSE or
OPTIMIZER_FEATURES_ENABLE < 12.1.0.2
■ IMCS is not disabled
■ Costs are based on the former cost model
11 December 2014 Oracle Database In-Memory and the Query Optimizer
8
IM-Aware Query Optimizer
2014 © Trivadis
■ The following statistics are used by the query optimizer:
■ Number of IMCUs
■ Number of blocks with data into the IMCUs
■ Number of rows stored into the IMCUs
■ Fraction of the rows stored into the IMCUs
- Value between 0 and 1
■ Number of rows stored into the transaction journal
■ IM statistics are computed on the fly during hard parses
■ A cursors is not invalidated when the IM statistics it is based on changes
11 December 2014 Oracle Database In-Memory and the Query Optimizer
9
IM Statistics
2014 © Trivadis
■ Two operations are related to IM scans:
■ TABLE ACCESS INMEMORY FULL
■ MAT_VIEW ACCESS INMEMORY FULL
■ Also used in case of Exadata storage
■ IM scans are selected by the query optimizer regardless of whether data is
actually stored into the IMCS
■ The current behavior seems buggy to me
11 December 2014 Oracle Database In-Memory and the Query Optimizer
10
IM Scans
2014 © Trivadis
■ Pruning based on storage indexes
■ Pruning based on compression metadata
■ Dictionary-based compression algorithms store metadata that contain a list of distinct
values for each column within IMCU
■ Evaluation against compressed data is possible
■ Except for CAPACITY LOW/HIGH
■ Evaluation takes advantage of SIMD
■ Execution plans provide a new type of predicate: INMEMORY
■ Also used in case of predicates offloaded to Exadata storage
11 December 2014 Oracle Database In-Memory and the Query Optimizer
11
Predicate Evaluation Taking Place During IM Scans
2014 © Trivadis
■ The query optimizer can generate predicates that are evaluated during IM scans
■ The generated predicates are used as explicit predicates
11 December 2014 Oracle Database In-Memory and the Query Optimizer
12
Generation of Implied Predicates
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
13
IM Scan Example
SELECT * FROM t WHERE id = 1 OR id > 1 AND n < 42
-------------------------------------------
| Id | Operation | Name |
-------------------------------------------
| 0 | SELECT STATEMENT | |
|* 1 | TABLE ACCESS INMEMORY FULL| T |
-------------------------------------------
1 - inmemory("ID">=1 AND ("N"<42 AND "ID">1 OR "ID"=1))
filter("N"<42 AND "ID">1 OR "ID"=1)
Implied predicate
2014 © Trivadis
■ The INMEMORY_QUERY initialization parameter enables or disables IM scans
at the session or system level
■ IM scans can also be controlled through the (NO_)INMEMORY hints
■ The hints override the INMEMORY_QUERY parameter
11 December 2014 Oracle Database In-Memory and the Query Optimizer
14
Controlling IM Scans
2014 © Trivadis
■ Based on Bloom filters
■ Similar concept than parallel joins in previous versions, but executed serially
■ Filter created by the left input
■ Filter used by the right input
■ Only available for hash joins
■ Only the probe input have to be “IM-enabled”
■ The join is used to filter the false positives
■ IM joins can be controlled through the (NO_)PX_JOIN_FILTER hints
11 December 2014 Oracle Database In-Memory and the Query Optimizer
15
IM Joins
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
16
IM Join Example
SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n < 1E3 ------------------------------------------------ | Id | Operation | Name | ------------------------------------------------ | 0 | SELECT STATEMENT | | |* 1 | HASH JOIN | | | 2 | JOIN FILTER CREATE | :BF0000 | |* 3 | TABLE ACCESS FULL | T1 | | 4 | JOIN FILTER USE | :BF0000 | |* 5 | TABLE ACCESS INMEMORY FULL| T2 | ------------------------------------------------ 1 - access("T1"."ID"="T2"."ID") 3 - filter("T1"."N"<1E3) 5 - inmemory(SYS_OP_BLOOM_FILTER(:BF0000,"T2"."ID")) filter(SYS_OP_BLOOM_FILTER(:BF0000,"T2"."ID"))
2014 © Trivadis
■ Table expansion is a cost-based transformation introduced in 11.2 to leverage
partially unusable indexes
■ It has been made IM aware
■ Useful in case not all partitions are “IM-enabled”
■ It can be controlled through the (NO_)EXPAND_TABLE hints
11 December 2014 Oracle Database In-Memory and the Query Optimizer
17
IM Table Expansion
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
18
IM Table Expansion Example
SELECT count(d) FROM t WHERE d BETWEEN :b1 AND :b2
---------------------------------------------------------------------------
| Operation | Name | Pstart| Pstop |
---------------------------------------------------------------------------
| SELECT STATEMENT | | | |
| SORT AGGREGATE | | | |
| VIEW | VW_TE_1 | | |
| UNION-ALL | | | |
| PARTITION RANGE SINGLE | | 11 | 11 |
| TABLE ACCESS INMEMORY FULL | T | 11 | 11 |
| PARTITION RANGE SINGLE | | 12 | 12 |
| TABLE ACCESS BY LOCAL INDEX ROWID BATCHED| T | 12 | 12 |
| INDEX RANGE SCAN | I | 12 | 12 |
---------------------------------------------------------------------------
The IM scan is only
enabled for the
partition 4
2014 © Trivadis
■ The plan hash value is independent from the kind of FTS
■ TABLE ACCESS FULL
■ TABLE ACCESS STORAGE FULL
■ TABLE ACCESS INMEMORY FULL
■ Features like SQL plan baselines do not break when IMCS is enabled
■ The “best” execution plan can change
11 December 2014 Oracle Database In-Memory and the Query Optimizer
19
Plan Hash Value
2014 © Trivadis
■ _INMEMORY_QUERY_SCAN controls IM scans (TRUE)
■ _OPTIMIZER_INMEMORY_ACCESS_PATH controls whether the query
optimizer costs for IM (TRUE)
■ _OPTIMIZER_INMEMORY_BLOOM_FILTER controls IM joins (TRUE)
■ _OPTIMIZER_INMEMORY_GEN_PUSHABLE_PREDS controls the generation
of implied predicates (TRUE)
■ _OPTIMIZER_INMEMORY_TABLE_EXPANSION controls IM table expansion
(TRUE)
11 December 2014 Oracle Database In-Memory and the Query Optimizer
20
Undocumented Parameters
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
In-Memory Aggregation
21
2014 © Trivadis
■ To optimize queries against a star schema, the query optimizer should do the
following:
■ Start evaluating each dimension table that has restrictions on it
■ Assemble a list with the resulting dimension keys
■ Use this list to extract the matching rows from the fact table
■ This approach cannot be implemented with regular joins
■ The query optimizer can join only two data sets at one time
■ Joining two dimension tables leads to a Cartesian product
■ To solve this problem, Oracle Database implements two query transformations:
■ Star transformation
■ Vector transformation (new in 12.1.0.2 – requires the In-Memory option)
11 December 2014 Oracle Database In-Memory and the Query Optimizer
22
The Star Schema Challenge
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
23
Star Transformation vs. Vector Transformation
Selectivity
Ela
psed T
ime
2014 © Trivadis
■ In the documentation Oracle refers to it as In-Memory Aggregation
■ It isn’t only about aggregation
■ It requires the In-Memory option because it takes advantage of some of its
features
■ INMEMORY_SIZE must be greater than 0
■ The query optimizer considers it when
■ equijoins are used
■ an aggregate function is applied to a column of the fact table
■ the query contains a GROUP BY clause
- ROLLUP, CUBE and GROUPING SETS are not yet supported
11 December 2014 Oracle Database In-Memory and the Query Optimizer
24
Vector Transformation (1)
2014 © Trivadis
■ It’s a cost-based query transformation
■ It can be controlled through the (NO_)VECTOR_TRANSFORM hints
■ As with the star transformation, sometimes is not possible to force it
■ It introduces new row source operations
■ KEY VECTOR CREATE
■ KEY VECTOR USE
■ VECTOR GROUP BY
11 December 2014 Oracle Database In-Memory and the Query Optimizer
25
Vector Transformation (2)
2014 © Trivadis
1. For every dimension table with a filter on it the following operations take place
■ Access table and filter data
■ Create key vector
■ Aggregate data
■ Create temporary table
2. Access the fact table through a FTS and filter data by applying the key vectors
■ On Exadata the filter is not yet offloaded
3. Aggregate data with either a vector or hash aggregation
4. Join data from the fact table to temporary tables
5. Join dimension tables without a filter on them
11 December 2014 Oracle Database In-Memory and the Query Optimizer
26
Vector Transformation – Processing Steps
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
27
Example
SELECT c.customer_class, sum(o.order_total)
FROM orders o, customers c, warehouses w
WHERE o.customer_id = c.customer_id
AND o.warehouse_id = w.warehouse_id
AND c.dob BETWEEN :b1 AND :b2
AND w.warehouse_name BETWEEN :b3 AND :b4
GROUP BY c.customer_class
customers orders warehouses
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
28
Execution Plan – Access Dimension Tables, Create Key Vectors,
and Create Temporary Tables
-------------------------------------------------------------------------
| Id | Operation | Name |
-------------------------------------------------------------------------
| 0 | SELECT STATEMENT | |
| 1 | TEMP TABLE TRANSFORMATION | |
| 2 | LOAD AS SELECT | SYS_TEMP_0FD9DE69C_28565764 |
| 3 | VECTOR GROUP BY | |
| 4 | KEY VECTOR CREATE BUFFERED | :KV0000 |
|* 5 | TABLE ACCESS STORAGE FULL | CUSTOMERS |
| 6 | LOAD AS SELECT | SYS_TEMP_0FD9DE69D_28565764 |
| 7 | VECTOR GROUP BY | |
| 8 | HASH GROUP BY | |
| 9 | KEY VECTOR CREATE BUFFERED | :KV0001 |
|* 10 | TABLE ACCESS STORAGE FULL | WAREHOUSES |
...
2014 © Trivadis
11 December 2014 Oracle Database In-Memory and the Query Optimizer
29
Execution Plan – Access Fact Table by Applying Key Vectors, and
Join Temporary Tables
... | 11 | HASH GROUP BY | | |* 12 | HASH JOIN | | | 13 | MERGE JOIN CARTESIAN | | | 14 | TABLE ACCESS STORAGE FULL | SYS_TEMP_0FD9DE69D_28565764 | | 15 | BUFFER SORT | | | 16 | TABLE ACCESS STORAGE FULL | SYS_TEMP_0FD9DE69C_28565764 | | 17 | VIEW | VW_VT_72AE2D8F | | 18 | VECTOR GROUP BY | | | 19 | HASH GROUP BY | | | 20 | KEY VECTOR USE | :KV0001 | | 21 | KEY VECTOR USE | :KV0000 | |* 22 | TABLE ACCESS STORAGE FULL| ORDERS | ------------------------------------------------------------------------- ... 22 - filter(SYS_OP_KEY_VECTOR_FILTER("O"."CUSTOMER_ID",:KV0000) AND SYS_OP_KEY_VECTOR_FILTER("O"."WAREHOUSE_ID",:KV0001))
2014 © Trivadis
Core Messages
■ In-Memory Column Store
■ The query optimizer is IM aware
■ In-Memory Aggregation
■ Very interesting concept, complements the
star transformation
11 December 2014 Oracle Database In-Memory and the Query Optimizer
30
2014 © Trivadis
Questions and answers ...
2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
Christian Antognini
Principal Senior Consultant
11 December 2014 Oracle Database In-Memory and the Query Optimizer
31