BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH
Effiziente Abfragen im Star Schema mit der Oracle 12c In-Memory OptionDani Schnider, Trivadis AG
Dani Schnider
2
Principal Consultant undDWH/BI Lead Architectbei Trivadis in Zürich
Kursleiter verschiedener Trivadis-Kurse
Co-Autor des Buches «Data Warehousing mit Oracle –Business Intelligence in der Praxis»
16.03.2016 Star Schema & Oracle 12c In-Memory
Star Schema & Oracle 12c In-Memory3 16.03.2016
Oracle Database In-Memory
Star Schema & Oracle 12c In-Memory4 16.03.2016
The innovation of the Oracle In-Memory option is: We store the data both ways -we store it two ways. We store it both in row format and in memory in columnarformat. Next slide please!
When we process queries, we have all thedata in memory, and we can scan it very veryquickly. So we speed up insert into thedatabase and update to the database two -three - four times by dropping these analyticindexes. Those analytic indexes are muchmore expensive to maintain than the columnstore that replaces the analytic indexes - andthat's magic.
Oracle Database In-Memory Architektur
Star Schema & Oracle 12c In-Memory5 16.03.2016
SGA
Buffer Cache IM Column Store TX Journal
DBWn User IMCO Wnnn
Row Format
C5C1 C2 C3 C4
SELECTUPDATE
Oracle Database In-Memory und Star Schema
Star Schema & Oracle 12c In-Memory6 16.03.2016
Star Schema ist idealer Kandidat für In-Memory Column Store
Nur benötigte Spalten müssen gelesen und aggregiert werden
Keine Bitmap Indizes auf Faktentabelle notwendig
Star Schema & Oracle 12c In-Memory7 16.03.2016
Star Query Optimierung
Query Optimierung auf Star Schemas
Typische Abfragen auf Star Schema:
– Filterkriterien auf Dimensionen
– Fakten werden durch Join mit Dimensionen ermittelt
Problemstellung:
– Tabellen mit Filterkriterien sollten zuerst evaluiert werden
– Join nur auf jeweils zwei Tabellen
– Keine Beziehungen zwischen Dimensionen
1
2
3
3
2
1
16.03.2016 Star Schema & Oracle 12c In-Memory8
Query Optimierung auf Star Schemas
Star Schema & Oracle 12c In-Memory9 16.03.2016
Star Join (Oracle 7)
– Kartesisches Produkt der Dimensionstabellen
Star Transformation (Oracle 8.0.4/8.1/11.2)
– „Join“ zwischen Dimensionen und Bitmap Indizes
– Kombination der Bitmap Indizes
Join Filter Pruning (Oracle 11.1)
– Erstellung von Bloom Filter für Dimension, Zugriff auf Faktentabelle via Bloom Filter
Vector Transformation oder In-Memory Aggregation (Oracle 12.1.0.2, In-Memory Option)
– Key Vector pro Dimension
– Ermittlung von Fakten durch Kombination der Key Vectors
ProductsBitmap Index
SALES_PROD_BIX
Sales(Fact Table)
CustomersBitmap Index
SALES_CUST_BIX
Star Transformation
Star Schema & Oracle 12c In-Memory10
AND Sales(Fact Table)
ProductsBitmap Index
SALES_PROD_BIX
CustomersBitmap Index
SALES_CUST_BIX
16.03.2016
Star Schema & Oracle 12c In-Memory11 16.03.2016
-----------------------------------------------------------| Id | Operation | Name |-----------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | HASH GROUP BY | ||* 2 | HASH JOIN | || 3 | MERGE JOIN CARTESIAN | ||* 4 | TABLE ACCESS FULL | PRODUCTS || 5 | BUFFER SORT | ||* 6 | TABLE ACCESS FULL | CUSTOMERS || 7 | VIEW | VW_ST_222BE09D || 8 | NESTED LOOPS | || 9 | PARTITION RANGE ALL | || 10 | BITMAP CONVERSION TO ROWIDS| || 11 | BITMAP AND | || 12 | BITMAP MERGE | || 13 | BITMAP KEY ITERATION | || 14 | BUFFER SORT | ||* 15 | TABLE ACCESS FULL | PRODUCTS ||* 16 | BITMAP INDEX RANGE SCAN| SALES_PROD_BIX || 17 | BITMAP MERGE | || 18 | BITMAP KEY ITERATION | || 19 | BUFFER SORT | ||* 20 | TABLE ACCESS FULL | CUSTOMERS ||* 21 | BITMAP INDEX RANGE SCAN| SALES_CUST_BIX || 22 | TABLE ACCESS BY USER ROWID | SALES |-----------------------------------------------------------
12
3
Star Schema & Oracle 12c In-Memory12 16.03.2016
| 0 | SELECT STATEMENT | | | 1 | TEMP TABLE TRANSFORMATION | | | 2 | LOAD AS SELECT | SYS_TEMP_0FD9D662B_84B3F4 | |* 3 | TABLE ACCESS FULL | CUSTOMERS | | 4 | HASH GROUP BY | | |* 5 | HASH JOIN | | | 6 | MERGE JOIN CARTESIAN | | |* 7 | TABLE ACCESS FULL | PRODUCTS | | 8 | BUFFER SORT | | | 9 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662B_84B3F4 | | 10 | VIEW | VW_ST_97ECF93 | | 11 | NESTED LOOPS | | | 12 | PARTITION RANGE ALL | | | 13 | BITMAP CONVERSION TO ROWIDS| | | 14 | BITMAP AND | | | 15 | BITMAP MERGE | | | 16 | BITMAP KEY ITERATION | | | 17 | BUFFER SORT | | |* 18 | TABLE ACCESS FULL | PRODUCTS | |* 19 | BITMAP INDEX RANGE SCAN| SALES_PROD_BIX | | 20 | BITMAP MERGE | | | 21 | BITMAP KEY ITERATION | | | 22 | BUFFER SORT | | | 23 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662B_84B3F4 | |* 24 | BITMAP INDEX RANGE SCAN| SALES_CUST_BIX | | 25 | TABLE ACCESS BY USER ROWID | SALES |
12
3
Star Schema & Oracle 12c In-Memory13 16.03.2016
Vector Transformation
Vector Transformation
Star Schema & Oracle 12c In-Memory14 16.03.2016
Phase 1 (für jede Dimension mit Filterkriterien)
1. Scan auf Dimensionstabelle (inkl. Filterung der Daten)
2. Ermittlung von Key Vector
3. Aggregation der Daten (In-Memory Accumulator)
4. Erstellen von temporärer Tabelle
Phase 25. Full Table Scan auf Faktentabelle, Filterung anhand von Key Vectors
6. Aggregation mittels HASH GROUP BY / VECTOR GROUP BY
7. Join auf temporäre Tabellen (Join Back)
8. Ev. Join von weiteren Dimensionen (ohne Filterkriterien)
Vector Transformation
Star Schema & Oracle 12c In-Memory15 16.03.2016
FACTS11 22 100011 24 120012 21 30012 22 320012 24 70013 22 110014 21 200014 24 80014 25 160014 26 70015 23 110015 24 120015 26 50016 22 240016 23 80017 22 130017 25 110018 21 90018 24 210018 26 600
DIM111 Alpha12 Alpha13 Beta14 Beta15 Beta16 Gamma17 Delta18 Delta
DIM221 X green22 X blue23 Y green24 Y blue25 Y red26 Z red
SELECT D1, D21, D22, SUM(FACTS.F)
FROM FACTSJOIN DIM1 ON (...)JOIN DIM2 ON (...)WHERE D1 IN ('Beta',
'Gamma')AND D21 = 'Y'
GROUP BY D1, D21, D22
Vector Transformation
Star Schema & Oracle 12c In-Memory16 16.03.2016
FACTS11 22 100011 24 120012 21 30012 22 320012 24 70013 22 110014 21 200014 24 80014 25 160014 26 70015 23 110015 24 120015 26 50016 22 240016 23 80017 22 130017 25 110018 21 90018 24 210018 26 600
DIM111 Alpha12 Alpha13 Beta14 Beta15 Beta16 Gamma17 Delta18 Delta
DIM221 X green22 X blue23 Y green24 Y blue25 Y red26 Z red
KV100111200
KV2001230
TMP11 Beta2 Gamma
TMP21 Y green2 Y blue3 Y red
0 00 20 00 00 21 01 01 21 31 01 11 21 02 02 10 00 30 00 20 0
Beta Y green 1100Beta Y blue 2000Beta Y red 1600Gamma Y green 800
Star Schema & Oracle 12c In-Memory17 16.03.2016
-------------------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | TEMP TABLE TRANSFORMATION | || 2 | LOAD AS SELECT | SYS_TEMP_0FD9D662E_84B3F4 || 3 | VECTOR GROUP BY | || 4 | KEY VECTOR CREATE BUFFERED| :KV0000 ||* 5 | TABLE ACCESS FULL | PRODUCTS || 6 | LOAD AS SELECT | SYS_TEMP_0FD9D662F_84B3F4 || 7 | VECTOR GROUP BY | || 8 | KEY VECTOR CREATE BUFFERED| :KV0001 ||* 9 | TABLE ACCESS FULL | CUSTOMERS || 10 | HASH GROUP BY | ||* 11 | HASH JOIN | || 12 | MERGE JOIN CARTESIAN | || 13 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662E_84B3F4 || 14 | BUFFER SORT | || 15 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662F_84B3F4 || 16 | VIEW | VW_VT_83032D7B || 17 | VECTOR GROUP BY | || 18 | HASH GROUP BY | || 19 | KEY VECTOR USE | :KV0000 || 20 | KEY VECTOR USE | :KV0001 || 21 | PARTITION RANGE ALL | ||* 22 | TABLE ACCESS FULL | SALES |-------------------------------------------------------------------
12
3
Star Schema & Oracle 12c In-Memory18 16.03.2016
-------------------------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | TEMP TABLE TRANSFORMATION | || 2 | LOAD AS SELECT | SYS_TEMP_0FD9D6696_84B3F4 || 3 | VECTOR GROUP BY | || 4 | KEY VECTOR CREATE BUFFERED | :KV0000 ||* 5 | TABLE ACCESS INMEMORY FULL | PRODUCTS || 6 | LOAD AS SELECT | SYS_TEMP_0FD9D6697_84B3F4 || 7 | VECTOR GROUP BY | || 8 | KEY VECTOR CREATE BUFFERED | :KV0001 ||* 9 | TABLE ACCESS INMEMORY FULL | CUSTOMERS || 10 | HASH GROUP BY | ||* 11 | HASH JOIN | || 12 | MERGE JOIN CARTESIAN | || 13 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6696_84B3F4 || 14 | BUFFER SORT | || 15 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6697_84B3F4 || 16 | VIEW | VW_VT_83032D7B || 17 | VECTOR GROUP BY | || 18 | HASH GROUP BY | || 19 | KEY VECTOR USE | :KV0000 || 20 | KEY VECTOR USE | :KV0001 || 21 | PARTITION RANGE ALL | ||* 22 | TABLE ACCESS INMEMORY FULL| SALES |-------------------------------------------------------------------------
12
3
19
Star Schema & Oracle 12c In-Memory20 16.03.2016
Star oder Vector Transformation?
Star Transformation oder Vector Transformation?
Star Schema & Oracle 12c In-Memory21 16.03.2016
-----------------------------------------------------------------------| Id | Operation | Name |-----------------------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | TEMP TABLE TRANSFORMATION | || 2 | LOAD AS SELECT | SYS_TEMP_0FD9D662B_84B3F4 ||* 3 | TABLE ACCESS FULL | CUSTOMERS || 4 | HASH GROUP BY | ||* 5 | HASH JOIN | || 6 | MERGE JOIN CARTESIAN | ||* 7 | TABLE ACCESS FULL | PRODUCTS || 8 | BUFFER SORT | || 9 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662B_84B3F4 || 10 | VIEW | VW_ST_97ECF93 || 11 | NESTED LOOPS | || 12 | PARTITION RANGE ALL | || 13 | BITMAP CONVERSION TO ROWIDS| || 14 | BITMAP AND | || 15 | BITMAP MERGE | || 16 | BITMAP KEY ITERATION | || 17 | BUFFER SORT | ||* 18 | TABLE ACCESS FULL | PRODUCTS ||* 19 | BITMAP INDEX RANGE SCAN| SALES_PROD_BIX || 20 | BITMAP MERGE | || 21 | BITMAP KEY ITERATION | || 22 | BUFFER SORT | || 23 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662B_84B3F4 ||* 24 | BITMAP INDEX RANGE SCAN| SALES_CUST_BIX || 25 | TABLE ACCESS BY USER ROWID | SALES |-----------------------------------------------------------------------
-------------------------------------------------------------------| Id | Operation | Name |-------------------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | TEMP TABLE TRANSFORMATION | || 2 | LOAD AS SELECT | SYS_TEMP_0FD9D662E_84B3F4 || 3 | VECTOR GROUP BY | || 4 | KEY VECTOR CREATE BUFFERED| :KV0000 ||* 5 | TABLE ACCESS FULL | PRODUCTS || 6 | LOAD AS SELECT | SYS_TEMP_0FD9D662F_84B3F4 || 7 | VECTOR GROUP BY | || 8 | KEY VECTOR CREATE BUFFERED| :KV0001 ||* 9 | TABLE ACCESS FULL | CUSTOMERS || 10 | HASH GROUP BY | ||* 11 | HASH JOIN | || 12 | MERGE JOIN CARTESIAN | || 13 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662E_84B3F4 || 14 | BUFFER SORT | || 15 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662F_84B3F4 || 16 | VIEW | VW_VT_83032D7B || 17 | VECTOR GROUP BY | || 18 | HASH GROUP BY | || 19 | KEY VECTOR USE | :KV0000 || 20 | KEY VECTOR USE | :KV0001 || 21 | PARTITION RANGE ALL | ||* 22 | TABLE ACCESS FULL | SALES |-------------------------------------------------------------------
Voraussetzungen
Star Schema & Oracle 12c In-Memory22 16.03.2016
Star Transformation Vector Transformation
Oracle DB Standard Edition 1)
Oracle DB Enterprise EditionOracle DB Enterprise Edition+ Oracle Database In-Memory Option
Bitmap Index (oder B-Tree Index)für jeden Dimensionsschlüssel auf Faktentabelle
Dimensions- und Faktentabellen in In-Memory Column Store empfohlen, aber nicht zwingend notwendig
star_transformation_enabled = true 2) inmemory_size ≥ 100MBcompatible ≥ 12.1.0.0.0
Hint /*+ star_transformation */ Hint /*+ vector_transform */
1) nur mit B-Tree Indizes (Bitmap Indizes in SE nicht vorhanden)2) oder star_transformation_enabled = temp_disable
16.03.2016 Star Schema & Oracle 12c In-Memory23
Star Transformation oder Vector Transformation?
Selectivity
Elap
sed
Tim
e
Empfehlungen
Star Schema & Oracle 12c In-Memory24
Wenn Oracle Database In-Memory Option vorhanden:
è Vector Transformation– Fakten- und Dimensionstabellen in IM Column Store laden
– Keine Indizes auf Faktentabelle notwendig
Wenn keine Oracle Database In-Memory Option vorhanden:
è Star Transformation– Bitmap Index auf jedem Dimensionsschlüssel der Faktentabelle
– star_transformation_enabled = true
16.03.2016
Quellen und weitere Informationen
Oracle Database Online Documentation 12c Release 1 (12.1)Database SQL Tuning Guide, Chapter 5: Query Transformationshttps://docs.oracle.com/database/121/TGSQL/tgsql_transform.htm#TGSQL95256
Oracle Database In-Memory: In-Memory AggregationOracle White Paper, January 2015 (William Endress)http://www.oracle.com/technetwork/database/bi-datawarehousing/inmemory-aggregation-twp-01282015-2412192.pdf
Oracle Database In-MemoryOracle White Paper, July 2015 (Maria Colgan)http://www.oracle.com/technetwork/database/in-memory/overview/twp-oracle-database-in-memory-2245633.html
Oracle Scratchpad, In-memory Aggregation, August 2014 (Jonathan Lewis)https://jonathanlewis.wordpress.com/2014/08/24/in-memory-aggregation/
Oracle Database In-Memory – A game-changer for Data Warehousing?Präsentation DOAG Konferenz, November 2014 (Maria Colgan, Hermann Baer)
Oracle Database In-Memory Option, Trivadis Work Bench, November 2014(Christian Antognini, Robert Bialek, Daniele Massimi, Peter Welker)
16.03.2016 Star Schema & Oracle 12c In-Memory25
Dani SchniderPrincipal Consultant
Tel. +41 58 459 50 [email protected]
16.03.2016 Star Schema & Oracle 12c In-Memory26
