Oracle Database 11g BI and Data Warehousing Student Guide

Oracle Database 11 g: BI and Data Warehousing

Student Guide

D52358GC10

Edition 1.0

October 2007

PRODUCTION

Copyright © 2007, Oracle. All rights reserved.

This documentation contains proprietary information of Oracle Corporation. It is provided under a license agreement containing restrictions on use and disclosure and is also protected by copyright law. Reverse engineering of the software is prohibited. If this documentation is delivered to a U.S. Government Agency of the Department of Defense, then it is delivered with Restricted Rights and the following legend is applicable:

Restricted Rights Legend

Use, duplication or disclosure by the Government is subject to restrictions for commercial computer software and shall be deemed to be Restricted Rights software under Federal law, as set forth in subparagraph (c)(1)(ii) of DFARS 252.227-7013, Rights in Technical Data and Computer Software (October 1988).

This material or any portion of it may not be copied in any form or by any means without the express prior written permission of the Education Products group of Oracle Corporation. Any other copying is a violation of copyright law and may result in civil and/or criminal penalties.

If this documentation is delivered to a U.S. Government Agency not within the Department of Defense, then it is delivered with “Restricted Rights,” as defined in FAR 52.227-14, Rights in Data-General, including Alternate III (June 1987).

The information in this document is subject to change without notice. If you find any problems in the documentation, please report them in writing to Worldwide Education Services, Oracle Corporation, 500 Oracle Parkway, Box SB-6, Redwood Shores, CA 94065. Oracle Corporation does not warrant that this document is error-free.

Oracle, JD Edwards, PeopleSoft, and Siebel are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

Author

Lauran Serhal, Jim Womack, Jean-Francois Verrier

Technical Contributors and Reviewers

John Haydu, Lei Sheng, Lilian Hobbs, Paul Lane, Diana Lorentz, Christine Jeal

This book was published using: oracle tutor


Oracle Database 11g: BI and Data Warehousing Table of Contents i

Table of Contents

Partitioning and Storage-Related Enhancements.........................................................................................1-2 Chapter 1Partitioning and Storage-Related Enhancements ...........................................................................1-2 Objectives......................................................................................................................................................1-3 Oracle Partitioning ........................................................................................................................................1-4 11g Partitioning Enhancements.....................................................................................................................1-5 Interval Partitioning.......................................................................................................................................1-6 Interval Partitioning: Example ......................................................................................................................1-7 Moving the Transition Point: Example .........................................................................................................1-8 System Partitioning .......................................................................................................................................1-9 System Partitioning: Example .......................................................................................................................1-10 System Partitioning: Guidelines ....................................................................................................................1-11 Virtual Column–Based Partitioning ..............................................................................................................1-12 Virtual Column–Based Partitioning: Example ..............................................................................................1-13 Reference Partitioning...................................................................................................................................1-14 Reference Partitioning: Benefit .....................................................................................................................1-15 Reference Partitioning: Example...................................................................................................................1-16 Composite Partitioning Enhancements..........................................................................................................1-18 Range-Range Partitioning: Example .............................................................................................................1-19 Table Compression: Overview......................................................................................................................1-20 Table Compression Concepts ........................................................................................................................1-21 Using Table Compression .............................................................................................................................1-22 Summary .......................................................................................................................................................1-23

Using the New and Updated Materialized Views (MV) Catalog Views......................................................2-2 Chapter 2Using the New and Updated Materialized Views (MV) Catalog Views .......................................2-2 Objectives......................................................................................................................................................2-3 The New and Updated MV Catalog Views ...................................................................................................2-4 PCT Catalog Views Showing Staleness Corresponding to Base Partitions ..................................................2-5 The New Columns in the USER/ALL/DBA_MVIEWS Catalog View .......................................................2-7 The USER/ALL/DBA_MVIEW_DETAIL_RELATIONS Catalog View Extension ...................................2-8 The New Catalog View: USER/ALL/DBA_MVIEW_DETAIL_PARTITION ...........................................2-9 The New Catalog View: USER/ALL/DBA_MVIEW_DETAIL_SUBPARTITION....................................2-10 Refresh Performance Improvements in Oracle Database 11g ......................................................................2-11 Summary .......................................................................................................................................................2-13

General Query Rewrite Enhancements.........................................................................................................3-2 Chapter 3General Query Rewrite Enhancements ..........................................................................................3-2 Objectives......................................................................................................................................................3-3 Using Summaries to Improve Performance...................................................................................................3-4 Summary Management..................................................................................................................................3-6 Summary Management Components ............................................................................................................3-7 Using Summary Management .......................................................................................................................3-8 Query Rewrite: Overview .............................................................................................................................3-9 Cost-Based Query Rewrite Process...............................................................................................................3-10 What Can Be Rewritten?...............................................................................................................................3-11 Query Rewrite Enhancement to Support Queries Containing Inline Views.................................................3-12 When Are Two Inline Views Equivalent?.....................................................................................................3-13 An MV Inline View Text Matches a Query's Inline View Text: Example...................................................3-14 An MV Inline View Text Is Equivalent to a Query's Inline View Text: Example .......................................3-15 Transforming and Rewriting the Query from the Two Previous Examples .................................................3-16 Inline View Rewrite Guidelines ....................................................................................................................3-17 Did Query Rewrite Occur?............................................................................................................................3-18 Query Rewrite Using Remote Tables in Oracle Database 11g.....................................................................3-19 Query Rewrite Using Remote Tables: Example............................................................................................3-20 Summary .......................................................................................................................................................3-21

Performing Pivoting and Unpivoting Operations.........................................................................................4-2


Oracle Database 11g: BI and Data Warehousing Table of Contents ii

Chapter 4Performing Pivoting and Unpivoting Operations ..........................................................................4-2 Objectives......................................................................................................................................................4-3 The Order Entry (OE) Schema Used in This Lesson ....................................................................................4-4 The Sales History (SH) Schema Used in This Lesson ..................................................................................4-5 The Sales History (SH) Schema Used in this eStudy....................................................................................4-6 The Benefits of Using Pivoting Operations...................................................................................................4-7 The PIVOT and UNPIVOT Clauses of the SELECT Statement..................................................................4-8 Pivoting on the QUARTER Column: Conceptual Example.........................................................................4-9 The PIVOT Clause Syntax ............................................................................................................................4-10 Creating a New View: Example ....................................................................................................................4-12 Selecting the SALES_VIEW Data ................................................................................................................4-14 Pivoting the QUARTER Column in the SH Schema: Example ...................................................................4-16 Pivoting on the ORDER_MODE Column in the OE Schema: Example......................................................4-18 Pivoting on Multiple Columns ......................................................................................................................4-19 Pivoting Using Multiple Aggregations..........................................................................................................4-21 Distinguishing PIVOT-Generated Nulls from Nulls in the Source Data......................................................4-22 Using the XML Keyword to Specify Pivot Values: Two Methods..............................................................4-24 Specifying Pivot Values: Using the ANY Keyword .....................................................................................4-25 Specifying Pivot Values: Using Subqueries..................................................................................................4-26 Unpivoting the QUARTER Column: Conceptual Example .........................................................................4-27 Using the UNPIVOT Operator......................................................................................................................4-28 Using the UNPIVOT Clause .........................................................................................................................4-29 The Data Types of the Value Columns in an UNPIVOT Operation ............................................................4-30 The UNPIVOT Clause Syntax ......................................................................................................................4-31 Creating a New Pivot Table: Example ..........................................................................................................4-32 Unpivoting the QUARTER Column in the SH Schema: Example ..............................................................4-33 Unpivoting the ORDER_MODE Column in the OE Schema: Example......................................................4-34 Unpivoting Multiple Columns in the SH Schema: Example........................................................................4-35 Unpivoting Multiple Aggregations in the SH Schema: Example.................................................................4-37 Summary .......................................................................................................................................................4-39


Partitioning and Storage-Related Enhancements Chapter 0 - Page 1

Partitioning and Storage-Related Enhancements



Chapter 1Partitioning and Storage-Related Enhancements

Partitioning and Storage-Related Enhancements



Objectives

Objectives

After completing this lesson, you should be able to :• Implement the new partitioning methods• Employ data compression



Oracle Partitioning

Interval PartitioningPartition Advisor

More composite choicesREF PartitioningVirtual Column Partitioning

Oracle Database 11 g

Fast drop tableMultidimensional pruning

1M partitions per tableOracle10 g R2

Local Index maintenance

Global hash indexesOracle10 g

Fast partition splitComposite range-list partitioning

Oracle9 i R2

Global index maintenance

List partitioningOracle9 i

Merge operationPartitionwise joinsDynamic pruning

Hash and composite range-hash partitioning

Oracle8 i

Basic maintenance operations: add, drop, exchange

Static partition pruning

Range partitioningGlobal range indexes

Oracle8

ManageabilityPerformanceCore functionality

Oracle Partitioning

The table above summarizes the 10 years of partitioning development at Oracle. Note: REF partitioning enables pruning and partitionwise joins against child tables. Though performance seems to be the most visible improvement, do not forget about the rest. Partitioning must address all business-relevant areas of performance, manageability, and availability.



11g Partitioning Enhancements

11g Partitioning Enhancements

• Interval partitioning• System partitioning• Composite partitioning enhancements• Virtual column-based partitioning• Reference partitioning

• Data Pump Enhancement:– Single Partition Transportable Tablespace

Partitioning allows the DBA to employ a “divide and conquer” methodology for managing database tables, especially as those tables grow. Partitioned tables allow a database to scale for very large datasets while maintaining consistent performance, without unduly impacting administrative or hardware resources. Partitioning enables faster data access within an Oracle database. Whether a database has 10 GB or 10 TB of data, partitioning can speed up data access by orders of magnitude. With the introduction of Oracle Database 11g, the DBA will find a useful assortment of partitioning enhancements. These enhancements include:

• Addition of interval partitioning

• Addition of system partitioning

• Composite partitioning enhancements

• Addition of virtual column-based partitioning

• Addition of reference partitioning

Data Pump Enhancement: You can now export one or more partitions of a table without having to move the entire table. On import, you can choose to load partitions as is, merge them into a single table, or promote each into a separate table. Please refer to the Oracle® Database Utilities 11g Release 1 (11.1) guide for more information on Data Pump.



Interval Partitioning

Interval Partitioning

• Interval partitioning is an extension of range partitioning.

• Partitions of a specified interval are created when inserted data exceeds all of the range partitions.

• At least one range partition must be created.• Interval partitioning automates the creation of ran ge

partitions.

Before the introduction of interval partitioning, the DBA was required to explicitly define the range of values for each partition. The problem is explicitly defining the bounds for each partition does not scale as the number of partitions grow. Interval partitioning is an extension of range partitioning, which instructs the database to automatically create partitions of a specified interval when data inserted into the table exceeds all of the range partitions. You must specify at least one range partition. The range partitioning key value determines the high value of the range partitions, which is called the transition point, and the database creates interval partitions for data beyond that transition point. Interval partitioning fully automates the creation of range partitions. Managing the creation of new partitions can be a cumbersome and highly repetitive task. This is especially true for predictable additions of partitions covering small ranges, such as adding new daily partitions. Interval partitioning automates this operation by creating partitions on demand. When using interval partitioning, consider the following restrictions:

• You can specify only one partitioning key column, and it must be of NUMBER or DATE type.

• Interval partitioning is not supported for index-organized tables.

• You cannot create a domain index on an interval-partitioned table.



Interval Partitioning: Example

Interval Partitioning: Example

CREATE TABLE SH.SALES_INTERVAL PARTITION BY RANGE (time_id)

INTERVAL (NUMTOYMINTERVAL(1,'month')) STORE IN (tbs1,t bs2,tbs3,tbs4)(

PARTITION P1 values less than (TO_DATE('1-1-2005',' dd-mm-yyyy')),PARTITION P2 values less than (TO_DATE('1-1-2006',' dd-mm-yyyy')),

PARTITION P3 values less than (TO_DATE('1-1-2007',' dd-mm-yyyy')))AS

SELECT *

FROM SH.SALESWHERE TIME_ID < TO_DATE('1-1-2007','dd-mm-yyyy');

P1 P2 P3 Pi1 Pin…

Range section Interval section

Transition point

Automatically createdwhen you insert data

Consider the example above, which illustrates the creation of an interval-partitioned table. The original CREATE TABLE statement specifies four partitions with varying widths. This portion of the table is range-partitioned. It also specifies that above the transition point of “1-1-2007,” partitions be created with a width of one month. These partitions are interval-partitioned. Partition Pi1 is automatically created using this information when a row with a value corresponding to January 2007 is inserted into the table. The high bound of partition P3 represents a transition point. P3 and all partitions below it (P1 and P2 in this example) are in the range section, whereas all partitions above it fall into the interval section. The only argument to the INTERVAL clause is a constant of the interval type. Currently, you can specify only one partitioning key column, and it must be of DATE or NUMBER type. You can use the optional STORE IN clause of the INTERVAL clause to specify one or more tablespaces into which the database will store interval partition data in a round-robin fashion.



Moving the Transition Point: Example

Moving the Transition Point: Example

< 01/01/07

PREVIOUS

< 01/08/07

SYS_Px

< 01/01/07

PREVIOUS

< 01/11/07

SYS_Py

< 01/12/07

SYS_Pz

< 01/03/08

SYS_Pt

< 01/08/07

SYS_Px

< 01/01/07

PREVIOUS

< 01/12/07

SYS_Pz

< 01/03/08

SYS_Pt

alter table orders_interval merge partitionsfor(TO_DATE('15-10-2007','dd-mm-yyyy')),for(TO_DATE ('15-11-2007','dd-mm-yyyy'))

into partition sys_p z;

Transition point

Transition point

Transition point

Not yetmaterialized

INSERT INTO orders_interval (…);

The graphic in the slide shows you a typical Information Lifecycle Management (ILM) scenario where after one year of automated partition creation, you merge the created partitions (SYS_Py and SYS_Pz in the example) to move the transition point. You can then move the resulting partitions to a different storage for ILM purposes. The example assumes that you created a table called ORDERS_INTERVAL that has initially one range partition called PREVIOUS, which holds orders from before 2007. The interval is defined to be one month. Then during the year 2007 and 2008, some orders are inserted, and it is assumed that four partitions are created. They are shown in the graphic. They are automatically named according to certain system rules. You then decide to merge the last two partitions of the year 2007 using the ALTER TABLE statement shown in the slide. You must merge two adjacent partitions. The new extended partition syntax is used to designate a partition without needing to know the partition name. The syntax uses an expression that must represent a possible value for the partition in question. This syntax works for all cases when you have to reference a partition, whether it is range, list, interval, or hash. It supports all operations such as drop, merge, split, and so on. As a result of your MERGE operation, you can see that the transition point moved. The bottom part of the graphic shows you the new range section that now contains three partitions. Note: You can change the interval of an interval-partitioned table; the existing intervals remain unaffected.



System Partitioning

System Partitioning

System partitioning:• Enables application-controlled partitioning for sel ected

tables• Provides the benefits of partitioning but the parti tioning

and data placement are controlled by the applicatio n• Does not employ partitioning keys like other

partitioning methods• Does not support partition pruning in the tradition al

sense

System partitioning enables application-controlled partitioning for arbitrary tables. This is mainly useful when you develop your own partitioned domain indexes. The database simply provides the ability to break down a table into meaningless partitions. All other aspects of partitioning are controlled by the application. System partitioning provides the well-known benefits of partitioning (scalability, availability, and manageability), but the partitioning and actual data placement are controlled by the application. The most fundamental difference between system partitioning and other methods is that system partitioning does not have any partitioning keys. Consequently, the distribution or mapping of the rows to a particular partition is not implicit. Instead, the user specifies the partition to which a row maps by using partition-extended syntax when inserting a row. Because system-partitioned tables do not have a partitioning key, the usual performance benefits of partitioned tables are not available for system-partitioned tables. Specifically, there is no support for traditional partition pruning, partitionwise joins, and so on. Partition pruning is achieved by accessing the same partitions in the system-partitioned tables as those that were accessed in the base table. System-partitioned tables provide the manageability advantages of equipartitioning. For example, a nested table can be created as a system-partitioned table that has the same number of partitions as the base table. A domain index can be backed up by a system-partitioned table that has the same number of partitions as the base table.



System Partitioning: Example

System Partitioning: Example

CREATE TABLE systab (c1 integer, c2 integer) PARTITION BY SYSTEM(

PARTITION p1 TABLESPACE tbs_1,PARTITION p2 TABLESPACE tbs_2,PARTITION p3 TABLESPACE tbs_3,PARTITION p4 TABLESPACE tbs_4

);

INSERT INTO systab PARTITION (p1) VALUES (4,5);

INSERT INTO systab PARTITION (p2) VALUES (150,2);

alter table systab merge partitions p1,p2 into part ition p1;

The syntax in the slide example creates a table with four partitions. Each partition can have different physical attributes. INSERT and MERGE statements must use partition-extended syntax to identify a particular partition that a row should go into. For example, the value (4,5) can be inserted into any one of the four partitions given in the example. Deletes and updates do not require the partition-extended syntax. However, because there is no partition pruning, if the partition-extended syntax is omitted, the entire table is scanned to execute the operation. Again, this example highlights the fact that there is no implicit mapping from rows to any partition.



System Partitioning: Guidelines

System Partitioning: Guidelines

The following operations are supported for system-partitioned tables:• Partition maintenance operations and other data

definition language (DDL) operations• Creation of local indexes• Creation of local bitmapped indexes• Creation of global indexes• All DML operations• INSERT. . . SELECT with partition-extended syntax:

INSERT INTO < table_name> PARTITION(< partition-name>) < subquery>

The following operations are supported for system-partitioned tables:

• Partition maintenance operations and other DDLs (see exceptions below)

• Creation of local indexes

• Creation of local bitmapped indexes

• Creation of global indexes

• All data manipulation language (DML) operations

• INSERT. . .SELECT with partition-extended syntax.

Because of the peculiar requirements of system partitioning, the following operations are not supported for system partitioning:

• Unique local indexes are not supported because they require a partitioning key.

• CREATE TABLE AS SELECT is not supported because there is no partitioning method. It is not possible to distribute rows to partitions. Instead, you should first create the table and then insert rows into each partition.

• SPLIT PARTITION operations



Virtual Column–Based Partitioning

Virtual Column–Based Partitioning

• Virtual column values are derived by the evaluation of a function or expression.

• Virtual columns can be defined within a CREATEor ALTERtable operation.

•

• Virtual column values are not physically stored in the table row on disk, but are evaluated on demand.

• Virtual columns can be indexed, and used in queries , DML, and DDL statements like other table column typ es.

• Tables and indexes can be partitioned on a virtual column and statistics can be gathered upon them.

CREATE TABLE employees (employee_id number(6) not null,

…total_compensation as (salary *( 1+commission_pct) )

Columns of a table whose values are derived by computation of a function or an expression are known as virtual columns. These columns can be specified during a CREATE or ALTER table operation. Virtual columns share the same SQL namespace as other real table columns and conform to the data type of the underlying expression that describes it. These columns can be used in queries like any other table columns, thereby providing a simple, elegant, and consistent mechanism of accessing expressions in a SQL statement. The values for virtual columns are not physically stored in the table row on disk, rather they are evaluated on demand. The functions or expressions describing the virtual columns should be deterministic and pure, meaning the same set of input values should return the same output values. Virtual columns can be used like any other table columns. They can be indexed, and used in queries, DML, and DDL statements. Tables and indexes can be partitioned on a virtual column and statistics can be gathered upon them. You can use virtual column partitioning to partition key columns defined on virtual columns of a table. Frequently, business requirements to logically partition objects do not match existing columns in a one-to-one manner. With the introduction of Oracle Database 11g, partitioning has been enhanced to allow a partitioning strategy defined on virtual columns, thus enabling a more comprehensive match of the business requirements.



Virtual Column–Based Partitioning: Example

Virtual Column–Based Partitioning: Example

CREATE TABLE employees (employee_id number(6) not null, first_name varchar 2(30),

last_name varchar2(40) not null, emailvarchar2(25), phone_number varchar2(20), hire_date date not null, job_id varchar2(10) not null, salary number(8,2), commission_pct number(2,2), manager_id number(6), department_id number(4),total_compensation as (salary *( 1+commission_pct) ))

PARTITION BY RANGE ( total_compensation )(

PARTITION p1 VALUES LESS THAN (50000),PARTITION p2 VALUES LESS THAN (100000),PARTITION p3 VALUES LESS THAN (150000),PARTITION p4 VALUES LESS THAN (MAXVALUE)

);

Consider the example in the slide. The EMPLOYEES table is created using the standard CREATE TABLE syntax. The total_compensation column is a virtual column calculated by multiplying the value of salary by the commission_pct plus one. The next statement declares total_compensation (a virtual column) to be the partitioning key of the EMPLOYEES table. Partition pruning takes place for virtual column partition keys when the predicates on the partitioning key are of the following types:

• Equality or Like

• List

• Range

• Partition-extended names

Given a join operation between two tables, the optimizer recognizes when a partitionwise join (full or partial) is applicable, decides whether to use it or not, and annotates the join properly when it decides to use it. This applies to both serial and parallel cases. To recognize full partitionwise joins, the optimizer relies on the definition of equipartitioning of two objects; this definition includes the equivalence of the virtual expression on which the tables were partitioned.



Reference Partitioning

Reference Partitioning

• A table can now be partitioned based on the partiti oning method of a table referenced in its referential con straint.

• The partitioning key is resolved through an existin g parent/child relationship.

• The partitioning key is enforced by active primary key or foreign key constraints.

• Tables with a parent/child relationship can be equipartitioned by inheriting the partitioning key from the parent table without duplicating the key column s.

• Partitions are automatically maintained.

Reference partitioning provides the ability to partition a table based on the partitioning scheme of the table referenced in its referential constraint. The partitioning key is resolved through an existing parent/child relationship, which is enforced by active primary key or foreign key constraints. The benefit of this is that tables with a parent/child relationship can be logically equipartitioned by inheriting the partitioning key from the parent table without duplicating the key columns. The logical dependency also automatically cascades partition maintenance operations, making application development easier and less error prone.



Reference Partitioning: Benefit

Reference Partitioning: Benefit

… …

… …

Range (ORDER_DATE)Primary key ( ORDER_ID)

Range (ORDER_DATE)Foreign key ( ORDER_ID)

Redundant storage/maintenance of ORDER_DATE

Table ORDERS

Table ORDER_ITEMS

Partition key inherited through PK/FK relationship

Without using reference partitioning Reference partitioning

As illustrated in the slide, you can see the benefit of using reference partitioning. The left part of the graphic shows you the situation where you have two tables, ORDERS and ORDER_ITEMS, that are equipartitioned on the ORDER_DATE column. In that case, both tables need to define the ORDER_DATE column. However, defining ORDER_DATE in the ORDER_ITEMS table is redundant because there is a primary key/foreign key relationship between the two tables. The right part of the graphic shows you the situation where you use reference partitioning. This time, you no longer need to define the ORDER_DATE column in the ORDER_ITEMS table. The partition key of the ORDER_ITEMS table is automatically inherited from the primary key/foreign key relationship that exists. When used for pruning and partitionwise joins, reference partitioning has the benefit that query predicates can be different and partitionwise joins still work—for example, partitioning on ORDER_DATE and search on ORDER_ID. With previous releases, both partitioning and predicates had to be identical for a partitionwise join to work. Note: This partitioning method can be useful for nested table partitioning.



Reference Partitioning: Example

Reference Partitioning: Example

CREATE TABLE orders

( order_id NUMBER(12) , order_date DATE,order_mode VARCHAR2(8), customer_id NUMBER(6),order_status NUMBER(2) , order_total NUMBER(8,2),sales_rep_id NUMBER(6) , promotion_id NUMBER(6),CONSTRAINT orders_pk PRIMARY KEY(order_id)

)PARTITION BY RANGE(order_date)(PARTITION Q105 VALUES LESS THAN (TO_DATE('1-4-2005 ','DD-MM-YYYY')),

PARTITION Q205 VALUES LESS THAN (TO_DATE('1-7-2005' ,'DD-MM-YYYY')),PARTITION Q305 VALUES LESS THAN (TO_DATE('1-10-2005 ','DD-MM-YYYY')),PARTITION Q405 VALUES LESS THAN (TO_DATE('1-1-2006' ,'DD-MM-YYYY')));

CREATE TABLE order_items( order_id NUMBER(12) NOT NULL, line_item_id NU MBER(3) NOT NULL,

product_id NUMBER(6) NOT NULL, unit_price NUMB ER(8,2),quantity NUMBER(8),CONSTRAINT order_items_fk

FOREIGN KEY(order_id) REFERENCES orders(order_id)) PARTITION BY REFERENCE(order_items_fk) ;

The example above creates the following two tables:

ORDERS: Range-partitioned table partitioned on order_date . It is created with four partitions, Q105, Q205, Q305, and Q405.

ORDER_ITEMS: Reference-partitioned child table:

• This table is created with four partitions—Q105, Q205, Q305, and Q405—with each containing rows corresponding to ORDERS in the respective parent partition.

• If partition descriptors are provided, the number of partitions described must be exactly equal to the number of partitions or subpartitions in the referenced table.

• If the parent table is a composite-partitioned table, then the table will have one partition for each subpartition of its parent.

• Partition bounds cannot be specified for the partitions of a reference-partitioned table. The partitions of a reference-partitioned table can be named unless there is a conflict with inherited names. In this case, the partition will have a system-generated name.

• Partitions of a reference-partitioned table will collocate with the corresponding partition of the parent table, if no explicit tablespace is specified. As with other partitioned tables, you can specify object-level default attributes, and partition descriptors that override object-level defaults.

• It is not possible to disable the foreign key constraint of a reference-partitioned table.



• It is not permitted to add or drop partitions of a reference-partitioned table. However, performing partition maintenance operations on the parent table is automatically cascaded to the child table.



Composite Partitioning Enhancements

Composite Partitioning Enhancements

• Range top level– Range-Range

• List top level– List-List– List-Hash– List-Range

• Interval top level– Interval-Range– Interval-List– Interval-Hash

RANGE, LIST, INTERVAL

SP1

SP2

SP3

SP4

…

…

…

…

LIST, RANGE, HASH

SP1 SP1 SP1 SP1

SP2 SP2 SP2 SP2

SP3 SP3 SP3 SP3

SP4 SP4 SP4 SP4

Before the release of Oracle Database 11g, the only composite partitioning methods supported were range-list and range-hash. With this new release, list partitioning can be a top-level partitioning method for composite partitioned tables giving us list-list, list-hash, list-range, and range-range composite methods. With the introduction of interval partitioning, interval-range, interval-list, and interval-hash are now supported composite partitioning methods.

Range-Range Partitioning Composite range-range partitioning enables logical range partitioning along two dimensions; for example, range partition by order_date and range subpartition by shipping_date .

List-Range Partitioning Composite list-range partitioning enables logical range subpartitioning within a given list partitioning strategy; for example, list partition by country_id and range subpartition by order_date .

List-Hash Partitioning Composite list-hash partitioning enables hash subpartitioning of a list-partitioned object; for example, to enable partitionwise joins.

List-List Partitioning Composite list-list partitioning enables logical list partitioning along two dimensions; for example, list partition by country_id and list subpartition by sales_channel .



Range-Range Partitioning: Example

Range-Range Partitioning: Example

CREATE TABLE sales (prod_id NUMBER(6) NOT NULL, cust_id NUMBER NOT NULL ,time_id DATE NOT NULL, channel_id char(1) NOT NULL,promo_id NUMBER (6) NOT NULL,quantity_sold NUMBER(3) NOT NULL,amount_sold NUMBER(10,2) NOT NULL )

PARTITION BY RANGE (time_id)SUBPARTITION BY RANGE (cust_id)SUBPARTITION TEMPLATE

( SUBPARTITION sp1 VALUES LESS THAN (50000),SUBPARTITION sp2 VALUES LESS THAN (100000),SUBPARTITION sp3 VALUES LESS THAN (150000),SUBPARTITION sp4 VALUES LESS THAN (MAXVALUE) )

(PARTITION VALUES LESS THAN (TO_DATE('1-4-2007','DD- MM-YYYY')),PARTITION VALUES LESS THAN (TO_DATE('1-7-2007','DD- MM-YYYY')),PARTITION VALUES LESS THAN (TO_DATE('1-8-2007','DD- MM-YYYY')),PARTITION VALUES LESS THAN (TO_DATE('1-1-2008','DD- MM-YYYY'))

);

Composite range-range partitioning enables logical range partitioning along two dimensions. In the example in the slide, the SALES table is created and range-partitioned on time_id . Using a subpartition template, the SALES table is subpartitioned by range using cust_id as the subpartition key. Because of the template, all partitions have the same number of subpartitions with the same bounds as defined by the template. If no template is specified, a single default partition bound by MAXVALUE (range) or DEFAULT value (list) is created. Although the example illustrates the range-range methodology, the other new composite partitioning methods use similar syntax and statement structure. All of the composite partitioning methods fully support the existing partition pruning methods for queries involving predicates on the subpartitioning key.



Table Compression: Overview

Table Compression: Overview

• Oracle Database 11 g extends compression for OLTP data.– Support for conventional DML operations

(INSERT, UPDATE, DELETE)

• New algorithm significantly reduces write overhead.– Batched compression ensures no impact for most OLTP

transactions.

• There is no impact on reads.– Reads may actually see improved performance due to

fewer I/Os and enhanced memory efficiency.

The Oracle database was the pioneer in terms of compression technology for databases with the introduction of table compression for bulk load operations in Oracle9i. Using this feature, you could compress data at the time of performing bulk load using operations such as direct loads, or Create Table As Select (CTAS). However, until now, compression was not available for regular data manipulation operations such as INSERT, UPDATE, and DELETE. Oracle Database 11g extends the compression technology to support these operations as well. Consequently, compression in Oracle Database 11g can be used for all kinds of workload, be it online transaction processing (OLTP) or data warehousing. It is important to mention that table compression enhancements introduced in Oracle Database 11g are not just incremental changes. An enormous amount of work has gone into making sure that the new compression technology has negligible impact on updates because any noticeable write-time penalty due to compression will not be acceptable in an OLTP environment. As a result, compression technology in Oracle Database 11g is very efficient and could reduce the space consumption by 50–75%. And while you do that, not only your write performance does not degrade, but also your read performance or queries improve. This is because unlike desktop-based compression techniques where you have to wait for data to be uncompressed, Oracle technology reads the compressed data (less fetches needed) directly and does not require any uncompress operation. Note: Compression technology is completely application transparent. This means that you can use this technology with any homegrown or packaged application such as SAP, Siebel, EBS, and so on.



Table Compression Concepts

Table Compression Concepts

Header

Unc

ompr

esse

dda

ta

Compresseddata

Freespace Inserts are

uncompressed.

PCTFREEreached,compression triggered

Inserts are againuncompressed.

PCTFREEreached,compression triggered

Data block

PCTFREElimit

The slide shows you a data block evolution when that block is part of a compressed table. You should read it from left to right. At the start, the block is empty and available for inserts. When you start inserting into this block, data is stored in an uncompressed format (like for uncompressed tables). However, as soon as you reach the PCTFREE of that block, the data is automatically compressed, potentially reducing the space it originally occupied. This allows for new uncompressed inserts to take place in the same block, until PCTFREE is reached again. At that point compression is triggered again to reduce space occupation in the block. Note: Compression eliminates holes created due to deletions, and maximizes contiguous free space in blocks.



Using Table Compression

Using Table Compression

• Requires database compatibility level at 11.1 or gr eater• New syntax extends the COMPRESSkeyword:

– COMPRESS [FOR { ALL | DIRECT_LOAD} OPERATIONS]

– FOR DIRECT_LOADis the default: Refers to bulk load operations from prior releases

– FOR ALL OPERATIONS: OLTP + direct loads

• Enable compression for new tables:

• Enable compression on existing table:

– Does not trigger compression on existing rows

CREATE TABLE t1 COMPRESS FOR ALL OPERATIONS;

ALTER TABLE t2 COMPRESS FOR ALL OPERATIONS;

To use the new compression algorithm, you must flag your table with the COMPRESS FOR ALL OPERATIONS clause. You can do so at table creation, or after creation. This is illustrated in the examples given above. If you use the COMPRESS clause without specifying any FOR option, or if you use the COMPRESS FOR DIRECT_LOAD OPERATIONS clause, you fall back to the old compression mechanism that was available in earlier releases. You can also enable compression at the partition or tablespace level. For example, you can use the DEFAULT storage clause of the CREATE TABLESPACE command to optionally specify a COMPRESS FOR clause. Note: You can view compression flags for your tables using the COMPRESS and COMPRESS_FOR columns in views such as DBA_TABLES and DBA_TAB_PARTITIONS.



Summary

Summary

In this lesson, you should have learned how to:• Implement the new partitioning methods• Employ data compression




Using the New and Updated Materialized Views (MV) Catalog Views Chapter 0 - Page 1

Using the New and Updated Materialized Views (MV) Catalog Views



Chapter 2Using the New and Updated Materialized Views (MV) Catalog Views

Using the New and Updated Materialized Views (MV) Catalog Views



Objectives

Objectives

After completing this lesson, you should be able to :• Identify the benefits of the new and updated MV

catalog views• List the new and updated MV catalog views• Describe the new and updated columns in the MV

catalog views• Identify the refresh performance enhancements



The New and Updated MV Catalog Views

The New and Updated MV Catalog Views

• New catalog views display the partition change tracking (PCT) information for a given materialized view.

• The new catalog views display which sections of the materialized views data are fresh or stale.

• This is a critical piece of information for the use r to be able to view the partition staleness information of the materialized view.

• It affects the usability and maintainability of the materialized view.

Note: For additional information about materialized views and partition change tracking, see the Oracle Database 10g: Implement and Administer a Data Warehouse Instructor-Led Training or the Oracle Database Data Warehousing Guide 11g Release 1 (11.1) documentation.



PCT Catalog Views Showing Staleness Corresponding to Base Partitions

PCTCatalog Views Showing Staleness Corresponding to Base Partitions

USER/ALL/DBA_MVIEW_DETAIL_RELATIONSviews (UPDATED)

USER/ALL/DBA_MVIEW_DETAIL_SUBPARTITION views (NEW)

USER/ALL/DBA_MVIEW_DETAIL_PARTITION views (NEW)

USER/ALL/DBA_MVIEWS views (UPDATED)

Partition Change Tracking (PCT) was introduced in Oracle Database 9i but its related information is not yet exposed to the user through catalog views. Therefore, users are unable to see this valuable information, which could help them make better decisions as to how their materialized views should be maintained. Oracle Database 11g exposes the MV freshness information corresponding to PCT base table partitions. This provides users with the information regarding which ranges of the MV data are fresh and which are not. The fresh part of MV is reliable and available for use. The USER/ALL/DBA_MVIEWS catalog view is extended where new columns are added to describe number of PCT tables, number of fresh and stale PCT regions. The USER/ALL/DBA_DETAIL_RELATIONS catalog view is extended where new columns are added to indicate if the detail table is PCT-enabled, and to show numbers of fresh and stale PCT partitions. A new catalog view for PCT partition (USER/ALL/DBA_MVIEW_DETAIL_PARTITION) is created to describe the freshness of each PCT partition. Create a new catalog view for PCT subpartition (USER/ALL/DBA_MVIEW_DETAIL_SUBPARTITION) to describe the freshness of each PCT subpartition. The USER/ALL/DBA_MVIEWS extension describes all materialized views in the database. The USER/ALL/DBA_MVIEW_DETAIL_RELATIONS extension represents the named detail



relations that are either in the FROM list of a materialized view or that are indirectly referenced through views in the FROM list. The new USER/ALL/DBA_MVIEW_DETAIL_PARTITION displays the freshness information of the materialized views, with respect to a PCT detail partition. The new USER/ALL/DBA_MVIEW_DETAIL_SUBPARTITION displays freshness information for all materialized views in the database, with respect to a PCT detail subpartition.



The New Columns in the USER/ALL/DBA_MVIEWS Catalog View

The New Columns in the USER/ALL/DBA_MVIEWSCatalog View

The name of the materialzed viewMVIEW_NAME

The name of the owner of the materialized viewOWNER

. . .. . .

Specifies the number of Stale PCT

partition regionsNUM_STALE_PCT_REGIONS (NEW)

Specifies the number of PCTdetail tablesNUM_PCT_TABLES (NEW)

NUM_FRESH_PCT_REGIONS (NEW)

Catalog View Column

Specifies the number of Fresh PCT

partition regions

Column Description

The syntax shown above is the extended specification for the DBA/ALL/USER_MVIEWS catalog view. This catalog view is extended to show how many detail partitions support PCT. In addition, two extended columns show how many fresh and stale PCT regions are present in that MV. The ALL_MVIEWS describes all materialized views accessible to the current user. The DBA_MVIEWS describes all materialized views in the database. The USER_MVIEWS describes all materialized views owned by the current user. Note: For the complete columns listing for the DBA/ALL/USER_MVIEWS catalog views, see the Oracle Database Reference 11g Release 1 (11.1) guide.



The USER/ALL/DBA_MVIEW_DETAIL_RELATIONS Catalog View Extension

The USER/ALL/DBA_MVIEW_DETAIL_RELATIONS Catalog View Extension

The number of Fresh PCTpartitionsNUM_FRESH_PCT_PARTITIONS(NEW)

Is the detail object PCTsupported? (Y or N )DETAILOBJ_PCT (NEW)

Implicit or explicit alias for detail relationDETAILOBJ_ALIAS

TABLE, VIEW, SNAPSHOT, CONTAINER, or UNDEFINEDDETAILOBJ_TYPE

The detail object name (table or view)DETAILOBJ_NAME

The name of the owner of the detail objectDETAILOBJ_OWNER

The number of Stale PCT partition regionsDETAILOBJ_PCT (NEW)


MVIEW_NAME

Catalog View Columns

The name of the materialzed view

Column Description

The DBA/ALL/USER_ MVIEW_DETAIL_RELATIONS catalog views are extended to show whether the detail partition supports PCT with respect to a given MV. If the detail partition does support PCT, the catalog views display how many fresh and stale PCT partitions are present in that detail table. The columns shown above in red are the new columns that were added to the DBA/ALL/USER_ MVIEW_DETAIL_RELATIONS catalog views. The ALL_MVIEW_DETAIL_RELATIONS describes the named detail relations that are either specified in the FROM list of the subquery that defines a materialized view accessible to the current user or that are indirectly referenced through views in that FROM list. Inline views in the materialized view definition are not represented in this view or the related views.



The New Catalog View: USER/ALL/DBA_MVIEW_DETAIL_PARTITION

The New Catalog View: USER/ALL/DBA_MVIEW_DETAIL_PARTITION

The freshness state: FRESH, STALE, UNKNOWN, NA, and so on

FRESHNESS

The position of the detail object partitionDETAIL_PARTITION_POSITION

The name of the detail object partitionDETAIL_PARTITION_NAME




MVIEW_NAME



Column Description

The new DBA/ALL/USER_MVIEW_DETAIL_RELATIONS catalog views shown above display the freshness information of the materialized views, with respect to a PCT detail partition, accessible to the current user. The DBA_MVIEW_DETAIL_PARTITION displays freshness information for all materialized views in the database, with respect to a PCT detail partition. The USER_MVIEW_DETAIL_PARTITION displays freshness information for all materialized views, with respect to a PCT detail partition, owned by the current user.



The New Catalog View: USER/ALL/DBA_MVIEW_DETAIL_SUBPARTITION

The New Catalog View: USER/ALL/DBA_MVIEW_DETAIL_SUBPARTITION

The name of the detail object subpartitionDETAIL_SUBPARTITION_NAME

The freshness state: FRESH, STALE, UNKNOWN, NA, and so on

FRESHNESS

The position of the detail object partitionDETAIL_SUBPARTITION_POSITION

The name of the detail object partitionDETAIL_PARTITION_NAME




MVIEW_NAME



Column Description

The ALL_MVIEW_DETAIL_SUBPARTITION displays the freshness information of the materialized views, with respect to a PCT detail subpartition, accessible to the current user. The DBA_MVIEW_DETAIL_SUBPARTITION displays freshness information for all materialized views in the database, with respect to a PCT detail subpartition. The USER_MVIEW_DETAIL_SUBPARTITION displays freshness information for all materialized views, with respect to a PCT detail subpartition, owned by the current user.



Refresh Performance Improvements in Oracle Database 11g

Refresh Performance Improvements in Oracle Database 11 g

Refresh operations on materialized views are now fa ster with the following improvements:• Refresh statement combinations (merge and delete)• Removal of unnecessary refresh hint• Index creation for UNION ALL MV

• PCTrefresh possible for UNION ALL MV

The refresh performance improvements reduce the time required to refresh materialized views. Query Rewrite Possible Using the Materialized View During Its Atomic Refresh Currently, when the materialized view is being refreshed, it is implicitly disabled for query rewrite even if its data is acceptable for the user. This is especially true when the atomic refresh is in progress and the user will see the data in the materialized view in a transactional state of the past refresh. In 11g, we plan to address the issue for the atomic refresh so that when the materialized view is refreshed in the atomic mode, it is eligible for query rewrite if the rewrite integrity mode is set to “STALE_TOLERATED”. Partition-Based Refresh for UNION ALL MVs In Oracle Database 9i Release 2, the fast refresh was extended to support the MV that has UNION ALL operators. However, the fast refresh of the UNION ALL MV did not apply partition-based (PCT) refresh. Using the PCT refresh can be beneficial during the refresh of UNION ALL MV . In Oracle Database 11g, PCT refresh is now allowed for UNION ALL MV fast refresh. Automatically Create Proper Index on MV (UNION ALL MV )

Currently, the materialized view with UNION ALL operators is fast refreshable but unlike most of the fast refreshable MVs, it is not created with an index. To speed up the refresh execution, the user needs to manually create such an index. In Oracle Database 11g, an index is created automatically during MV creation. Note: For information about refreshing materialized views, see the Oracle Database 10g:



Implement and Administer a Data Warehouse Instructor-Led Training or the Oracle Database Data Warehousing Guide 11g Release 1 (11.1) documentation.



Summary

Summary

In this lesson, you should have learned how to:• Identify the benefits of the new and updated MV

catalog views• List the new and updated MV catalog views• Describe the new and updated columns in the MV

catalog views• Identify the refresh performance enhancements




General Query Rewrite Enhancements Chapter 0 - Page 1

General Query Rewrite Enhancements



Chapter 3General Query Rewrite Enhancements

General Query Rewrite Enhancements



Objectives

Objectives

After completing this lesson, you should be able to :• Review the benefits of using materialized views• Review the benefits of query rewrite• Use the query rewrite enhancement to support querie s

with inline views• Identify the query rewrite enhancement that support s

queries with remote tables



Using Summaries to Improve Performance

Using Summaries to Improve Performance

• Special types of aggregate views• Improved query execution time through

precomputing of expensive joins and aggregation operations before execution and storing of results in a database table

• Created using a schema object called a materialized view

Summaries are aggregate views that are created to improve query execution times. When you use a summary, the results of joins and aggregate operations are calculated before query execution and stored in a database table. In an Oracle database, summaries are implemented with a materialized view. There are many well-known techniques you can use to increase query performance. For example, you can create additional indexes, or you can partition your data. Many data warehouses are also using a technique called summaries. The basic process for a summary is to precompute the result of a long-running query and store this result in a database table called a summary table, which is comparable to a CREATE TABLE AS SELECT (CTAS) statement. Instead of precomputing the same query result many times, the user can directly access the summary table. Although this approach has the benefit of enhancing query response time, it also has many drawbacks. The user needs to be aware of the summary table’s existence in order to rewrite the query to use that table instead. Also, the data contained in a summary table is frozen, and must be manually refreshed whenever modifications occur on the real tables. With Oracle Database summary management, the end user no longer has to be aware of the summaries that have been defined. The DBA creates materialized views that are automatically used by the system when rewriting SQL queries. Using materialized views offers another advantage over manually creating summaries tables, in that the data can be refreshed automatically. Note: The term summary comes from the fact that most of the time users in data warehouse



environments are computing expensive joins with aggregations. When creating summaries using materialized views in the Oracle Database, you are not required to use joins or aggregations.



Summary Management

Summary Management

DBA creates materialized view (summary table).

End user queries tables and views.

Oracle server rewrites SQL query to use summary tables.

In a typical use of summary management, the database administrator creates the materialized view (summary table). When the end user queries tables and views, the Oracle server query rewrite mechanism automatically rewrites the SQL query to use the summary table. The use of the materialized view is transparent to the end user or application querying the data.



Summary Management Components

Summary Management Components

• Mechanisms to define materialized views and dimensions

• Refresh mechanism to ensure materialized views contain the latest data

• Query rewrite capability to transparently rewrite a query to use a materialized view

• SQL Access Advisor: Recommends materialized views and indexes to be created

• DBMS_ADVISOR.TUNE_MVIEWprocedure: Shows you how to make your materialized view fast refreshable and use general query rewrite

The implementation of summary management in Oracle Database includes the use of the components listed above.



Using Summary Management

Using Summary Management

1. Use the SQL Access Advisor to determine how you will use materialized views.

2. Create materialized views and design how queries will be rewritten.

3. Use DBMS_ADVISOR.TUNE_MVIEWto obtain an optimized materialized view as necessary.

After your data has been transformed, staged, and loaded into the detail tables, you invoke the summary management process as described above.



Query Rewrite: Overview

• Tries to use materialized views instead of base tables to return query results

• Can save orders of magnitude of CPU and elapsed time to return results because queries are precomputed

• Queries rewritten even if not in the exact form of the MV

• Various requirements for query rewrite to take place

Query Rewrite: Overview



Cost-Based Query Rewrite Process

Cost-Based Query Rewrite Process

Query rewrite Generate plan

Compare plan costs

Generate plan Pick best plan

End user queries tables and views.

Query rewrite is available only with the cost-based optimizer. Oracle optimizes the input query with and without rewrite and selects the least costly alternative. The optimizer rewrites a query by rewriting one or more query blocks, one at a time. If the rewrite logic has a choice between multiple materialized views to rewrite a query block, it selects the one that can result in reading the least amount of data. After a materialized view has been picked for a rewrite, the optimizer performs the rewrite and then tests whether the rewritten query can be rewritten further with another materialized view (this could be the case only when nested materialized views exist). This process continues until no further rewrites are possible. Query rewrite is attempted recursively to take advantage of nested materialized views.



What Can Be Rewritten?

What Can Be Rewritten?

• Queries and subqueries in the following types of SQL statements:– SELECT

– CREATE TABLE … AS SELECT

– INSERT INTO … SELECT

• Subqueries in DML statements:— INSERT

— UPDATE

— DELETE

• Subqueries in the set operators:— UNION

— UNION ALL

— INTERSECT

— MINUS

Query rewrite operates on queries and subqueries in the following types of SQL statements:

• SELECT

• CREATE TABLE … AS SELECT

• INSERT INTO … SELECT

Query rewrite operates on subqueries in DML statements: INSERT, DELETE, and UPDATE. It also operates on subqueries in set operators: UNION, UNION ALL, INTERSECT, and MINUS.



Query Rewrite Enhancement to Support Queries Containing Inline Views

Query Rewrite Enhancement to Support Queries Containing Inline Views

Query with inline view

Query inline view ’s (IV) text matches the MV ’s IV text?

Rewrite query

Yes

Query inline view ’s (IV) text equivalent to the MV ’s IV text?

No query rewrite

Yes

No

No

Oracle Database 10g supported general query rewrite when the user query contained an inline view, or a subquery in the FROM list. Query rewrite matched inline views in the materialized view with inline views in the request query when the text of the two inline views exactly match. In that case, Oracle Database 10g treated the matching inline view as it would a named view, and general rewrite processing was possible. The query rewrite in Oracle Database 11g supports queries containing inline views. More queries are now eligible for query rewrite thus improving system throughput and performance. Oracle Database 11g supports query rewrite with inline views in two circumstances:

• When the text from the inline views in the materialized view exactly matches the text in the request query, or

• When the request query contains inline views that are equivalent to the inline views in the materialized view



When Are Two Inline Views Equivalent?

When Are Two Inline Views Equivalent?

Two inline views are considered equivalent if: • The SELECTlists and GROUP BYlists are equivalent• The FROMclauses contain the same or equivalent

objects• The join graphs, including all the selections in th e

WHEREclauses are equivalent• The HAVINGclauses are equivalent



An MV Inline View Text Matches a Query’s Inline View Text: Example

An MV Inline View Text Matches a Query’s Inline View Text: Example

CREATE MATERIALIZED VIEW SUM_SALES_MVENABLE QUERY REWRITE ASSELECT mv_iv.prod_id, mv_iv.cust_id,sum(mv_iv.amount_sold) sum_amount_soldFROM (SELECT products.prod_id, cust_id, amount_sold

FROM sales, productsWHERE sales.prod_id = products.prod_id) MV_IV

GROUP BY mv_iv.prod_id, mv_iv.cust_id;

-- The text of the IV matches exactly the text of th e-- MV; therefore, the query is rewritten with the MVSELECT iv.prod_id, iv.cust_id,SUM(iv.amount_sold) sum_amount_soldFROM (SELECT products.prod_id, cust_id, amount_sold

FROM sales, productsWHERE sales.prod_id = products.prod.id) IV

GROUP BY iv.prod_id, iv.cust_id;

The first example shown above, displays a materialized view that contains an inline view. The query in the second example shown above has an inline view whose text matches exactly with that of the materialized view's inline view; therefore, the query inline view will be internally replaced with the materialized view's inline view so that the query can be rewritten.



An MV Inline View Text Is Equivalent to a Query’s Inline View Text: Example

An MV Inline View Text Is Equivalent to a Query’s Inline View Text: Example

CREATE MATERIALIZED VIEW SUM_SALES_MVENABLE QUERY REWRITE ASSELECT mv_iv.prod_id, mv_iv.cust_id,sum(mv_iv.amount_sold) sum_amount_soldFROM (SELECT products.prod_id, cust_id, amount_sold

FROM sales, productsWHERE sales.prod_id = products.prod_id) MV_IV

GROUP BY mv_iv.prod_id, mv_iv.cust_id;

-- The text of the IV doesn’t match the text of the MV;-- however, they are equivalentSELECT iv.prod_id, iv.cust_id,SUM(iv.amount_sold) sum_amount_soldFROM (SELECT products.prod_id, cust_id, amount_sold

FROM products, salesWHERE sales.prod_id = products.prod_id) IV


The first example shown above, displays a materialized view that contains an inline view. The second example shown above, displays a query that has an equivalent inline view to the inline view found in the materialized view example above. Note that their texts do not match. It gets the same object number and rewrite takes place. The first example shown above has an inline view that does not have exact text match with the inline view in the preceding materialized view. Note that the join predicate in the query inline view is switched. Even though this query does not textually match with that of the materialized view's inline view, query rewrite will identify the query's inline view as equivalent to the materialized view's inline view. As before, the query inline view will be internally replaced with the materialized view's inline view so that the query can be rewritten. Both of the two earlier queries are rewritten with SUM_SALES_MV as shown in the code example above.



Transforming and Rewriting the Query from the Two Previous Examples

Transforming and Rewriting the Query from the Two Previous Examples

-- Both above queries are finally re-written as foll ows:SELECT p_id, c_id, sum_soldFROM SUM_SALES_MV;

-- Both above queries are first transformed as follo ws:SELECT prod_id, cust_id, sum(amount_sold)FROM MV_IVGROUP BY prod_id, cust_id;

SELECT iv.prod_id, iv.cust_id,SUM(iv.amount_sold) sum_amount_soldFROM (SELECT products.prod_id,

cust_id, amount_soldFROM products, salesWHERE sales.prod_id = products.prod_id) IV


SELECT iv.prod_id, iv.cust_id,SUM(iv.amount_sold) sum_amount_soldFROM (SELECT products.prod_id, cust_id,

amount_soldFROM sales, productsWHERE sales.prod_id = products.prod_id) IV


Both previous query examples with the matching and equivalent inline view texts to that of the MV are first transformed as shown in the second example above to use the MV. Next the query inline view for both examples will be internally replaced with the materialized view's inline view so that the query can be rewritten.



Inline View Rewrite Guidelines

Inline View Rewrite Guidelines

General inline view rewrite is not supported for qu eries that contain the following: • Set operators• GROUPING SETclauses

• Nested subqueries• Nested inline views• Remote tables



Did Query Rewrite Occur?

Did Query Rewrite Occur?

Execute the query.

Use the DBMS_MVIEW.EXPLAIN_

REWRITE procedure.

Was the query rewritten?

Use the EXPLAIN PLANstatement.

Because query rewrite occurs transparently, it is not always evident that it has taken place. The rewritten statement is not stored in the V$SQL view, nor can it be dumped in a trace file. Of course, if the query runs faster, rewrite should have occurred; but there is no proof to confirm this. There are two ways to confirm that the query rewrite has occurred:

• Use the EXPLAIN PLAN statement and check whether the OBJECT_NAME column contains the name of a materialized view.

• Use the DBMS_MVIEW.EXPLAIN_REWRITE procedure to see whether a query will be rewritten or not.

Note: For additional information about verifying whether or not the query was rewritten using the EXPLAIN PLAN statement and the DBMS_MVIEW.EXPLAIN_REWRITE procedure, see the Oracle Database 10g: Implement and Administer a Data Warehouse Instructor-Led Training or the Oracle Database Data Warehousing Guide 11g Release 1 (11.1) documentation.



Query Rewrite Using Remote Tables in Oracle Database 11g

Query Rewrite Using Remote Tables in Oracle Database 11 g

• Oracle supports query rewrite with MVs that referen ce tables at a single remote database site.

• The MV should be present at the site where the quer y is being issued.

• Because any remote table update cannot be propagated to the local site simultaneously, query rewrite will only work in the stale_tolerated mode.

• Whenever a query contains columns that are not foun d in the MV, a join back is used to rewrite the query .

• If the join back table is not found at the local si te, query rewrite will not take place.

Oracle Database 11g extends the current query rewrite capability. Query rewrite can reference remote objects using MVs that reference the remote objects. This reduces or eliminates the data from network (network round trips), which is a costly operation. Note: Because the constraint information of the remote tables is not available at the remote site, query rewrite will not make use of any constraint information.



Query Rewrite Using Remote Tables: Example

Query Rewrite Using Remote Tables: Example

SELECT p.prod_id, t.week_ending_day, s.cust_id,SUM(s.amount_sold) AS sum_amount_sold

FROM sales@remotedbl s, products@remotedbl p, times@remotedbl t

WHERE s.time_id=t.time_id AND s.prod_id=p.prod_idGROUP BY p.prod_id, t.week_ending_day, s.cust_id;

CREATE MATERIALIZED VIEW sum_sales_prod_week_mvENABLE QUERY REWRITE ASSELECT p.prod_id, t.week_ending_day, s.cust_id,

SUM(s.amount_sold) AS sum_amount_soldFROM sales@remotedbl s, products@remotedbl p, times@remotedbl tWHERE s.time_id=t.time_id AND s.prod_id=p.prod_idGROUP BY p.prod_id, t.week_ending_day, s.cust_id;

SELECT prod_id, week_ending_day, cust_id, sum_amou nt_soldFROM sum_sales_prod_week_mv;

The materialized view shown in the first example above is present at the local site, but it references tables that are all found at the remote site. The query in the second example shown above contains tables that are found at a single remote site. Even though the query references remote tables, it will be rewritten using the above materialized view as shown in the third example above.



Summary

Summary

In this lesson, you should have learned how to:• Review the benefits of using materialized views• Review the benefits of query rewrite• Use the query rewrite enhancement to support querie s

with inline views• Identify the query rewrite enhancement that support s

queries with remote tables




Performing Pivoting and Unpivoting Operations Chapter 0 - Page 1

Performing Pivoting and Unpivoting Operations



Chapter 4Performing Pivoting and Unpivoting Operations

Performing Pivoting and UnpivotingOperations



Objectives

Objectives

After completing this lesson, you should be able to :• Identify the benefits of pivoting and unpivoting

operations• Write crosstab queries to pivot (rotate) column val ues

into new columns and to unpivot (rotate) columns in to column values

• Pivot and unpivot with multiple columns and multipl e aggregates

• Use wildcards and aliases with pivoting operations



The Order Entry (OE) Schema Used in This Lesson

The Order Entry (OE) SchemaUsed in This Lesson

WAREHOUSESwarehouse_id

warehouse_namelocation_id

ORDERSorder_id

order_dateorder_modecustomer_idorder_statusorder_total

sales_rep_idpromotion_id

ORDER_ITEMorder_id

line_item_idproduct_idunit_pricequantity

PRODUCT_INFORMATION

product_idproduct_name

product_descriptioncategory_idweight_class

warranty_periodsupplier_id

product_statuslist_pricemin_price

catalog_url

CUSTOMERScustomer_id

cust_first_namecust_ last_name

cust_ address_typ

phone_numbersnls_languagenls_territorycredit_limitcust_ email

account_mgr_iddate_of_birthmarital_status

genderIncome_level

street_addresspostal_code

citystate_province

country_id

PRODUCT_DESCRIPTION

product_idlanguage_idproduct_name

product_description

INVENTORIESproduct_id

warehouse_idquantity_on_hand

The ER-diagram for the OE schema is displayed above and on the following pages to get you familiar with the names of tables and columns that are used throughout this lesson. The company sells several categories of products, including computer hardware and software, music, clothing, and tools. The company maintains product information that includes product identification numbers, the category into which the product falls, the weight group (for shipping purposes), the warranty period if applicable, the supplier, the status of the product, a list price, a minimum price at which a product will be sold, and a URL for manufacturer information.



The Sales History (SH) Schema Used in This Lesson

The Sales History (SH) SchemaUsed in This Lesson

COSTSprod_idtime_id

promo_idchannel_id

unit_costunit_price

PROMOTIONSpromo_id

promo_namepromo_subcategory

promo_subcategory_idpromo_category

promo_category_idpromo_cost

promo_begin_datepromo_end_date

promo_totalpromo_total_id

SALESprod_idcust_idtime_id

channel_idpromo_id

quantity_soldamount_sold

CHANNELSchannel_id

channel_descchannel_class

channel_class_idchannel_total

channel_total_id

TIMEStime_id

day_nameday_number_in_weekday_number_in_monthcalendar_week_number

fiscal_week_numberweek_ending_day

week_ending_day_idcalendar_month_number

fiscal_month_numbercalendar_month_desc

calendar_month_idfiscal_month_id

days_in_cal_monthdays_in_fis_monthend_of_cal_ monthend_of_fis_month

calendar _month _namefiscal _month _name

calendar _quarter _desccalendar_quarter_idfiscal _quarter _desc

fiscal _quarter _iddays_in_cal_quarterdays_in_fis_quarterend_of_cal_quarterend_of_fis_quarter

calendar_quarter_numberfiscal_quarter_number

calendar_yearcalendar_year_id

fiscal_yearfiscal_year_id

days_in_cal_yeardays_in_fis_yearend_of_cal_yearend_of_fis_year

PRODUCTS

The company does a high volume of business, so it runs business statistics reports to aid in decision support. Many of these reports are time based and nonvolatile. That is, they analyze past data trends. The company loads data into its data warehouse regularly to gather statistics for the reports. The reports include annual, quarterly, monthly, and weekly sales figures by product. The company also runs reports on the distribution channels through which its sales are delivered. When the company runs special promotions on its products, it analyzes the impact of the promotions on sales. It also analyses sales by geographical area.



The Sales History (SH) Schema Used in this eStudy

The Sales History (SH) SchemaUsed in this eStudy

PRODUCTSprod_id

prod_nameprod_desc

prod_subcategoryprod_subcategory_id

prod_subcategory_descprod_category

prod_category_idprod_category_descprod_weight_class

prod_unit_of_measureprod_pack_size

supplier_idprod_status

prod_list_priceprod_min_price

prod_totalprod_total_idprod_src_id

prod_eff_fromprod_eff_toprod_valid

CUSTOMERScust_id

cust_first_namecust_last_name

cust_gendercust_year_of_birthcust_marital_statuscust_street_address

cust_postal_codecust_city

cust_city_idcust_state_province

cust_state_province_idcountry_id

cust_main_phone_numbercust_income_levelcust_credit_limit

cust_emailcust_total

cust_total_idcust_src_id

cust_eff_fromcust_eff_tocust_valid

COUNTRIEScountry_id

country_iso_codecountry_name

country_subregioncountry_subregion_id

country_regioncountry_region_id

country_totalcountry_total_id

Country_name_hist

COSTS SALESSALES



The Benefits of Using Pivoting Operations

The Benefits of Using Pivoting Operations

• Data returned by business intelligence (BI) queries is more useful if presented in a cross-tabular format.

• Pivoting enables you to transform multiple rows of input into fewer rows, generally with more columns.

• When pivoting, an aggregation operator is applied, enabling the query to condense large data sets into smaller more readable results.

• Performing pivots on the server side can:– Enhance processing speed– Reduce network load

Pivoting is a key technique in BI queries where you transform multiple rows of input into fewer rows, generally with more columns. By performing pivots on the server side:

• Processing burden is removed from client applications, simplifying client-side development and potentially enhancing processing speed

• Network load is reduced, because only aggregated pivot results need to traverse the network and not the detail data

Data that was originally on multiple rows can be transformed into a single row of output, enabling intrarow calculations without a SQL Join operation.



The PIVOT and UNPIVOT Clauses of the SELECT Statement

The PIVOT and UNPIVOTClauses of the SELECTStatement

• You can use the PIVOT operator of the SELECTstatement to write cross-tabular queries that rotat e the column values into new columns, aggregating data in the process.

• You can use the UNPIVOToperator of the SELECTstatement to rotate columns into values of a column .

PIVOT UNPIVOT

Note: An UNPIVOT does not reverse a PIVOT operation; instead, it rotates data from columns into rows.



Pivoting on the QUARTER Column: Conceptual Example

Pivoting on the QUARTERColumn: Conceptual Example

30,000

40,000

60,000

30,000

40,000

20,000

AMOUNT_SOLD

2,500Q1IUSAKids Jeans

2,000Q2CJapanKids Jeans

2,000Q3SUSAShorts

I

P

C

CHANNEL

Kids Jeans

Shorts

Shorts

PRODUCT

1,000Q2Germany

1,500Q4USA

Q2

QUARTER

2,500Poland

QUANTITY_SOLD

COUNTRY

2,000

Q3

Kids Jeans

Shorts

PRODUCT

3,500

2,000

Q2

1,5002,500

Q4Q1

The first example above displays the columns of a table. The table in the second example displays the results of pivoting on the QUARTER column. The values of the QUARTER column, Q1, Q2, Q3, and Q4 are rotated into new columns. The quantity sold is grouped by products and quarters. To accomplish the desired result, you can use the PIVOT clause to PIVOT the QUARTER column, that is, turn the values of this column into separate columns and aggregate data using a group function such as SUM on the QUANTITY_SOLD along the way for each PRODUCT. In the second example, the QUARTER column is pivoted. The values of the QUARTER column namely Q1, Q2, Q3, and Q4 are rotated into columns. In addition, the quantity sold is calculated for each quarter for ALL products, ALL channels, and ALL countries. For example, the total number of Shorts sold for all channels, all countries, for quarter 2 is 3500. The total number Kids Jeans sold for all channels, all countries, for quarter 2 is 2000. Note that the first example contains six rows before the pivoting operation. In the second example and after pivoting the QUARTER column, only two rows are displayed. Pivoting transforms multiple rows of input into fewer and generally wider rows.



The PIVOT Clause Syntax

The PIVOT Clause Syntax

table_reference PIVOT [ XML ]( aggregate_function ( expr ) [[AS] alias ]

[, aggregate_function ( expr ) [[AS] alias ] ]...pivot_for_clausepivot_in_clause )

-- Specify the column(s) to pivot whose values are t o -- be pivoted into columns.pivot_for_clause = FOR { column |( column [, column]... ) }

-- Specify the pivot column values from the columns you -- specified in the pivot_for_clause.pivot_in_clause = IN ( { { { expr | ( expr [, expr]... ) } [ [ AS] al ias] }...

| subquery | { ANY | ANY [, ANY]...} } )

You can use the pivot clause to write cross-tabular queries that rotate rows into columns, aggregating the data in the process of the rotation. The XML keyword is required when you use either a subquery or the wildcard ANY in the pivot_in_clause to specify pivot values. You cannot specify XML when you specify explicit pivot values using expressions in the pivot_in_clause . If the XML keyword is used, the output will include grouping columns and one column of XMLType rather than a series of pivoted columns. The optional AS alias enables you to specify an alias for each measure. Note that the aggregate function has an implicit GROUP BY based on the columns in the source data. The aggregate_function operates on the table’s data, and the result of the computation appears in the crosstab report. The expr argument for the aggregate function is the measure to be pivoted. It must be a column or expression of the query_table_expression on which the pivot_clause is operating. The optional AS alias enables you to specify an alias for each measure. In the pivot_for_clause , specify one or more columns whose values are to be pivoted into columns. In the pivot_in_clause , specify the pivot column values from the columns you specified in the pivot_for_clause . For expr , specify a constant value of a pivot column. You can optionally provide an alias for each pivot column value. Use a subquery to extract the pivot column values by way of a nested subquery. If you specify ANY, then all values of the pivot columns are pivoted into columns. Subqueries and wildcards are useful if you do not know the specific values in the pivot columns.



However, you will need to do further processing to convert the XML output into a tabular format. The values evaluated by the pivot_in_clause become the columns in the pivoted data.



Creating a New View: Example

Creating a New View: Example

CREATE OR REPLACE VIEW sales_view ASSELECT

prod_name AS product, country_name AS country, channel_id AS channel, SUBSTR(calendar_quarter_desc, 6,2) AS quarter,SUM(amount_sold) AS amount_sold, SUM(quantity_sold) AS quantity_sold

FROM sales, times, customers, countries, productsWHERE sales.time_id = times.time_id AND

sales.prod_id = products.prod_id ANDsales.cust_id = customers.cust_id ANDcustomers.country_id = countries.country_id

GROUP BY prod_name, country_name, channel_id,SUBSTR(calendar_quarter_desc, 6, 2);

In this lesson, you use the newly created view, sales_view . The definition of sales_view is displayed as follows:

DESCRIBE sales_view Name Null? Type -------------------------- -------- --------------- --- PRODUCT NOT NULL VARCHAR2(50) COUNTRY NOT NULL VARCHAR2(40) CHANNEL NOT NULL NUMBER QUARTER VARCHAR2(8) AMOUNT_SOLD NUMBER QUANTITY_SOLD NUMBER

The calendar_quarter_desc column values in the SH schema are as follows: SELECT DISTINCT calendar_quarter_desc FROM times; CALENDAR_QUARTER_DESC --------------------- 1999-02 2000-04 2002-03 2000-03 2001-04 1998-01



1999-04 2002-02 2002-04 2000-02 2001-01 1998-02 1998-04 1999-01 2001-02 2001-03 2002-01 1999-03 1998-03 2000-01

20 rows selected.

Note: The two-character quarter value is extracted from the calendar_quarter_desc column starting at position 6 as shown in the syntax in the example above.



Selecting the SALES_VIEW Data

Selecting the SALES_VIEWData

SELECT product, country, channel, quarter, quantity _soldFROM sales_view;

PRODUCT COUNTRY CHANNEL QUARTER QUANT ITY_SOLD------------ ------------ ---------- -------- --------- ----Y Box Italy 4 01 21Y Box Italy 4 02 17Y Box Italy 4 03 20. . .Y Box Japan 2 01 35Y Box Japan 2 02 39Y Box Japan 2 03 36Y Box Japan 2 04 46Y Box Japan 3 01 65. . .Bounce Italy 2 01 34Bounce Italy 2 02 43. . .9502 rows selected.

The two boxes around QUARTER and “9502 rows selected ” displayed in the second example above, highlight the two key changes performed by the PIVOT operator:

• The QUARTER column becomes multiple columns, each holding one quarter.

• The row count will drop to just 71 from 9502, representing the distinct products in the schema.

The following displays the distinct channel_id and channel_desc column values from the CHANNEL table and the distinct QUARTER column values from the QUARTER table in the SH schema.

SELECT DISTINCT channel_id, channel_desc FROM channels ORDER BY channel_id;

CHANNEL_ID CHANNEL_DESC ---------- -------------------- 2 Partners

3 Direct Sales 4 Internet 5 Catalog 9 Tele Sales

COLUMN QUARTER FORMAT A7



SELECT DISTINCT quarter FROM sales_view; QUARTER ------- 04 01 02 03



Pivoting the QUARTER Column in the SH Schema: Example

Pivoting the QUARTERColumn in the SHSchema: Example

SELECT *FROM

(SELECT product, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR quarter IN ('01', '02', '03', '04'))

ORDER BY product DESC;

PRODUCT '01' '02' '03' '04'------------------ ---------- ---------- ---------- --- -------Y Box 1455 1766 1716 1992Xtend Memory 3146 4121 4122Unix/Windows 1-user 4259 3887 4601 4049Standard Mouse 3376 1699 2654 2427Smash up Boxing 1608 2127 1999 2110. . .71 rows selected.

The example above uses a subquery (inline view) in the FROM clause. This is needed because if you issue a SELECT * directly from the sales view, the query outputs rows for each row of sales_view . The result set shows the product column followed by a column for each value of quarter specified in the IN clause. The numbers shown in the pivoted output are the sum of quantity_sold for each product at each quarter. If you also specify an alias for each measure, then the column name is a concatenation of the pivot column value or alias, an underscore (_), and the measure alias. You can use aliases in the above example as shown below:

SELECT * FROM (SELECT product, quarter, quantity_sold FROM sales_view) PIVOT (sum(quantity_sold) FOR quarter in ('01' AS Q1, '02' AS Q2, '03' AS Q 3, '04' AS Q4)) ORDER by PRODUCT DESC;

PRODUCT Q1 Q2 Q3 Q4 ---------------- ---------- ---------- ---------- - --------- Y Box 1455 1766 1716 1992 Xtend Memory 3146 4121 4122 Unix/Windows 4259 3887 4601 4049 Standard Mouse 3376 1699 2654 2427



. . . 71 rows selected.

Pivoting on the QUARTER Column: Example

Before Oracle Database 11g, the above example can be accomplished using the following syntax: SELECT product, SUM(CASE when quarter = ' 01 ' THEN quantity_sold ELSE NULL END) Q1, SUM(CASE when quarter = ' 02 ' THEN quantity_sold ELSE NULL END) Q2, SUM(CASE when quarter = ' 03 ' THEN quantity_sold ELSE NULL END) Q3, SUM(CASE when quarter = ' 04 ' THEN quantity_sold ELSE NULL END) Q4 FROM ( SELECT product, quarter, quantity_sold FROM SALES_VIEW ) GROUP BY product;

The advantage of the new syntax over the syntax used before Oracle Database 11g is that it enables greater query optimization by Oracle. The query optimizer recognizes the PIVOT keyword and as a result, uses algorithms optimized to process it efficiently.



Pivoting on the ORDER_MODE Column in the OE Schema: Example

Pivoting on the ORDER_MODEColumn in the OESchema: Example

CREATE TABLE pivot_table ASSELECT * FROM(SELECT EXTRACT(YEAR FROM order_date) year, order_m ode,

order_total FROM orders)PIVOT(SUM(order_total) FOR order_mode IN ('direct' AS St ore,

'online' AS Internet))ORDER BY year;

SELECT * FROM pivot_table ORDER BY year;YEAR STORE INTERNET---------- ---------- ----------

1990 61655.71996 5546.61997 3101998 309929.8 100056.61999 1274078.8 1271019.52000 252108.3 393349.4

6 rows selected.

The example above uses the Order Enter (OE) schema. The oe.orders table contains information about when an order was placed (order_date ), how it was placed (order_mode ), and the total amount of the order (order_total ), as well as other information. The example shows how to use the PIVOT clause to pivot order_mode values into columns, aggregating order_total data in the process, to get yearly totals by order mode. The example creates a new table, pivot_table , using a subquery. The example, creates the new table, inserts rows returned by the subquery, and uses aliases for the direct and online pivot column values.



Pivoting on Multiple Columns


• To pivot on more than one column: – A pivoting column is required to be a column of the table

reference on which the pivot is operating– The pivoting column cannot be an arbitrary expressi on

• If you need to pivot on an expression, then you sho uld use an alias for the expression in a view before th e pivot operation.





PRODUCT DIRECT_SALES_Q1 INTERNET_ SALES_Q1------------------------- --------------- ----------- ------Y Box 771 253Xtend Memory 1935 350Unix/Windows 1-user pack 2544 397Standard Mouse 2326 256Smash up Boxing 1114 129. . .71 rows selected.

SELECT *FROM

(SELECT product, channel, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR (cha nnel,quarter) IN ((3, '01') AS Direct_Sales_Q1,

(4, '01') AS Internet_Sales_Q1))ORDER BY product DESC;

The example above pivots on both the CHANNEL and QUARTER columns. The example uses only the CHANNEL values 3 (Direct Sales ) and 4 (Internet ) and only the Q1 value for the QUARTER column. The example below specifies more values for the QUARTER column:

SELECT * FROM (SELECT product, channel, quarter, quantity_sol d FROM sales_view ) PIVOT (sum(quantity_sold) FOR (channel, quart er) IN ((3, '01') AS Direct_Sales_Q1, (3, '02') AS Direct_Sales_Q2, (3, '03') AS Direct_Sales_Q3, (3, '04') AS Direct_Sales_Q4, (4, '01') AS Internet_Sales_Q1, (4, '02') AS Internet_Sales_Q2, (4, '03') AS Internet_Sales_Q3, (4, '04') AS Internet_Sales_Q4)) ORDER BY product DESC;



Pivoting Using Multiple Aggregations

Pivoting Using Multiple Aggregations

SELECT *FROM

(SELECT product, channel, amount_sold, quantity_sol dFROM sales_view) PIVOT (SUM(amount_sold) AS sums, SUM(quantity_sold) as sumqFOR channel IN (3 AS Dir_Sales, 4 AS Int_Sales))

ORDER BY product DESC;

PRODUCT DIR_SALES_SUMS DIR_SALES_SUMQ INT_SALE S_SUMS INT_SALES_SUMQ------------ -------------- -------------- ----------- --- --------------Y Box 1081050.96 3552 382 767.45 1339Xtend Memory 217011.38 8562 40 553.93 1878Unix/Windows 1999882.17 9313 376 071.62 1872Standard Mouse 153199.63 6140 28 768.04 1195Smash up Boxing 174592.24 5106 27 858.84 904...

71 rows selected.

The query in the example above pivots SALES_VIEW on the CHANNEL column. The amount_sold and quantity_sold measures get pivoted. Note that the query creates column headings by concatenating the pivot columns with the aliases of the aggregate functions, plus an underscore. When you use multiple aggregations, you can omit the alias for only one aggregation. If you omit an alias, then the corresponding result column name is the pivot value (or the alias for the pivot value).



Distinguishing PIVOT-Generated Nulls from Nulls in the Source Data

Distinguishing PIVOT-Generated Nulls from Nulls in the Source Data

SELECT * FROM sales2;

PROD_ID QTR AMOUNT_SOLD------- --- -----------100 Q1 10100 Q1 20100 Q2 200 Q1 50

SELECT *FROM(SELECT prod_id, qtr, amount_sold

FROM sales2) PIVOT (SUM(amount_sold), COUNT(*) AS c ount_total FOR qtr IN ('Q1', 'Q2') )

ORDER BY prod_id DESC;PROD_ID Q1 Q1_COUNT_TOTAL Q2 Q2_COUNT_TOTAL------- --- -------------- --------- ----------------

100 30 2 1200 50 1 0

You can distinguish between NULL values that are generated from the use of PIVOT and those that exist in the source data. The following example illustrates NULL that PIVOT generates. The first code example above, assumes an existing table named sales2 . The definition of the sales2 table structure and the commands used to insert the data into sales2 are as follows:

CREATE TABLE sales2

(prod_id NUMBER,

qtr VARCHAR2(5),

amount_sold NUMBER);

INSERT INTO sales2 VALUES(100, 'Q1', 10);


INSERT INTO sales2 VALUES(100, 'Q2', null);


The query in the second code example returns rows prod_id and the resulting pivot columns: Q1, Q1_COUNT_TOTAL, Q2, Q2_COUNT_TOTAL. For each unique value of prod_id , Q1_COUNT_TOTAL, the query returns the total number of rows whose QTR value is Q1. The unique value of Q2_COUNT_TOTAL returns the total number of rows whose QTR value is Q2. From the result set shown in the second example above, there are two sales rows for prod_id



100 for quarter Q1, and one sales row for prod_id 100 and quarter Q2. For prod_id 200, there is one sales row for quarter Q1 and no sales row for quarter Q2. Using Q2_COUNT_TOTAL, you can identify that the NULL for PROD_ID 100 in Q2 is a result of a row in the original table whose measure is of NULL value. The NULL for PROD_ID 200 in Q2 is due to no row being present in the original table for prod_id 200 in quarter Q2.



Using the XML Keyword to Specify Pivot Values: Two Methods

Using the XML Keyword to Specify Pivot Values: Two Methods

ANY

If you use theXML keyword with the Pivot

syntax

Use the ANYkeyword, or

Use a subquery

XML

The XML string for each row includes only the pivot values found in the input

data for that row.

The XML string includes allpivot values found by the subquery even if there are

no aggregate values.

Using the XML keyword in the Pivot syntax requires including either the ANY keyword or a subquery. Each output row will include the following:

• The implicit Group By columns

• A single column of XML Type containing an XML string for all value and measure pairs

The XML string for each row will hold aggregated data corresponding to the row’s implicit GROUP BY value. The values of the pivot column are evaluated at execution time.



Specifying Pivot Values: Using the ANY Keyword

Specifying Pivot Values: Using the ANYKeyword

PRODUCT--------------------------------------------------CHANNEL_XML--------------------------------------------------- ---------------------

. . .1.44MB External 3.5" Diskette<PivotSet><item><column name = "CHANNEL">3</column><column na me = "SUM(QUANTITY_SOLD)">14189</column></item><item><column name = "CHANNEL">2</column><column na me = "SUM(QUANTITY_SOLD)">6455</column></item><item><column name = "CHANNEL">4</column><column na me = "SUM(QUANTITY_SOLD)">2464</column></item></PivotSet >71 rows selected.

SET LONG 1024;SELECT *FROM

(SELECT product, channel, quantity_soldFROM sales_view

) PIVOT XML (SUM(quantity_sold) FOR channel IN (ANY ) )ORDER BY product DESC;

The ANY keyword acts as a wildcard. If you specify ANY, all values found in the pivot column will be used for pivoting. Note that when using the ANY keyword, the ANY string for each output row will include only the pivot values found in the input data corresponding to that row. The example above uses the ANY wildcard keyword. The XML output includes all channel values in the sales_view view created earlier in this lesson. The ANY keyword is only available in PIVOT operations as part of an XML operation. This output includes data for cases where the channel exists in the data set. You can use wildcards or subqueries to specify the pivot IN list members when the values of the pivot column are not known.



Specifying Pivot Values: Using Subqueries

Specifying Pivot Values: Using Subqueries

PRODUCT----------CHANNEL_XML--------------------------------------------------- --------------------. . .Y Box<PivotSet><item><column name = "CHANNEL">9</column><column na me = "SUM(QUANTITY_SOLD)">1</column></item><item><column name = "CHANNEL">2</column><column na me ="SUM(QUANTITY_SOLD)">2037</column></item><item><column name = "CHANNEL">5</column><column na me = "SUM(QUANTITY_SOLD)"></column></item><item><column name = "CHANNEL">3</column><column na me ="SUM(QUANTITY_SOLD)">3552</column></item>. . .

SELECT *FROM

(SELECT product, channel, quantity_soldFROM sales_view

) PIVOT XML(SUM(quantity_sold)FOR channel IN (SELECT distinct channel_id FROM channels));

The example above illustrates the use of a subquery with the XML keyword.The XML output includes all channel values and the sales data corresponding to each channel and for each product. Subquery-based pivots give results different from those generated by the use of the ANY wildcard. In the example above, when you use a subquery, the XMLtype column will show value and measure pairs for all channels for each product even if the input data has no such product/channel combination. For example, the XML string in the slide shows the highlighted Channel 5 although it has no value for the SUM(QUANTITY_SOLD)column. Pivots that use a subquery will, therefore, often have longer output than queries based on the ANY keyword. Depending on how you process the query results, subquery-style output may be more convenient to work with than the results derived from ANY. Note: The results in the above example do not show a complete row of output due to space limitations.



Unpivoting the QUARTER Column: Conceptual Example

Unpivoting the QUARTERColumn: Conceptual Example

2,000

Q3

Kids Jeans

Shorts

PRODUCT

3,500

2,000

Q2

1,5002,500

Q4Q1

2,500Q1Kids Jeans

2,000Q2Kids Jeans

3,500Q2Shorts

1,500Q4Kids Jeans

Q3

QUARTER

2,000Shorts

SUM_OF_QUANTITYPRODUCT

The example above unpivots the QUARTER column, thus turning the quarters columns into the values of the QUARTER column. An UNPIVOT operation does not reverse a PIVOT operation; instead, it rotates data found in multiple columns of a single row into multiple rows of a single column. If you are working with pivoted data, an UNPIVOT operation cannot reverse any aggregations that have been made by PIVOT or any other means. Note that the first example contains two rows before the unpivoting operation. In the second example, the unpivoting operation on the QUARTER column displays five rows. Unpivoting transforms fewer rows of input into generally more rows.



Using the UNPIVOT Operator

Using the UNPIVOTOperator

• An UNPIVOTdoes not reverse a PIVOT operation; instead, it rotates data from columns into rows.

• If you are working with pivoted data, an UNPIVOToperation cannot reverse any aggregations that have been made by PIVOT or any other means.

UNPIVOT

Data from sources such as spreadsheets and flat files is often in pivoted form. For instance, sales data will often be stored in a separate column for each time period. UNPIVOT can normalize such data, transforming multiple columns into a single column. When the data is normalized with UNPIVOT, it is much more accessible for relational database processing with SQL. By placing data in a normalized layout, queries can readily apply SQL aggregate and analytic functions, enabling powerful analysis. Similarly, it is more efficient to specify WHERE clause predicates on normalized data.



Using the UNPIVOT Clause

Using the UNPIVOTClause

• The UNPIVOTclause rotates columns from a previously pivoted table or a regular table into rows. You spe cify:– The measure columns to be unpivoted– The names for the columns that will result from the

unpivot operation– The columns that will be unpivoted back into values of

the column specified in the pivot_for_clause

• You can use an alias to map the column name to another value.



The Data Types of the Value Columns in an UNPIVOT Operation

The Data Types of the Value Columns in an UNPIVOT Operation

BINARY_DOUBLEIf ANY value column is BINARY_DOUBLE

NUMBERIf ALL the value columns are NUMBER

CHARIf ALL the value columns are CHAR

BINARY_FLOATIf NO value column is BINARY_DOUBLEbut ANY value column is BINARY_FLOAT

If ANY value column is VARCHAR2

Data Type for the Value Columns?

VARCHAR2

Resulting Unpivoted Column Data Type

The unpivot operation turns a set of value columns into one column. Therefore, the data types of all the value columns must be of the same data type such as numeric or character.

• If all the value columns are CHAR, then the unpivoted column is CHAR.

• If any value column is VARCHAR2, then the unpivoted column is VARCHAR2.

• If all the value columns are NUMBER, then the unpivoted column is NUMBER.

• If any value column is BINARY_DOUBLE, then the unpivoted column is BINARY_DOUBLE.

• If no value column is BINARY_DOUBLE but any value column is BINARY_FLOAT, then the unpivoted column is BINARY_FLOAT.



The UNPIVOT Clause Syntax

The UNPIVOTClause Syntax

table_reference UNPIVOT [{INCLUDE|EXCLUDE } NULLS]-- specify the measure column(s) to be unpivoted.( { column | ( column [, column]... ) }

unpivot_for_clauseunpivot_in_clause )

-- Specify one or more names for the columns that wi ll-- result from the unpivot operation.

unpivot_for_clause = FOR { column | ( column [, column]... ) }

-- Specify the columns that will be unpivoted into v alues of -- the column specified in the pivot_for_clause.

unpivot_in_clause = ( { column | ( column [, column]... ) }

[ AS { constant | ( constant [, constant]... ) } ][, { column | ( column [, column]... ) }

[ AS { constant | ( constant [, constant]...) } ] ] ...)

The UNPIVOT clause rotates columns into rows. The [INCLUDE] | [EXCLUDE] [NULLS] clause gives you the option of including or excluding nullvalued rows. [INCLUDE] [NULLS] causes the unpivot operation to include nullvalued rows; [EXCLUDE] [NULLS] eliminates nullvalued rows from the return set. If you omit this clause, then the unpivot operation excludes nulls. For column , specify a name for each output column that will hold measure values, such as sales_quantity . In the pivot_for_clause , specify a name for each output column that will hold descriptor values, such as quarter or product . In the unpivot_in_clause , specify the input data columns whose names will become values in the output columns of the pivot_for_clause . These input data columns have names specifying a category value, such as Q1, Q2, Q3, and Q4. The optional alias enables you to map the column name to any desired value.



Creating a New Pivot Table: Example

Creating a New Pivot Table: Example

CREATE TABLE pivotedtable ASSELECT *FROM

(SELECT product, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR quarter IN ('01' AS Q1, '02' AS Q2,

'03' AS Q3, '04' AS Q4));

Table created.

SELECT * FROM pivotedtable ORDER BY product DESC;

PRODUCT Q1 Q2 Q3 Q4------------------ ---------- ---------- ---------- --- -------Y Box 1455 1766 1716 1992Xtend Memory 3146 4121 4122 3802. . .

71 rows selected.

The example above creates a new table named pivotedtable .



Unpivoting the QUARTER Column in the SH Schema: Example

Unpivoting the QUARTERColumn in the SHSchema: Example

SELECT *FROM pivotedtable UNPIVOT (quantity_sold For Quarter IN (Q1, Q2, Q3, Q4))ORDER BY product DESC, quarter;

PRODUCT QUARTER QUANTITY_SOLD-------------------------- ------- -------------Y Box Q1 1455Y Box Q2 1766Y Box Q3 1716Y Box Q4 1992Xtend Memory Q1 3146Xtend Memory Q2 4121Xtend Memory Q3 4122Xtend Memory Q4 3802Unix/Windows 1-user pack Q1 4259. . .

283 rows selected.

The QUARTER column in the above example was formatted using the following command:

COLUMN quarter FORMAT A7

Before Oracle Database 11g, you can simulate the UNPIVOT syntax in the above example using existing SQL as shown below:

SELECT product, ' Q1' as quarter, Q1 as quantity_sold FROM pivotedTable WHERE Q1 is not NULL union all SELECT product, ' Q2' as quarter, Q2 as quantity_sold FROM pivotedTable WHERE Q2 is not NULL union all SELECT product, ' Q3' as quarter, Q3 as quantity_sold FROM pivotedTable WHERE Q3 is not NULL union all SELECT product, ' Q4' as quarter, Q4 as quantity_sold FROM pivotedTable WHERE Q4 is not NULL;

However the UNPIVOT syntax enables more efficient query processing. The UNPIVOT keyword alerts the query optimizer to the desired behavior. As a result, the optimizer calls highly efficient algorithms.



Unpivoting the ORDER_MODE Column in the OE Schema: Example

Unpivoting the ORDER_MODEColumn in the OESchema: Example

SELECT * FROM pivot_tableUNPIVOT (yearly_total FOR order_mode IN (store AS ' direct',

internet AS 'online'))ORDER BY year, order_mode;

YEAR ORDER_MODE YEARLY_TOTAL----- ----------- ------------1990 direct 61655.71996 direct 5546.61997 direct 3101998 direct 309929.81998 online 100056.61999 direct 1274078.81999 online 1271019.52000 direct 252108.32000 online 393349.4

9 rows selected.

The UNPIVOT clause enables you to restore a pivoted table, or a table with similar structure, so that selected columns are pivoted into values in a single column. Note the use of column aliases in the above example.



Unpivoting Multiple Columns in the SH Schema: Example

Unpivoting Multiple Columns in the SHSchema: Example

CREATE TABLE multi_col_pivot ASSELECT *FROM

(SELECT product, channel, quarter, quantity_soldFROM sales_view) PIVOT (sum(quantity_sold) FOR (cha nnel,quarter) IN ((3, '01') AS Direct_Sales_Q1,

(4, '01') AS Internet_Sales_Q1))ORDER BY product DESC;Table created.

SELECT * FROM multi_col_pivot;

PRODUCT DIRECT_SALES_Q1 INTERN ET_SALES_Q1---------------------------- --------------- -------- ---------Y Box 771 253Xtend Memory 1935 350. . .71 rows selected.

The example above creates a pivot table using the CHANNEL and QUARTER columns in the SH schema. This is similar to the “Pivoting Using Multiple Columns” example mentioned earlier. The example uses only the CHANNEL values 3 (Direct Sales) and 4 (Internet) and only the Q1 value for the QUARTER column. You can use other values for both columns. The structure of the newly created table is shown below:

DESCRIBE multi_col_pivot Name Null? Type --------------------- -------- ------------------- ----- PRODUCT NOT NULL VARCHAR2(50) DIRECT_SALES_Q1 NUMBER INTERNET_SALES_Q1 NUMBER

The query in the example above returns 71 rows.



-- Provide explicit values for the unpivot columns

SELECT *FROM multi_col_pivotUNPIVOT (quantity_sold For (channel, quarter) IN

( Direct_Sales_Q1 AS ('Direct', 'Q1'), Internet_Sales_Q1 AS ('Internet', 'Q1') ) )

ORDER BY product DESC, quarter;

PRODUCT CHANNEL QUARTER QUA NTITY_SOLD------------------------------ -------- ------- ------ -------Y Box Internet Q1 253Y Box Direct Q1 771Xtend Memory Internet Q1 350Xtend Memory Direct Q1 1935. . .142 rows selected.

The example above unpivots the CHANNEL and QUARTER columns using the multi_col_pivot table created on the previous page. Note that the example uses explicit values for the unpivoted CHANNEL and QUARTER columns. The example below demonstrates unpivoting on the CHANNEL and QUARTER columns without using aliases as explicit values for the unpivoted columns. In this case, each unpivoted column uses the column name as its value. The query in the example above returns 142 rows.

SELECT * FROM multi_col_pivot UNPIVOT (quantity_sold For (channel, quarter) IN (Direct_Sales_Q1, Internet_Sales_Q1 ) );

PRODUCT CHANNEL QUARTER Q UANTITY_SOLD ------------ ----------------- ----------------- -- ----------- Y Box DIRECT_SALES_Q1 DIRECT_SALES_Q1 771 Y Box INTERNET_SALES_Q1 INTERNET_SALES_Q1 253 Xtend Memory DIRECT_SALES_Q1 DIRECT_SALES_Q1 1935 ... 142 rows selected.



Unpivoting Multiple Aggregations in the SH Schema: Example

Unpivoting Multiple Aggregations in the SHSchema: Example

CREATE TABLE multi_agg_pivot ASSELECT *FROM

(SELECT product, channel, quarter, quantity_sold, a mount_soldFROM sales_view) PIVOT

(sum(quantity_sold) sumq, sum(amount_sold) sumaFOR channel IN (3 AS Direct, 4 AS Internet) )

ORDER BY product DESC;Table created.

SELECT * FROM multi_agg_pivot;

PRODUCT QUARTER DIRECT_SUMQ DIRECT_SUMA INTERNET_SUMQ INTERNET_SUMA---------- ------- ----------- ----------- ------------ - -------------. . .Bounce 01 1000 21738.97 347 6948.76Bounce 02 1212 26417.37 453 9173.59Bounce 03 1746 37781.27 528 10029.99Bounce 04 1741 38838.63 632 12592.07. . .283 row selected.

The example shown above creates the multi_agg_pivot table using the CHANNEL column and the amount_sold and quantity_sold measures in the SH schema. This is similar to the “Pivoting Using Multiple Aggregates” example mentioned earlier. The example uses only the CHANNEL values 3 (Direct Sales ) and 4 (Internet ). You can use other values for the CHANNEL column. Note that the query creates column headings by concatenating the pivot columns with the aliases of the aggregate functions, plus an underscore. When you use multiple aggregations, you can omit the alias for only one aggregation. If you omit an alias, then the corresponding result column name is the pivot value (or the alias for the pivot value). The structure of the newly created table is shown below:

DESCRIBE multi_agg_pivot

Name Null? Type ------------------- --------- --------------------- ---- PRODUCT NOT NULL VARCHAR2(50) QUARTER VARCHAR2(8) DIRECT_SUMQ NUMBER DIRECT_SUMA NUMBER INTERNET_SUMQ NUMBER INTERNET_SUMA NUMBER



SELECT *FROM multi_agg_pivotUNPIVOT ((total_amount_sold, total_quantity_sold)For channel IN ((Direct_sumq, Direct_suma) AS 3,

(Internet_sumq, Internet_suma) AS 4 ))ORDER BY product DESC, quarter, channel;

PRODUCT QUARTER CHANNEL TOTAL_AMOUNT_SOLD TOTAL_QUANTITY_SOLD------- ------- ------- ----------------- ------------- ------Bounce 01 3 1000 21738.97Bounce 01 4 347 6948.76Bounce 02 3 1212 26417.37Bounce 02 4 453 9173.59Bounce 03 3 1746 37781.27Bounce 03 4 528 10029.99Bounce 04 3 1741 38838.63Bounce 04 4 632 12592.07. . .566 rows selected.

The Unpivot example shown above uses the newly created multi_agg_pivot table. The measures amount_sold and quantity_sold are unpivoted. Channels are mapped to the value “3” for Direct_sumq and Direct_suma and to the value “4” for Internet_sumq and Internet_suma . The channel mapping is consistent with the values used in the pivot operation that created the multi_agg_pivot table. However, any values could have been used for the channel mappings.



Summary

Summary

In this lesson, you should have learned how to:• Identify the benefits of pivoting and unpivoting

operations• Write crosstab queries to pivot (rotate) column val ues

into new columns and to unpivot (rotate) columns in to column values

• Pivot and unpivot with multiple columns and multipl e aggregates

• Use wildcards and aliases with pivoting operations



Date post:	03-Jan-2016
Category:	Documents
Upload:	anton-bushmelev
View:	325 times
Download:	9 times

Oracle Database 11g BI and Data Warehousing Student Guide

Documents