obiee-data-federation1 (1)

5/21/2018 obiee-data-federation1 (1)

1/67

Limor Fledel-VagmanBI Practice Manager

Oracle Israel

OBIEE Data Federation


2/67

OBIEE Federation

Vertical Federation

Integrating two or more disparate data sou rce having dif ferent

levelsof granularity joined by one or more conformed dimensions

Example: Essbase Sales cube to relational Sales detail data (drill-

through)

Horizontal Federation

Integrating two or more disparate data sou rce having the same

level of granularity joined by one or more conformed dimensions Example: Essbase Sales cube with HR relational database


3/67

LTS ???

Logical table sources (LTSs) are a key feature

within the OBIEE semantic model


4/67

OBIEE Aggregate Navigation


5/67

Why Aggregate Navigation?

Problem: Summary questions are slow

Example, All sales worldwide for last 2 years by quarter by family

This will potentially sum/group by millions of rows!

The disk spin time to scan all these rows will be very long

Solution: ETL to batch build high level aggregates

Oracle BI EE has aggregate navigation to use them

Aggregates can be in the same database as the source

They can also be in a different relational source or Essbase

Often orders of magnitude performance improvement


6/67

Oracle BI EE Architecture

RelationalTables

BI Server

Semantic

LayerMetadata


7/67

Oracle BI EE Aggregate Navigation

RelationalDetail

Summary

Semantic

LayerMetadata

Metadata

describes

aggregate

mappings

At query time, BI Server queries

the fastest source that has

enough detail to satisfy the user

request

Performance of highly

summarized requests isdramatically improved

BI Server


8/67

Federation Enables External Aggregates

RelationalDetail

Summary

Semantic

LayerMetadata

Summary

Aggregates can be in

other federated

sources

BI Server


9/67

External Aggregates Can Be in Essbase

RelationalDetail

Summary

Essbase

Semantic

LayerMetadata Aggregate navigation

can include

multidimensional

BI Server


10/67

External Aggregates Can Be Oracle OLAP

RelationalDetail

Summary

Oracle

OLAP AW

Semantic

LayerMetadata Aggregate navigation

can include

multidimensional

BI Server

"This information is not a commitment to deliver any material, code, or functionality. The development,

release, and timing of any features or functionality described remains at the sole discretion of Oracle"


11/67

How is Aggregate Navigation Set Up?

Database Design time:

DBA defines aggregate stars at various grain combinations

Builds ETL to load the aggregates each night

Star Time Product

Base Day SKUAgg 1 Month Brand

Agg 2 Qtr -


12/67

Mapping a Base-Level Physical Star

Business Model and

Mapping Layer

Physical LayerMappings

D0 Time

D4 Product

F0 Rev Base Measures

D_Day

D_Prod_SKU

F_Bill


13/67

Adding a Moderately Aggregated Star

Business Model and

Mapping Layer

D0 Time

D4 Product


D_Day

D_Prod_SKU

F_Bill

D_Month

D_Brand

F_Bill_agg2



14/67

Adding a Highly-Aggregated Star

Business Model and

Mapping Layer

D0 Time

D4 Product


D_Day

D_Prod_SKU

F_Bill

D_Month

D_Brand

F_Bill_agg2

D_Qtr

F_Bill_agg1



15/67

Essbase as an Aggregate Source

Business Model and

Mapping Layer

D0 Time

D4 Product


D_Day

D_Prod_SKU

F_Bill


Month

Brand

Billed Amt


16/67

Adding a Highly-Aggregated Star

Business Model and

Mapping Layer

D0 Time

D4 Product


D_Day

D_Prod_SKU

F_Bill

D_Month

D_Brand

F_Bill_agg2

D_Qtr

F_Bill_agg1

How does the query

planner decide which

FJC to use as its sourcefor a given request?



17/67

Selecting an Aggregate or Base Star at Query Time

How does the query planner decide which star to use

as its source for a given LSQL request?

First, it eliminates sources that dont have enough

detail to answer the question

Query grain star grain star qualified

Second, it estimates the fastest/prioritized source


18/67

How BI Server Knows the Grain of a Star

At query time, the BI Server checks the available

mappings to find which ones are qualified by grain


19/67

How Does It Know Which Star is Fastest?

Number of elements

Be sure to correctly populate the

Dimension Levels Number of

elements at this level

OBIEE 10g


20/67

How Does It Know Which Star is Fastest?

LTS Priority Group Order

LTS Priority Group Order

Lower value = higher priority

LTS Priority Group becomes

main decider in which LTS to

use

OBIEE 11g New Feature


21/67

Aggregate Persistence


22/67

Aggregate Persistence Feature

Managing aggregates manually has high TCO

Oracle BI EE automates creation of aggregates

Leverages existing metadata

Orders of magnitude performance boost; low TCO


23/67

Aggregate Persistence Automates Create & Load

Define agg

navigation

metadata

Detail

Summary

Semantic

LayerMetadata

Admin Tool:

Aggregate

build

automation

wizard

Run scripts

Create Tables

Aggregation

BI Server


24/67

Aggregate PersistenceManual Option

Detail

Summary

Semantic

LayerMetadata

Admin Tool:

Aggregate

build

automation

wizard

Create scripts

Create Table DDL

Aggregation

Run scripts

Create Tables

Aggregation

DBA

BI Server

Define agg

navigation

metadata


25/67

Aggregate Persistence with Essbase & Oracle OLAP

Detail

Semantic

LayerMetadata

Admin Tool:

Aggregate

build

automation

wizard

Define cube schema

Summary

Essbase or

AW

BI Server

Define agg

navigation

metadata

"This information is not a commitment to deliver any material, code, or functionality. The development,

release, and timing of any features or functionality described remains at the sole discretion of Oracle"


26/67

Administration Steps

Design aggregate facts and grains

Wizard to create metadata

1 Target database for create/load

3

2


27/67

Aggregate Persistence Wizard


28/67

Aggregate Persistence Wizard

Automates the creation of physical aggregate tables andtheir corresponding objects in the repository

Use wizard to build

script to generate

aggregate tables.Script generates

physical tables,

repository objects,

and mappings.

Fact

aggregate

Dimension

aggregates


29/67

Aggregate Persistence Wizard Steps

1. Open Aggregate Persistence Wizard.2. Specify the file name and location.

3. Select business model and measures.

4. Select dimensions and levels

5. Select connection pool, container, and name.

6. Review aggregate definition.7. View complete aggregate script.

8. Verify script is created.

9. Create and run a batch file.

10. Verify aggregates in the Physical layer.

11. Verify aggregates in the BMM layer.

12. Verify aggregates in the database.

13. Verify results in Answers.


30/67

1. Open Aggregate Persistence Wizard

Select Tools > Utilities > Aggregate Persistence Wizardand click the Execute button.


31/67

2. Specify File Name and Location

Specify a file and location where the output script shouldbe saved.


32/67

3. Select Business Model and Measures

Select business model.

Select fact table.

Select measures.


33/67

4. Select Dimensions and Levels

Select corresponding aggregate dimensions and levels.


34/67

5. Select Connection Pool, Container, and Name

Select the database object.

Select the schema.

Select the connection pool.

Name the aggregate table.


35/67

6. Review Aggregate Definition

View the aggregate

definition.


36/67

7. View Complete Aggregate Script

Confirmation of scriptcreation and location

Script


37/67

8. Verify that the Script Is Created

Navigate to the directory where the file was saved andverify that the script was created as expected.


38/67

9. Create and Run a Batch File

Create and run a batch file with the following format todrive the aggregate creation process:

nqcmd Oracle BI Server command utility

-d Oracle BI Server data

source name-u Repository username

-p Repository password

-s Path to the create

aggregate SQL script


39/67

10. Verify Aggregates in the Physical Layer

Verify that the aggregates are created in the Physicallayer of the repository as expected.

Fact aggregate

Dimension aggregates


40/67

11. Verify Aggregates in the BMM Layer

Verify that the aggregates are created in the BusinessModel and Mapping layer of the repository as expected.

Fact aggregate

Dimension aggregate


41/67

12. Verify Aggregates in the Database

Verify that the aggregates are created in the database.


42/67

13. Verify Results in Answers

Activate the aggregate tables:

Run a query in Answers:

Check the log and verify that the aggregate tables are accessed

as expected:


43/67

Considerations

Using aggregates comes with a price:Additional time is required to build and load these tables.

Additional storage is necessary.

Build only the aggregates you need:

Look at query patterns and build aggregates to speed upcommon queries that require summarized results.

Ensure that enough data is combined to offset the cost of

building aggregates.

Monitor and adjust to account for changing query patterns.


44/67

Using Partitions and Fragments


45/67

Business Challenge

Data is often partitioned into multiple physical sources for asingle logical table.

Organizations need to seamlessly and efficiently access and

process data from multiple sources to satisfy user requests.

Business applications must know where to go for what type of

data and under what conditions.


46/67

Business Solution: Oracle BI Server

Oracle BI repository can be configured so that Oracle BI Serverhandles the navigation to the appropriate source.

Oracle BI Server seamlessly and efficiently accesses and

processes data from multiple sources to satisfy user requests.


47/67

Partition

Is a database element that contains part of the data for a fact or adimension

Combines with other data fragments as necessary

May be:

Fact-based

Value-based

Level-based

Complex


48/67

Partitioning by Fact

Data is partitioned by fact when different fact data is stored indifferent tables.

Example: Actual sales versus quota targets

1000

Product

Sale

1100

Sales Rep

2000

Product

Quota

1100

Sales Rep

Actual sales Quota targets


49/67

Partitioning by Value

Data is partitioned by value when the data is split into separatetables according to the values of the data.

Example: Invoice data is stored separately for each region.

1000

Dollars

Central1135293

RegionInvNbrInvoices for

Central

Region

Invoices for

West Region200

Dollars

West114444

RegionInvNbr


50/67

Partitioning by Level

Data is partitioned by level when the same facts are stored inseparate tables at different levels of aggregation.

Example: Detailed sales data is summarized and stored by

year and region.

19981010

19981001

19980105

Date

100001100

250001100

10000

Product

Sale

1100

Sales Rep

Sales detailed data Sales by year and region

Central3000001999

200000

Total

Dollars

Central1998

RegionYear


51/67

Complex Partitioning

Data is partitioned using more than one technique. Example: Invoice sales data is partitioned by value and level.

Central25000199802

10000

Total

Dollars

Central199801

RegionMonth

Invoices by month for Central Invoices by year for Central

200000

Total

Dollars

Central1998

RegionYear

Invoices by month for West Invoices by year for West

West350000199802

300000

Total

Dollars

West199801

RegionMonth

3000000

Total

Dollars

West1998

RegionYear


52/67

ABC Example: Fact-Based (Quota)

Allow users to query for actual sales data and quota datain a single query.

1000

Product

Sale

1100

Sales

Rep

2000

Product

Quota

1100

Sales

Rep

Actual sales Quota targets


53/67

ABC Example: Value-Based (Customer)

Replace the current, single source for customer datawith two value-based partitions.

Clifton LunchName

1000NewKey

Customers

with namesstarting with

letters AM

Customers

with namesstarting with

letters NZ Tongs WokName

1002NewKey


54/67

ABC Example: Value-Based (Inventory)

Build a business model for inventory data that isfragmented into multiple tables.

Eight quarters of

inventory data

stored in eight

separate tables


55/67

Implementation Steps

Import physical sources.

Create physical joins.

Add sources to the Business Model and Mapping layer.

Specify fragmentation content.

Test the results.

New step


56/67

Specify Fragmentation Content

Use the Expression Builder to define the type of content that thefragment contains.

Set the flag to specify whether to combine this fragment withother data.

Specifies that customer data

from AM is contained in

this fragment

Specifies combining this

fragment with the other

S


57/67

Summary

Identify reasons for segmenting data and

implement itbenefit is promised!

Question:

The end user needs to know what data is contained in which

partition before submitting a query. True or false?


58/67

Real-time BIFederated OLTP/EDW Reporting

H i t l F d ti f R l Ti BI


59/67

Horizontal Federation for Real-Time BI

Using horizontal fragmentation in OBIEE, we can mapa single logical fact table to multiple LTSs.

Physical fact table in our EDW called SALES_FACT

Physical fact table in our source system to represent

real-time data

One logical fact table in the BMM called Sales

Fact Realtime



60/67


Fact table mapping



61/67


Customer dimension mapping



62/67


Use variable as the threshold between reporting againstthe EDW schema and the source system schema



63/67


LTS mapping using threshold date

Hori ontal Federation for Real Time BI


64/67


OBIEE does

the logical

union

Horizontal Federation for Real Time BI


65/67


When the BI Server has enough information to

know that the entire result set will come from a

single source, then the SQL will be issued against

only one of the LTSs.


66/67

Oracle by Example and Relevant Blogs


67/67

Oracle by Example and Relevant Blogs

Web tutorial :Oracle by ExampleFederating Essbase and Relational Data Sources in OBIEE

http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/

http://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdf

http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/
http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/

Date post:	12-Oct-2015
Category:	Documents
Upload:	amoyeed
View:	20 times
Download:	0 times

obiee-data-federation1 (1)

Documents