5/21/2018 obiee-data-federation1 (1)
1/67
Limor Fledel-VagmanBI Practice Manager
Oracle Israel
OBIEE Data Federation
5/21/2018 obiee-data-federation1 (1)
2/67
OBIEE Federation
Vertical Federation
Integrating two or more disparate data sou rce having dif ferent
levelsof granularity joined by one or more conformed dimensions
Example: Essbase Sales cube to relational Sales detail data (drill-
through)
Horizontal Federation
Integrating two or more disparate data sou rce having the same
level of granularity joined by one or more conformed dimensions Example: Essbase Sales cube with HR relational database
5/21/2018 obiee-data-federation1 (1)
3/67
LTS ???
Logical table sources (LTSs) are a key feature
within the OBIEE semantic model
5/21/2018 obiee-data-federation1 (1)
4/67
OBIEE Aggregate Navigation
5/21/2018 obiee-data-federation1 (1)
5/67
Why Aggregate Navigation?
Problem: Summary questions are slow
Example, All sales worldwide for last 2 years by quarter by family
This will potentially sum/group by millions of rows!
The disk spin time to scan all these rows will be very long
Solution: ETL to batch build high level aggregates
Oracle BI EE has aggregate navigation to use them
Aggregates can be in the same database as the source
They can also be in a different relational source or Essbase
Often orders of magnitude performance improvement
5/21/2018 obiee-data-federation1 (1)
6/67
Oracle BI EE Architecture
RelationalTables
BI Server
Semantic
LayerMetadata
5/21/2018 obiee-data-federation1 (1)
7/67
Oracle BI EE Aggregate Navigation
RelationalDetail
Summary
Semantic
LayerMetadata
Metadata
describes
aggregate
mappings
At query time, BI Server queries
the fastest source that has
enough detail to satisfy the user
request
Performance of highly
summarized requests isdramatically improved
BI Server
5/21/2018 obiee-data-federation1 (1)
8/67
Federation Enables External Aggregates
RelationalDetail
Summary
Semantic
LayerMetadata
Summary
Aggregates can be in
other federated
sources
BI Server
5/21/2018 obiee-data-federation1 (1)
9/67
External Aggregates Can Be in Essbase
RelationalDetail
Summary
Essbase
Semantic
LayerMetadata Aggregate navigation
can include
multidimensional
BI Server
5/21/2018 obiee-data-federation1 (1)
10/67
External Aggregates Can Be Oracle OLAP
RelationalDetail
Summary
Oracle
OLAP AW
Semantic
LayerMetadata Aggregate navigation
can include
multidimensional
BI Server
"This information is not a commitment to deliver any material, code, or functionality. The development,
release, and timing of any features or functionality described remains at the sole discretion of Oracle"
5/21/2018 obiee-data-federation1 (1)
11/67
How is Aggregate Navigation Set Up?
Database Design time:
DBA defines aggregate stars at various grain combinations
Builds ETL to load the aggregates each night
Star Time Product
Base Day SKUAgg 1 Month Brand
Agg 2 Qtr -
5/21/2018 obiee-data-federation1 (1)
12/67
Mapping a Base-Level Physical Star
Business Model and
Mapping Layer
Physical LayerMappings
D0 Time
D4 Product
F0 Rev Base Measures
D_Day
D_Prod_SKU
F_Bill
5/21/2018 obiee-data-federation1 (1)
13/67
Adding a Moderately Aggregated Star
Business Model and
Mapping Layer
D0 Time
D4 Product
F0 Rev Base Measures
D_Day
D_Prod_SKU
F_Bill
D_Month
D_Brand
F_Bill_agg2
Physical LayerMappings
5/21/2018 obiee-data-federation1 (1)
14/67
Adding a Highly-Aggregated Star
Business Model and
Mapping Layer
D0 Time
D4 Product
F0 Rev Base Measures
D_Day
D_Prod_SKU
F_Bill
D_Month
D_Brand
F_Bill_agg2
D_Qtr
F_Bill_agg1
Physical LayerMappings
5/21/2018 obiee-data-federation1 (1)
15/67
Essbase as an Aggregate Source
Business Model and
Mapping Layer
D0 Time
D4 Product
F0 Rev Base Measures
D_Day
D_Prod_SKU
F_Bill
Physical LayerMappings
Month
Brand
Billed Amt
5/21/2018 obiee-data-federation1 (1)
16/67
Adding a Highly-Aggregated Star
Business Model and
Mapping Layer
D0 Time
D4 Product
F0 Rev Base Measures
D_Day
D_Prod_SKU
F_Bill
D_Month
D_Brand
F_Bill_agg2
D_Qtr
F_Bill_agg1
How does the query
planner decide which
FJC to use as its sourcefor a given request?
Physical LayerMappings
5/21/2018 obiee-data-federation1 (1)
17/67
Selecting an Aggregate or Base Star at Query Time
How does the query planner decide which star to use
as its source for a given LSQL request?
First, it eliminates sources that dont have enough
detail to answer the question
Query grain star grain star qualified
Second, it estimates the fastest/prioritized source
5/21/2018 obiee-data-federation1 (1)
18/67
How BI Server Knows the Grain of a Star
At query time, the BI Server checks the available
mappings to find which ones are qualified by grain
5/21/2018 obiee-data-federation1 (1)
19/67
How Does It Know Which Star is Fastest?
Number of elements
Be sure to correctly populate the
Dimension Levels Number of
elements at this level
OBIEE 10g
5/21/2018 obiee-data-federation1 (1)
20/67
How Does It Know Which Star is Fastest?
LTS Priority Group Order
LTS Priority Group Order
Lower value = higher priority
LTS Priority Group becomes
main decider in which LTS to
use
OBIEE 11g New Feature
5/21/2018 obiee-data-federation1 (1)
21/67
Aggregate Persistence
5/21/2018 obiee-data-federation1 (1)
22/67
Aggregate Persistence Feature
Managing aggregates manually has high TCO
Oracle BI EE automates creation of aggregates
Leverages existing metadata
Orders of magnitude performance boost; low TCO
5/21/2018 obiee-data-federation1 (1)
23/67
Aggregate Persistence Automates Create & Load
Define agg
navigation
metadata
Detail
Summary
Semantic
LayerMetadata
Admin Tool:
Aggregate
build
automation
wizard
Run scripts
Create Tables
Aggregation
BI Server
5/21/2018 obiee-data-federation1 (1)
24/67
Aggregate PersistenceManual Option
Detail
Summary
Semantic
LayerMetadata
Admin Tool:
Aggregate
build
automation
wizard
Create scripts
Create Table DDL
Aggregation
Run scripts
Create Tables
Aggregation
DBA
BI Server
Define agg
navigation
metadata
5/21/2018 obiee-data-federation1 (1)
25/67
Aggregate Persistence with Essbase & Oracle OLAP
Detail
Semantic
LayerMetadata
Admin Tool:
Aggregate
build
automation
wizard
Define cube schema
Summary
Essbase or
AW
BI Server
Define agg
navigation
metadata
"This information is not a commitment to deliver any material, code, or functionality. The development,
release, and timing of any features or functionality described remains at the sole discretion of Oracle"
5/21/2018 obiee-data-federation1 (1)
26/67
Administration Steps
Design aggregate facts and grains
Wizard to create metadata
1 Target database for create/load
3
2
5/21/2018 obiee-data-federation1 (1)
27/67
Aggregate Persistence Wizard
5/21/2018 obiee-data-federation1 (1)
28/67
Aggregate Persistence Wizard
Automates the creation of physical aggregate tables andtheir corresponding objects in the repository
Use wizard to build
script to generate
aggregate tables.Script generates
physical tables,
repository objects,
and mappings.
Fact
aggregate
Dimension
aggregates
5/21/2018 obiee-data-federation1 (1)
29/67
Aggregate Persistence Wizard Steps
1. Open Aggregate Persistence Wizard.2. Specify the file name and location.
3. Select business model and measures.
4. Select dimensions and levels
5. Select connection pool, container, and name.
6. Review aggregate definition.7. View complete aggregate script.
8. Verify script is created.
9. Create and run a batch file.
10. Verify aggregates in the Physical layer.
11. Verify aggregates in the BMM layer.
12. Verify aggregates in the database.
13. Verify results in Answers.
5/21/2018 obiee-data-federation1 (1)
30/67
1. Open Aggregate Persistence Wizard
Select Tools > Utilities > Aggregate Persistence Wizardand click the Execute button.
5/21/2018 obiee-data-federation1 (1)
31/67
2. Specify File Name and Location
Specify a file and location where the output script shouldbe saved.
5/21/2018 obiee-data-federation1 (1)
32/67
3. Select Business Model and Measures
Select business model.
Select fact table.
Select measures.
5/21/2018 obiee-data-federation1 (1)
33/67
4. Select Dimensions and Levels
Select corresponding aggregate dimensions and levels.
5/21/2018 obiee-data-federation1 (1)
34/67
5. Select Connection Pool, Container, and Name
Select the database object.
Select the schema.
Select the connection pool.
Name the aggregate table.
5/21/2018 obiee-data-federation1 (1)
35/67
6. Review Aggregate Definition
View the aggregate
definition.
5/21/2018 obiee-data-federation1 (1)
36/67
7. View Complete Aggregate Script
Confirmation of scriptcreation and location
Script
5/21/2018 obiee-data-federation1 (1)
37/67
8. Verify that the Script Is Created
Navigate to the directory where the file was saved andverify that the script was created as expected.
5/21/2018 obiee-data-federation1 (1)
38/67
9. Create and Run a Batch File
Create and run a batch file with the following format todrive the aggregate creation process:
nqcmd Oracle BI Server command utility
-d Oracle BI Server data
source name-u Repository username
-p Repository password
-s Path to the create
aggregate SQL script
5/21/2018 obiee-data-federation1 (1)
39/67
10. Verify Aggregates in the Physical Layer
Verify that the aggregates are created in the Physicallayer of the repository as expected.
Fact aggregate
Dimension aggregates
5/21/2018 obiee-data-federation1 (1)
40/67
11. Verify Aggregates in the BMM Layer
Verify that the aggregates are created in the BusinessModel and Mapping layer of the repository as expected.
Fact aggregate
Dimension aggregate
5/21/2018 obiee-data-federation1 (1)
41/67
12. Verify Aggregates in the Database
Verify that the aggregates are created in the database.
5/21/2018 obiee-data-federation1 (1)
42/67
13. Verify Results in Answers
Activate the aggregate tables:
Run a query in Answers:
Check the log and verify that the aggregate tables are accessed
as expected:
5/21/2018 obiee-data-federation1 (1)
43/67
Considerations
Using aggregates comes with a price:Additional time is required to build and load these tables.
Additional storage is necessary.
Build only the aggregates you need:
Look at query patterns and build aggregates to speed upcommon queries that require summarized results.
Ensure that enough data is combined to offset the cost of
building aggregates.
Monitor and adjust to account for changing query patterns.
5/21/2018 obiee-data-federation1 (1)
44/67
Using Partitions and Fragments
5/21/2018 obiee-data-federation1 (1)
45/67
Business Challenge
Data is often partitioned into multiple physical sources for asingle logical table.
Organizations need to seamlessly and efficiently access and
process data from multiple sources to satisfy user requests.
Business applications must know where to go for what type of
data and under what conditions.
5/21/2018 obiee-data-federation1 (1)
46/67
Business Solution: Oracle BI Server
Oracle BI repository can be configured so that Oracle BI Serverhandles the navigation to the appropriate source.
Oracle BI Server seamlessly and efficiently accesses and
processes data from multiple sources to satisfy user requests.
5/21/2018 obiee-data-federation1 (1)
47/67
Partition
Is a database element that contains part of the data for a fact or adimension
Combines with other data fragments as necessary
May be:
Fact-based
Value-based
Level-based
Complex
5/21/2018 obiee-data-federation1 (1)
48/67
Partitioning by Fact
Data is partitioned by fact when different fact data is stored indifferent tables.
Example: Actual sales versus quota targets
1000
Product
Sale
1100
Sales Rep
2000
Product
Quota
1100
Sales Rep
Actual sales Quota targets
5/21/2018 obiee-data-federation1 (1)
49/67
Partitioning by Value
Data is partitioned by value when the data is split into separatetables according to the values of the data.
Example: Invoice data is stored separately for each region.
1000
Dollars
Central1135293
RegionInvNbrInvoices for
Central
Region
Invoices for
West Region200
Dollars
West114444
RegionInvNbr
5/21/2018 obiee-data-federation1 (1)
50/67
Partitioning by Level
Data is partitioned by level when the same facts are stored inseparate tables at different levels of aggregation.
Example: Detailed sales data is summarized and stored by
year and region.
19981010
19981001
19980105
Date
100001100
250001100
10000
Product
Sale
1100
Sales Rep
Sales detailed data Sales by year and region
Central3000001999
200000
Total
Dollars
Central1998
RegionYear
5/21/2018 obiee-data-federation1 (1)
51/67
Complex Partitioning
Data is partitioned using more than one technique. Example: Invoice sales data is partitioned by value and level.
Central25000199802
10000
Total
Dollars
Central199801
RegionMonth
Invoices by month for Central Invoices by year for Central
200000
Total
Dollars
Central1998
RegionYear
Invoices by month for West Invoices by year for West
West350000199802
300000
Total
Dollars
West199801
RegionMonth
3000000
Total
Dollars
West1998
RegionYear
5/21/2018 obiee-data-federation1 (1)
52/67
ABC Example: Fact-Based (Quota)
Allow users to query for actual sales data and quota datain a single query.
1000
Product
Sale
1100
Sales
Rep
2000
Product
Quota
1100
Sales
Rep
Actual sales Quota targets
5/21/2018 obiee-data-federation1 (1)
53/67
ABC Example: Value-Based (Customer)
Replace the current, single source for customer datawith two value-based partitions.
Clifton LunchName
1000NewKey
Customers
with namesstarting with
letters AM
Customers
with namesstarting with
letters NZ Tongs WokName
1002NewKey
5/21/2018 obiee-data-federation1 (1)
54/67
ABC Example: Value-Based (Inventory)
Build a business model for inventory data that isfragmented into multiple tables.
Eight quarters of
inventory data
stored in eight
separate tables
5/21/2018 obiee-data-federation1 (1)
55/67
Implementation Steps
Import physical sources.
Create physical joins.
Add sources to the Business Model and Mapping layer.
Specify fragmentation content.
Test the results.
New step
5/21/2018 obiee-data-federation1 (1)
56/67
Specify Fragmentation Content
Use the Expression Builder to define the type of content that thefragment contains.
Set the flag to specify whether to combine this fragment withother data.
Specifies that customer data
from AM is contained in
this fragment
Specifies combining this
fragment with the other
S
5/21/2018 obiee-data-federation1 (1)
57/67
Summary
Identify reasons for segmenting data and
implement itbenefit is promised!
Question:
The end user needs to know what data is contained in which
partition before submitting a query. True or false?
5/21/2018 obiee-data-federation1 (1)
58/67
Real-time BIFederated OLTP/EDW Reporting
H i t l F d ti f R l Ti BI
5/21/2018 obiee-data-federation1 (1)
59/67
Horizontal Federation for Real-Time BI
Using horizontal fragmentation in OBIEE, we can mapa single logical fact table to multiple LTSs.
Physical fact table in our EDW called SALES_FACT
Physical fact table in our source system to represent
real-time data
One logical fact table in the BMM called Sales
Fact Realtime
H i t l F d ti f R l Ti BI
5/21/2018 obiee-data-federation1 (1)
60/67
Horizontal Federation for Real-Time BI
Fact table mapping
H i t l F d ti f R l Ti BI
5/21/2018 obiee-data-federation1 (1)
61/67
Horizontal Federation for Real-Time BI
Customer dimension mapping
H i t l F d ti f R l Ti BI
5/21/2018 obiee-data-federation1 (1)
62/67
Horizontal Federation for Real-Time BI
Use variable as the threshold between reporting againstthe EDW schema and the source system schema
H i t l F d ti f R l Ti BI
5/21/2018 obiee-data-federation1 (1)
63/67
Horizontal Federation for Real-Time BI
LTS mapping using threshold date
Hori ontal Federation for Real Time BI
5/21/2018 obiee-data-federation1 (1)
64/67
Horizontal Federation for Real-Time BI
OBIEE does
the logical
union
Horizontal Federation for Real Time BI
5/21/2018 obiee-data-federation1 (1)
65/67
Horizontal Federation for Real-Time BI
When the BI Server has enough information to
know that the entire result set will come from a
single source, then the SQL will be issued against
only one of the LTSs.
5/21/2018 obiee-data-federation1 (1)
66/67
Oracle by Example and Relevant Blogs
5/21/2018 obiee-data-federation1 (1)
67/67
Oracle by Example and Relevant Blogs
Web tutorial :Oracle by ExampleFederating Essbase and Relational Data Sources in OBIEE
http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/
http://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdf
http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/
http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://www.rittmanmead.com/2011/05/real-time-bi-federated-oltpedw-reporting/http://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://odtugspconference.com/Slides/OBIEE%20Integration%20with%20Essbase.pdfhttp://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/http://www.artofbi.com/index.php/2009/07/federation-in-obiee-what-are-you-talking-about/