Date post: | 17-Jun-2015 |
Category: |
Technology |
Upload: | oracle-analytics |
View: | 2,538 times |
Download: | 1 times |
Extreme BI with Oracle Exadata
and Business Intelligence Apps
October 4, 2011
Presented by: Martin Paynter - Enkitec Brad Salva – Southwestern Energy
Who am I • Martin Paynter – BI Practice Manager
• EBS Developer since 2001 • 16 EBS implementations
• Business Intelligence Developer since 1999 • OWB • ODI • Informatica • Hyperion IR • OBIEE – BI apps • Discoverer
3
Who is Enkitec • Oracle-centric Consulting Partner with a broad set of DBA & application
development experience
• Database, application development, Oracle training
• Database consultants averaging 15+ years Oracle experience
• Exadata Specialized Partner status (one of a handful globally)
• Dedicated, In-house Exadata Lab (POV, Patch Validation)
• 60+ Successful Exadata Implementations
• Enkitec Europe – Partnering with Tanel Poder
4
What is Exadata? � Revolutionary Approach to Oracle Database Processing
� Database Servers, Storage and Network in a Single Enclosure
� High Bandwidth Network Connects Database and Storage Servers
� Intelligent Storage Filters Data and Minimizes Network Traffic
� Optimized for Both Data Warehouse and OLTP Activity
5
What is Exadata?
6
Exadata Storage Servers Exadata Database Servers
11gR2 / ASM iDB / RDS
cellsrv
How is Exadata Different?
7
The Bottleneck on Many (Most) Large Databases is between the Disk and the DB Server(s)! How to Speed it Up? Make the Pipe Bigger/Faster Reduce Traffic on the Pipe
* The fast way to do anything is not to do it!
8
Offloading – The “Secret Sauce” Offloading vs. Smart Scan (what’s the difference) Offloading – generic term meaning doing work at the storage layer instead of at the database layer Smart Scan – query optimizations covered by “cell smart table/index scan” wait events
Traditional Architecture vs Exadata
9
Sun e20K 2-Node RAC
48 cores each
Enterprise Class SAN
Batch Process
Execution Time:
24 Hours +
Exadata Quarter
Rack
2-Node RAC 8 cores each
Exadata Storage
Batch Process
Execution Time:
45 Minutes
Traditional Exadata
Execut ion Time Improved by 32x
Business Benefits of Exadata � Pre-configured - Ready for Production in Weeks, not Months
� Single Source for Support – Hardware / Storage / Network
� Extreme Performance*
� No Need to Modify Oracle-based Applications
� Highly Scalable
� Optimized for Mixed Workload Environments
10
Who is Using Exadata? � Healthcare Providers
� Market Research Companies
� Retail (Grocer)
� Logistics
� Government – State and Federal
� Mortgage Processing
� Oil and Gas
� Document Processing Outsourcing
11
Who am I • Brad Salva - Lead DBA (Southwestern Energy)
• Team of 6 DBA’s (managing SQL Server and Oracle instances)
• Database Administration since 1997 • Oracle and SQL Server • 12 years in the Energy Industry • Lead DBA at Southwestern Energy since 2006
12
Who is SWN • Southwestern Energy Company (SWN) is a growing
independent energy company primarily engaged in natural gas and crude oil exploration, development and production within North America.
• We are also focused on creating and capturing additional value through our natural gas gathering and marketing businesses, which we refer to as Midstream Services.
• www.swn.com
13
SWN EBS Project • Started E-business implementation in 2010 to replace legacy
systems. Project includes Oracle E-business R12, P2ES Tobin Enterprise Land, P2ES Energy Upstream, P2ES Wellcore, OBIEE, Hyperion, Oracle UCM, Single sign-on, and more.
• 16 EBS instances
• 7 OBIEE instances
• 12 Hyperion instances
• 2 Standby instances
14
Why SWN Chose Exadata
� Lower TCO � Exadata v. “Traditional Hardware” v. Hosted
� Performance
� Easy Installation/Configuration
� High Availability/Redundancy
� Integrated easy to maintain tech stack
15
SWN EBS Production � Apps Tier (4 Servers – 2 internal, 2 DMZ)
� OS: OEL5
� Memory: 17GB � CPU: Intel Xeon X5670 @ 2.93GHz X 4
� Concurrent Processing Tier ( 2 Servers) � OS: OEL5
� Memory: 17GB � CPU: Intel Xeon X5670 @ 2.93GHz X 4
� DB Tier (RAC – 4 node) � OS: OEL5
� Memory: 96GB per node � CPU: Intel Xeon X5670 @ 2.93GHz X 24
� 8 GB SGA
16
Client X EBS Production � Apps Tier/ (1 Server)
� OS: RHEL 5.5
� Memory: 64GB
� CPU: Intel Xeon X5670 @ 2.93GHz X 12
� Concurrent Processing Tier (1 server) � OS: RHEL 5.5
� Memory: 64GB
� CPU: Intel Xeon X5670 @ 2.93GHz X 12
� DB Tier (RAC – 2 node) � OS: RHEL 5.5
� Memory: 64GB per node
� CPU: Intel Xeon X5670 @ 2.93GHz X 12
� Version: 11.2.0.2
� 8 GB SGA
17
SWN BI Production � BI Apps Tier (VM)
� OS: Windows Server 2003 R2 – x64 � Memory: 16GB � CPU: Intel Xeon X5670 @ 2.93GHz X 4
� BI Database Tier (Exadata 4 Node RAC) � OS: OEL5 � Memory: 96GB per node � CPU: Intel Xeon X5670 @ 2.93GHz X 24 � Version: 11.2.0.2 � 8 GB SGA
18
Client X BI Production � BI Apps Tier (VM)
� OS: Windows Server 2008 – x64 � Memory: 16GB � CPU: Intel Xeon X5670 @ 2.93GHz X 4
� BI Database Tier (2 Node RAC) � OS: RHEL 5.5 � Memory: 64GB per node � CPU: Intel Xeon X5670 @ 2.93GHz X 12 � Version: 11.2.0.2 � 8 GB SGA
19
Typical BI Considerations � Source System
� Availability � Performance
� ETL � Architecture installation/configuration � Development � Performance
� Visualization � Architecture installation/configuration � Dashboard/Report development
20
SWN/Client X BI Platform � Source System
� Oracle EBS R12.1.2 � 11.2.0.2 4 node RAC Database (Client X on 2 Node)
� ETL � Informatica 9.0.1 Hotfix 2 � Oracle BI Applications 7.9.6.3 � DAC 10.1.3.4 � 11.2.0.2 4 node RAC Database (Client X on 2 Node)
� Visualization � OBIEE 11.1.1.5 � Weblogic 10.3.5.0
21
Areas of Focus � Implementation
� Exadata configuration � Installation of Oracle RAC databases � Create EBS clones and backups � Install/configure BI Technology stack
� Performance � EBS on Exadata � Informatica/DAC on Exadata � OBIEE/OBI Apps on Exadata
22
� Configure Exadata and 4 node RAC DB (40 hours) � Number of DB instances on Exadata ½ rack – 16
� Production EBS 4 node RAC � All other installs single node
� Time to Backup – 1 minute 30 seconds � RMAN backup, high compression, 16 channels to ASM +RECO disk
group � Netapp backup takes 30 minutes � Restore from backup takes 10 minutes
� Time to Clone – 10 minutes
SWN EBS Implementation Metrics
23
SWN BI Implementation Metrics � OBIEE installed/configured in 8 hours
� Weblogic � Enterprise Manager � BI Server � Presentation Server � Admin Tools
� Informatica and DAC installed/configured in 8 hours � Powercenter Services and Client � DAC Server and Client
� OBI Apps: � Configured General Ledger subject area and ran initial full load in 24 hours
24
Extreme Performance Data Warehousing Integrated Technology Stack
• Single source of truth
• Easy to deploy and manage
• Extreme performance
• Meets all end user requirements
• Lower cost of ownership Smart Storage
Database
Data Models ELT Tools
BI Tools
BI Applications
Oracle BI Apps Product Components Example: Financial Analytics
Pre-mapped metadata, including embedded best practice calculations and metrics for financial, executives and other business users
A “best practice” library of over 360 pre-built metrics, 30 intelligent dashboards, 200+ reports plus alerts for CFO, Finance Controller, Financial Analyst, AR/AP Managers and Executives
Pre-built ETL to extract data from over 3,000 operational tables and load it into the DW, sourced from PSFT, Oracle EBS and other sources
Pre-built warehouse with 16 star-schemas designed for analysis and reporting on financial analytics
• Presentation layer • Logical business
model • Physical sources
1 3
2 4
Performance � EBS on Exadata
� GL Journal Load
� Informatica/DAC on Exadata � Extract from EBS � General Ledger subject area load into OBI DW
� OBIEE/OBI Apps on Exadata � General Ledger Dashboard/report processing against OBI DW
27
Tuning Options � OBIEE
� Aggregate navigation
� Presentation cache
� BI Server cache
� Informatica/DAC on Exadata
� Commit frequency
� DTM Buffer size
� Indexes
� Integration Services
� OBI DW
� Partitioning
� Parallelism
� Compression
� OLAP
� Materialized views
28
EBS Journal Load
SWN and Client X use the gl_interface table to load in ~3,000,000 journal lines per day
29
Task # Rows SWN Time Client X Gain
Load to gl_interface from external table
~3,000,000 3 min 40 min 13X
Import Journals ~3,000,000 15 min 7 hrs 28X
Post Journals 10 min 2 hrs 12X
Total Time Spent 28 min ~10 hrs 21X
OBI General Ledger Subject Area
� 268 Tasks
� Over 600 bitmap indexes
� Two main fact tables: � W_GL_OTHER_F – GL Transactions � W_GL_BALANCE_F – GL Balances
30
OBI DW Load � Full load for General Ledger subject area with creation of
indexes ~ 40,000,000 in 4 hours
� Incremental loads with indexes ~ 3,000,000 in 2 hours
� Low effort tuning: � Identify long running tasks and change DTM Buffer and commit
interval in PowerCenter Workflow Manager � Increase number of Integration Services � Confirm Bulk loading is occurring (/*+ SYS_DL_CURSOR*/) � Drop bitmap indexes in DAC
31
Informatica Performance
� DTM Buffer Size � Changed default of 32MB to Auto
� Target Commit Frequency � For high volume loads changed commit frequency from
10,000 to 1,000,000
� Make sure direct loading is occurring: � /*+ SYS_DL_CURSOR*/ in v$sql
32
ETL – Extraction from EBS
33
EBS on Exadata provides an edge during the extraction process.
� SDE_ORA_GLJournals (2,995,320)
� SWN – 1 minute (All time spent on writing)
� Client X – 10 minutes (50/50 read and write)
SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE ------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- fjh00b5mzq64c 1 11.61 Yes 5,917,486,584 10,090,102,784
DAC Performance
� Increase number of Integration Services � Evaluated the DAC execution plan and determined an
increase from 10 to 30 was optimal
34
DAC Performance
� Drop bitmap index creation � Saves 46 minutes for SWN, but allows for partitioning
without additional configuration as well as facilitates smart scan query processing
Before Dropping indexes: After Dropping indexes:
35
DAC Performance
� Client X spends 82 minutes creating indexes during incremental load. It is not an option for them to drop indexes because queries would never return.
36
OBI DW Load Times - Total
37
0 2 4 6 8 10 12
Client X
SWN OOB
SWN Tuned
Time - Hours
Time
OBI DW Load Times – Top Tasks
38
TIME (minutes)
Task Client X SWN OOB SWN Tuned Gain
SDE_ORA_GLJournals 18 3.5 1 18X
SDE_ORA_GLBalanceFact 15 2 2 8X
SDE_ORA_Stage_GLJournals_Derive 20 2 1 20X
SDE_ORA_STAGE_GLOtherFact_Derive 28 8 1.5 19X
SIL_GLOtherFact 35 8 6.5 5X
SIL_GLBalanceFact 38 8 6.5 6X
PLP_GLBalanceAggrByAcctSegCodes 62 21 11 6X
OBI DW Load – Incremental Tuned
� Top Task Metrics – Incremental load Tuned (32 minutes)
39
Task # Rows Time (min) Throughput/min SDE_ORA_GLJournals 2,995,320 1 2,995,320
SDE_ORA_GLBalanceFact 3,174,070 2 1,587,035
SDE_ORA_Stage_GLJournals_Derive 2,995,320 1 2,995,320
SDE_ORA_STAGE_GLOtherFact_Derive 2,995,320 1.5 1,330,213
SIL_GLOtherFact 990,404 6.5 152,369
SIL_GLBalanceFact 3,174,070 6.5 488,319
PLP_GLBalanceAggrByAcctSegCodes 18,666,814 11 1,116,676
OBI DW Query Performance
How will my queries perform with… • No Bitmaps • Parallelism
• Partitioning • Compression
40
Look ma, no bit maps
41
� Every query is different, however there is no negative impact to performance overall (Client X ~ 10 min)
SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE IO Saved ------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- ------------ 5ycwp44p7crmg 1 13.61 Yes 1,173,569,000 27,537,399,808 95.74%
But is it faster…
42
� Query execution with bit maps in place:
SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE IO Saved ------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- ------------ 5m8q3bkuyrguk 1 439.25 NO 0 0 0%
But is it faster…
43
� Query execution with no bitmaps:
SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE IO Saved ------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- ------------ Drdwhzcsm4cyf 1 65.98 Yes 1,264,079,768 28,597,270,528 95.58%
Parallelism
44
Query dropped from 3 min to 8 seconds (Client X – 20 min)
Parallelism
45
� Where can we add parallelism: � OBIEE Analysis report � OBIEE RPD physical table (previous slide) � Informatica source definition � OBI DW tables
Partitioning � Our desire is to get direct path reads in order to take advantage
of Smartscan functionality � Risk size of partition being too small
� We now have the option of partitioning without having to spend time adding in DAC Actions
� Partitioned W_GL_OTHER_F and W_GL_BALANCE_F by period � Load time stayed at 32 minutes
� Mixed query performance results depending on size of partitions
46
Compression
47
� Now that we have the tables partitioned, we can compress historical data that is no longer being actively updated: � W_GL_OTHER_FACT SEP11 partition HCC Query High:
� Before HCC ~ 23 GB, query execution 11.61 seconds � After HCC ~ 699 MB, query execution 8.24 seconds
Compress ion of 33x
Exadata Capabilities – Wrap-up
48
• Speed to Market • Easy Transition from Legacy Systems to Exadata
• Simple and Accurate Application Migration
• Consolidation Platform
• Extreme Performance
49
The Kübler-Ross grief cycle
Exposure to Exadata
50
Expert Oracle Exadata, co-authored by Kerry Osborne, Randy Johnson & Tanel Poder to be published July 18, 2011
Visit Enkitec at Oracle OpenWorld at Moscone Center in San Francisco from October 2 – 6 at Booth 1721
51
Questions?
Fastest Growing Companies in Dallas
Contact Information : Martin Paynter
[email protected] kerryosborne.oracle-guy.com
www.enkitec.com