Real World Performance Benchmarking

transcript

Real World Performance Benchmarking Oracle Transportation Management/Global Trade Management August 2012

Safe Harbor Statement

• The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Agenda

• Internal Exa-data/Exa-logic Certification Testing • Customer Exa-data/Exa-logic POC • Other Performance Activities • Performance Benchmarking Advice

Exalogic/Exadata Performance Update

• The OTM Exadata/Exalogic performance comparisons were captured as part of the OTM Certification testing for Exadata/Exalogic. These tests were not intended to be a true benchmark for directly comparing Exa/non-Exa hardware alternatives. In the future OTM may perform a true set of benchmark tests to capture this data.

OTM/GTM Exadata/Exalogic Certification • Certification Scenarios

• Basic Regression Test • Multi-stop Planning • Multi-leg Planning • “End-End” Orders through Shipments • FTI Initial ETL using ODI

• Baseline OTM 6.2.5 • Exadata/Exalogic OTM 6.2.6

OTM Certification Deployment Exalogic and Exadata

15X 5X

OTM WebTier

Web Server

OTM App Tier

App Server

Exalogic

Certification Scenario #1 • Certification of OTM version 6.2 • Test Description

• Multi-leg Planning • 5,000 Orders • Building into 20,000 Shipments

• OTM Settings • 120 Planning Threads • 16 GB Heap

Certification Scenario #1 Hardware Comparison Database EXADATA

Hardw are Exadata X2-2 Half Rack (4 nodes, 48 physical cores) 2x Intel Xeon, 3 GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2

Storage

4x 512GB SSD for read cache (“Readzilla”, 2TB total) 4x 18GB SSD in disk shelf for w rite cache 20x 2TB 7200 rpm SAS HDD in disk shelf – RAID-1

Application Server

EXALOGIC

Hardw are Exalogic X2-2 (1 node, 12 physical cores) 2x Intel Xeon, 3GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Weblogic Server 10.3.4

Database BASELINE

Hardw are Dell 2950 2 x 3.1 GHz processors (4 cores), 32GB RAM

Softw are OEL 5.5 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2.

Storage

Direct Attached Storage 8 arrays 800GB each RAID 10=800GB

Application Server

BASELINE

Hardw are Dell 2950 2 x 3.1 GHz processors (4 cores), 32GB RAM

RuntimeRuntime

Certification Scenario #1 Exalogic and Exadata for OTM 6.2

• Improved Engine Performance: – 4.7x Faster Bulk Plan Runtime*

• Higher Through-put – 8x Improvement in Shipment Persistence* – 7.5 ms per shipment

* Some changes made could benefit baseline scenario

OTM Shipment Persistence 8x Faster Performance

OTM Bulk Plan

4.7x Faster Performance

47minutes

10 minutes

Baseline

Exalogic and Exadata

Certification Scenario #2 • Test Description

• 5,000 inbound transmissions, generating workflow resulting in: • 5,000 Order Releases • 10,000 Order Movements • 10,000 Shipments • 10,000 Invoices • 5,000 Tender Offers • 25,000 Outbound Transmissions

• OTM Settings • 36 threads each for various dataqueue and worker thread groups • 16 GB Heap

Certification Scenario #2 Hardware Comparison Database EXADATA

Hardw are Exadata X2-2 Half Rack (4 nodes, 48 physical cores) 2x Intel Xeon, 3 GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2

Storage

4x 512GB SSD for read cache (“Readzilla”, 2TB total) 4x 18GB SSD in disk shelf for w rite cache 20x 2TB 7200 rpm SAS HDD in disk shelf – RAID-1

Application Server

EXALOGIC

Hardw are Exalogic X2-2 (1 node, 12 physical cores) 2x Intel Xeon, 3GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Database BASELINE

Hardw are HP DL360 2 x 2.8 GHz processors (6 cores), 32GB RAM

Softw are OEL 5.6 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2.

Storage

Direct Attached Storage 8 arrays 800GB each RAID 10=800GB

Application Server

BASELINE

Hardw are HP DL360 2 x 2.8 GHz processors (6 cores), 32GB RAM

Runtime

Certification Scenario #2 Exalogic and Exadata for OTM 6.2

OTM Overall Processing Time 9x Faster Performance

2 hr 35 m

16 minutes

• Improved Performance: – 9x reduction in processing time*

* Some changes made could benefit baseline scenario

Baseline

Exalogic and Exadata

Certification Details • Worked closely with

• Java team to fix contentions observed in JRE • Database team to fix an obscure bug in SDP protocol • OEL and Hardware team to measure/tweak OS and hard disk

performance • Of course, OTM team to

• Make ExaLogic specific changes • Certifying OTM with newer releases of the stack (weblogic, jdbc,

OEL, Database)

Future • Explore Advanced Exadata Capabilities

• SDP over InfiniBand – High Performance Network Connection, in lieu of TCP/IP

• Smart Scan – Indexing Feature to Enhance Full Table Scan Performance

• Exadata Hybrid Columnar Compression – Advanced Feature for High Performance Data Compression

• Smart Flash Cache – Flash Memory Cache for Reduced i/o

Agenda

• Internal Exa-data/Exa-logic Certification Testing • Customer Exa-data/Exa-logic POC • Other Performance Activities • Performance Testing Advice

Analyzed Process

• Bulk Plan Order Release : To Buy Side Shipments – Average planning of 6,000 orders in 20 minutes, resulting in 250 shipments

• Hardware

• Software

Produto Server GPA Server PoC

Weblogic Server 10.3.1 10.3.1

OTM 6.1 6.1

Database Server 11gR2 11gR2

JVM Software 1.6 JRockit 1.6 JRockit

SO Software Oracle Linux 5.5 Oracle Linux 5.5

Server Server GPA Server PoC

Application Tier

Intel Linux - 2 x 2 Intel quad core 64 GB (cluster) - 1 x 16 cores 64 GB (fail over)

Exalogic Half Rack - 4 Compute Nodes HA setup - Used just 1 Compute Node*

* 1 Compute Node = 2 Intel 6 cores 96 GB

Database Tier Exadata Quarter Rack - 2 Db Nodes HA

Exadata Quarter Rack - 2 Db Nodes HA

Results

Date Bulk Plan ID Orders Shipments Total Time

22/11/2011 - 08h54m 20111122-0001 3,512 156 5 minutes

21/11/2011 - 19h17m 20111121-0007 5,114 215 9 minutes

22/11/2011 - 11h55m 20111122-0004 10,028 392 22 minutes

22/11/2011 – 20h10m 20111122-0006 15,342 565 32 minutes

• Bulk Plan Order Release : To Buy Side Shipments

Reference from PROD

Orders 6,000

Shipments 250

Total Time 20 minutes

Agenda

Other Performance Activities

• UI Benchmark on Sun OEL – Goal: Prove linear scalability on web tier for UI processing – Result: Achieved linear scalability, response degraded after 300 auto

users, linearity returned by adding a second web server – Other benefits: identified and corrected some single threaded web

functions

• Order XML Benchmark on Sun OEL – Goal: Process XML integration through order creation in 1 sec per – Result: 16 orders / second (24 orders / second scaled) – Other benefits: Provided new data queue feature in 6.2

Other Performance Activities (Continued)

• Client Integration prepare for Go-Live – Goal: Process 200,000+ i_transmissions per hour – Result: 230,000+ achieved (limited by sending system) – Other Benefits: Improved Object Locking, lessons learned re database

contention and ways to alleviate

Agenda

Performance Testing Advice

• Types of Performance Testing – Load Testing – 1x, 2x, – Stress Testing – Extreme Load, Testing for Breakage, not

Performance – Endurance Testing – Sustained Volume w/ Consistent

Performance – Spike Testing – Peak Volume (# users, transactions, etc.)

• Follow a Methodical Process – Ramp up Tests Starting from a Point of Success – Increase Volume to Failure Point – Tune to Address Failure Point – Repeat

• What to Monitor – CPU, Memory Utilization on all tiers – OTM Diagnostic Servlets

• Event Diag • Cache Diag

– Database AWR/ADDM Reports

• Note: What to Collect When Performance Problems Start to Exist in OTM? (Doc ID 887885.1)

• Tuning Options – Planning Parameters – Data Cache Sizes – OTM Event Threads – JVM Memory – Number of CPUs – Database: Queries, Indexes, Init.ora Params – Servers (Web, App, DB RAC)

• Common Mistakes – Failure to monitor all systems, including upstream and

downstream servers – Jumping to solutions, rather than analyzing data – Not leaving enough time to address issues – Using unrealistic test data – Throttling test w/ too much wait time

• Sustaining Performance – Create a Baseline for Comparison – Compute DB Statistics Regularly – Purge/Archive Unnecessary Data – Institute a Change Control Process

Real World Performance Benchmarking

Technology