Real Application Testing

Kevin Patterson, Senior Sales Consultant

Systems and Applications Management (SAM)

The following is intended to outline our general

product direction. It is intended for information

purposes only, and may not be incorporated into any

contract. It is not a commitment to deliver any

material, code, or functionality, and should not be

relied upon in making purchasing decisions.

The development, release, and timing of any

features or functionality described for Oracle’s

products remains at the sole discretion of Oracle.

Load Testing andLoad Testing andAccelerators forAccelerators forPackaged Apps & Packaged Apps & Web ServicesWeb Services

Functional Testing andFunctional Testing andAccelerators forAccelerators forPackaged Apps & Packaged Apps & Web ServicesWeb Services

Test ManagementTest Management

Real Application TestingReal Application Testing

Data MaskingData Masking

Real UserReal UserExperience MonitoringExperience Monitoring

Service Level Service Level ManagementManagement

Packaged Application Packaged Application ManagementManagement

SOA Application ManagementSOA Application Management

Diagnostic and TuningDiagnostic and Tuning

Provisioning, Configuration Provisioning, Configuration ManagementManagement

Change Tracking Change Tracking and Synchronizationand Synchronization

Oracle Enterprise ManagerOracle Enterprise Manager

ApplicationApplicationOperationsOperationsManagementManagement

TopTop--Down Application Management Down Application Management

ApplicationApplicationQuality Quality

ManagementManagement

Application Testing Suite

Real Application TestingLoad and SQL impact testing solution providing

highest quality testing for the database tier

RemediateRemediate

TestTestTest

DeployDeploy

• Value

• Rapid technology adoption

• Higher testing quality

• Business Benefit

• Lower cost

• Lower risk

• Features

• Database Replay

• SQL Performance Analyzer (SPA)

Business Agility through Superior Testing

Change

Testing TodayProduction – 1,000s of Real Online Users

PRODUCTION

Application

Servers

Database

Servers

Testing TodayTest – 1-2 testers trying to be 1,000s of users

TESTPRODUCTION

Application

Servers

Database

Servers

Database

Servers

Database ReplayReal workload for 1,000s of online users captured

Capture

Workload

PRODUCTION

Application

Servers

Database

Servers

Database Replay Real workload for 1,000s of online users replayed

Capture Replay

PRODUCTION

Application

Servers

Database

Servers

Workload

Database

Servers

TEST

Database ReplayTest your system changes at production levels

Capture9.2.0.8,

10.2.0.3+, 11g

PRODUCTION

Application

Servers

Database

Servers

Workload

11g ONLY

Database

Servers

TEST

Replay

Application

Servers

Real Application Testing with

Database Replay

• Replay production workload in test environment

• Identify, analyze and fix potential instabilities before making changes to production

• Capture Workload in Production

• Capture full production workload with real load, timing & concurrency characteristics

• Move the captured workload to test system

• Replay Workload in Test

• Make the desired changes in test system

• Replay workload with full production characteristics

• Honor commit ordering

• Analyze & Report

• Errors

• Data divergence

• Performance divergence

Analysis & Reporting

To:From:

Why DB Replay?

Complete workflowsPartial workflows

Low riskHigh risk

AutomatedManual intensive

Production workloadsArtificial workloads

Days of developmentMonths of development

150

Days

10

Days

Supported Changeswith Real Application Testing

Changes

Unsupported

Changes Supported

•Database Upgrades, Patches

•Schema, Parameters

•RAC nodes, Interconnect

•OS Platforms, OS Upgrades

•CPU, Memory

•Storage

•Etc.

Recording of External Client Requests

Application

Servers

Database

Servers

…

…

Database Replay WorkflowProduction

(9.2.0.8, 10gR2, 11g) Test (11g)

Capture ReplayAnalysis &

ReportingProcess

StorageStorage

Mid-Tier

Replay Driver

Clients

<Insert Picture Here>

Application Quality

Management

Oracle Real Application Testing

Database Replay

Demonstration

Database Replay Restrictions

• Database Replay does not support the following

features in the current release

• Direct Path Load (SQL Loader)

• Shared server (Oracle MTS)

• Oracle Streams & Advanced Replication Streams

• Non-PL/SQL based Advanced Queuing (AQ)

• Flashback queries

• OCI-based Object Navigation

• Non SQL-Based Object Access

• Distributed transactions, remote describe/commit

operations (will be replayed as local transactions)

……

ClientClient

…Client

Capture SQL

• Test impact of change on SQL query performance

• Capture SQL workload in production including statistics & bind variables

• Re-execute SQL queries in test environment

• Tune regressed SQL and seed SQL plans for production

Middle Tier

Storage

Oracle DB

Re-execute SQL Queries

Production Test

Use SQL Tuning Advisor to tune regression

Real Application Testing with

SQL Performance Analyzer (SPA)

9i, 10g, 11g 10g, 11g

Why SQL Performance Analyzer (SPA)?

• Businesses want systems that are performant and meet SLA’s

• SQL performance regressions are #1 cause of poor system performance

• Solution for proactively detecting all SQL regressions resulting from changes not available

• DBA’s use ineffective and time-consuming manual scripts to identify problems

SPA identifies all changes in SQL performance before impacting users

…

SQL Performance Analyzer Workflow

Storage

Production (9.2, 10.1, 10.2, 11g)

Test (10.2 or 11g)

Capture

SQL

Transport

SQL

Execute SQL

Pre-change

Execute SQL

Post-change

Compare

Perf

Storage

Mid-Tier

Clients

SQL Performance Analyzer

Restrictions

• SQL Performance Analyzer does not support the

following features in the current release

• Shared server (Oracle MTS)

• 9i Parallel Query (SQL Trace Capture)

<Insert Picture Here>

Application Quality

Management

Oracle Real Application Testing

SQL Performance Analyzer

Demonstration

Real Application Testing: Tools of the Trade

• Replays real database workload

on test system

• Predicts SQL performance deviations

before end-users can be impacted

What is it?

• Comprehensive testing of all

sub-systems of the database

server using real production

workload

• Unit testing of SQL with the goal to

identify the set of SQL statements

with improved/regressed performance

When to use?

• Captures workloads and replays it

with production characteristics

including concurrency,

synchronization &

dependencies

• Executes each SQL, stored in SQL

Tuning Set, in isolation using

production context and then

compares before and after execution

plans and run-time statistics

How it works?

• Assess impact of change on

workload throughput

• Assess impact of change on SQL

response time

What Purpose?

Database ReplaySQL Performance Analyzer

SQL Performance Analyzer and Database Replay are complementary

Why Consider Oracle 11g for Upgrades?

• Oracle Database 9i Release 2 Premium Support ended Jul 30, 2007

• 1 yr free Extended Support offer ends Jul 30, 2008

• Help enable upgrades from 9.2.0.8

t

2008

2003

2004

2005

2006

2007

2002

2009

2010

Today

2011

2012

2013

2014

Sustaining SupportPremier Support

R2 July 2007

Extended Support

July 2010

January 2009 January 2012

R2 July 2010 July 2013

http://www.oracle.com/support/library/data-sheet/oracle-lifetime-support-policy-datasheet.pdf

2015

2016

August 2012 August 2015

Jul 2008

“Oracle Real Application Testing reduces the time

required to test changes by as much as 80%, lower

testing costs by as much as 70%, mitigate risks by

reducing the number of unexpected outages, and

improve the quality of service for their IT operations.”

David Mitchell

Senior Vice President, OVUM

Source: Oracle Real ApplicationTesting – business agility through superior testing, Jan 2008

Further Information

• Oracle Real Application Testing Homepage on OTN

http://www.oracle.com/technology/products/manageability/database/index.html

• Data Sheets

• Technical and Customer Whitepapers

• Analyst Reports

• Tech Notes on support for Earlier Releases

• Examples and Scripts for Command Line Interface Usage

• Demo Viewlets

• Oracle Real Application Testing User Guide

http://download.oracle.com/docs/cd/B28359_01/server.111/e12253.pdf

Further Information (Continued)

• 11G Top Features for DBAs and Developers

• http://www.oracle.com/technology/pub/articles/oracle-database-11g-top-features/index.html

• 11G Oracle By Example Series:

• http://www.oracle.com/technology/obe/11gr1_db/index.htm

• Database Replay

• http://www.oracle.com/technology/obe/11gr1_db/manage/dbreplay/dbreplay.htm

• SQL Performance Analyzer

• http://www.oracle.com/technology/obe/11gr1_db/manage/spa/spa_guide.htm

• Metalink Notes for Patches and Steps to Implement:

• 560977.1: Real Application Testing Now Available for Earlier Releases

• 562899.1: Oracle Real Application Testing: Test the SQL Performance Impact of an Oracle 9i to Oracle Database 10g Release 2 upgrade with SQL Performance Analyzer

Database Replay for Earlier Releases

• Help customers smoothly transition to Oracle Database 11g

• “Workload Capture” portion of Database Replay is now available with Oracle 9i and Oracle Database 10g

• “Workload Replay” only possible on Oracle Database 11g and above

• For more details:

• Metalink Note: 560977.1 – “Real Application Testing for Earlier Releases”

Upgrade

From

Upgrade

To

Release / Patches

needed

Comments

10gR2 11g • 10.2.0.4 Patchset

• 10.2.0.x + one-off patch

• Need to apply one-off patch for 10.2.0.2 and

10.2.0.3

9iR2 11g • 9.2.0.8 + one-off patch • Limited performance data available for

captured workload – can use STATSPACK

SQL Performance Analyzer

for Earlier Releases• Help customers smoothly transition to Oracle Database 10g and 11g

• SQL Test Execution feature of SPA backported to 10gR2

• For more details:

• Metalink Note: 560977.1 – “Real Application Testing for Earlier Releases”

• Metalink Note: 562899.1 – “Testing Performance Impact of an Oracle 9i to

Oracle Database 10g Release 2 Upgrade with SQL Performance Analyzer”

Upgrade

From

Upgrade

To

Release / Patches

needed

Comments

10gR2 10gR2 or

11g

• 11.1.0.6 + one-off patch

or 11.1.0.7

• 10gR2 + one-off patch

• Uses STS to capture SQL stats

9i

10gR1

10gR2 or

11g

• 11.1.0.6 + one-off patch

or 11.1.0.7

• 10gR2 + one-off patch

• Uses SQL*Trace to capture SQL stats