Abstract
This validation guide describes the system architecture, design, testing methodology, and testing results of Dell EMC Ready Solutions for Oracle using Dell EMC XtremIO X2 all-flash and Data Domain protection storage.
Validation Guide
Dell EMC Solutions
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain
Enterprise-Class Storage Provisioning and Data Protection
November 2018
H17447.1
2
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The information in this publication is provided as is. Dell Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copying, and distribution of any software described in this publication requires an applicable software license.
Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, the Intel logo, the Intel Inside logo and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. Published in the USA 11/18 Validation Guide H17447.1.
Dell Inc. believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
Contents
3
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Contents
Chapter 1 Executive Summary 5
Business challenge ............................................................................................... 6
Benefits of Ready Solutions for Oracle .................................................................. 7
Key results ............................................................................................................ 8
Scope.................................................................................................................. 10
Audience ............................................................................................................. 10
We value your feedback ...................................................................................... 11
Chapter 2 Technology Overview 12
Solution overview ................................................................................................ 13
Solution architecture ........................................................................................... 14
Key components ................................................................................................. 15
Chapter 3 Architecture Overview 19
Logical architecture overview .............................................................................. 20
Physical architecture overview ............................................................................ 24
Data Domain DD6300 systems for database backup .......................................... 29
Chapter 4 Design Considerations 30
Compute design .................................................................................................. 31
Network design ................................................................................................... 37
Storage design .................................................................................................... 45
XtremIO Virtual Copy (XVC) database design ..................................................... 51
Data Domain backup system design ................................................................... 54
Chapter 5 Test Methodology and Results 60
Test objective ...................................................................................................... 61
Test tools and methods ....................................................................................... 62
Use case 1: Ease of storage provisioning with XtremIO ...................................... 62
Use case 2: Baseline – One production OLTP RAC database ............................ 66
Use case 3: Impact of one 11gR2 and one 12cR2 virtualized database on baseline ........................................................................................................ 78
Results Summary: Mixed physical and virtual environments running mixed OLTP and OLAP workloads on Ready Solution for Oracle with XtremIO X2 ........................................................................... 83
Contents
4
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Chapter 6 Test Methodology and Results: Data Protection 84
Test objective ...................................................................................................... 85
Use case 1: Full backup of one OLTP RAC database ......................................... 85
Use case 2: Restore and recovery of one OLTP RAC database from full backup ............................................................................. 88
Data protection testing summary ......................................................................... 88
Chapter 7 Conclusion 90
Conclusion .......................................................................................................... 91
Benefits ............................................................................................................... 91
Summary ............................................................................................................ 92
Chapter 8 References 93
Dell EMC documentation..................................................................................... 94
VMware documentation ...................................................................................... 94
Oracle documentation ......................................................................................... 94
HammerDB documentation ................................................................................. 94
Appendix A Configuration Details 95
Database performance data collection ................................................................ 96
Database parameters .......................................................................................... 98
HammerDB configuration parameters ................................................................. 98
OLAP query customization ................................................................................ 100
Chapter 1: Executive Summary
5
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Chapter 1 Executive Summary
This chapter presents the following topics:
Business challenge ............................................................................................. 6
Benefits of Ready Solutions for Oracle ............................................................. 7
Key results........................................................................................................... 8
Scope ................................................................................................................. 10
Audience ............................................................................................................ 10
We value your feedback ................................................................................... 11
Chapter 1: Executive Summary
6
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Business challenge
Oracle databases often support a company’s most complex and critical applications,
frequently enabling Enterprise Resource Planning (ERP) and Customer Relationship
Management (CRM) systems that are responsible for all back-office processes. The
pressure to modernize the database infrastructure means that businesses are looking for
solutions that offer greater agility, operational efficiencies, and resiliency in a single
solution.
Dell EMC Ready Solutions for Oracle are solutions that are designed to boost
performance and operational agility for your database ecosystem. Ready Solutions for
Oracle integrate Dell EMC PowerEdge servers, networking, and storage. They incorporate
proven design, testing, and release phases that might take months or weeks to complete,
to deliver superior performance, significant cost savings, and future-ready scalability. The
engineered Ready Solutions for Oracle provide your business with faster time-to-value to
reach operational readiness.
Dell EMC XtremIO X2 storage, a purpose-built all-flash array, has been added to the
Ready Solutions for Oracle offerings. XtremIO X2 storage offers consistently high
performance with low latency, unmatched storage efficiency with inline, all-the-time data
services, rich application, integrated copy services, and unprecedented management
simplicity.
XtremIO storage is designed and optimized for business applications, databases, and for
DBAs to:
Provide an extremely balanced system across all X-Bricks for compute power,
storage bandwidth, and capacity.
Provide consistent and predictable database performance by balancing data
distribution to eliminate hot spots.
Inline deduplication saves physical data capacity without impacting database
performance.
Only copy and store unique data that is added or changed from the point that the
copy is made due to inline deduplication. This feature helps with data reduction
for Oracle databases.
Provide a flash-based data protection algorithm (XDP) that offers better
performance than RAID-10 with RAID-5 capacity savings.
Chapter 1: Executive Summary
7
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Benefits of Ready Solutions for Oracle
Traditionally, application owners worked collaboratively with the IT organization to design
a new database platform. The multivendor process took months and required extensive
research and analysis to ensure that all the components worked together. Without any
assurance that the new system would perform as expected, the endeavor also entailed
significant risk.
Dell EMC Ready Solutions, such as Ready Solutions for Oracle, transform the design,
buy, and build process by providing a fully integrated and tested platform. In designing
Ready Solutions for Oracle, we focus on key priorities such as performance, resiliency,
and automation. Ready Solutions for Oracle with Dell EMC XtremIO X2 fully integrate
these components:
Dell EMC PowerEdge R740 and R940 servers
Red Hat Enterprise Linux 7.4
VMware vSphere 6.5
Dell EMC Networking
Dell EMC XtremIO X2 storage
Complete testing with customer-purchased Oracle 12c Release 2 and Oracle 11g
Release 2 databases
Dell EMC Data Domain 6300 storage
This paper focuses on the configuration that includes R740 and R940 servers, however,
the R840 server is also supported. Ready Solutions for Oracle can also be based on a
VMAX system configuration.
By eliminating the time-consuming and complex process of designing a system, this
pretested, prevalidated solution streamlines the purchase and update cycles for the IT
organization. It also accelerates delivery times of complex mission-critical databases and
applications. Features of this Ready Solution for Oracle include:
Sub-0.85 millisecond latencies for OLTP databases and applications such as ERP
and CRM systems with higher IOPS, network throughput, and transactions per
second
High density of IOPS for greater database consolidation
High throughput for OLTP and mixed workloads
Inline deduplication and compression to increase space savings without impacting
database performance
Chapter 1: Executive Summary
8
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Integration with VMware vSphere for centralized virtualization management
Automated database copies, repurposing, and protection with Dell EMC XtremIO
Virtual Copy (XVC)
Provisioning XtremIO LUN and presentation with a few clicks in XtremIO WebUI
Designed to modernize how databases are managed, Ready Solutions for Oracle are:
Engineered—Compute, networking, and storage integrated with prerequisites and
dependencies have been tested to deliver a seamless solution experience.
Agile—A modern Oracle management experience with simpler and intuitive
provisioning capabilities featuring 3-step database creation provides faster time to
value.
Optimized—Proven and documented consistent performance, simpler
management 3-step storage provisioning, higher data reduction (while creating
faster snapshots) and resiliency best practices ensure a highly effective Oracle
database environment.
Key results
Ready Solutions for Oracle accelerate adoption of a modern database platform. We tested
and validated every component, including servers, storage, and software, with Oracle
databases. We conducted extensive testing to ensure integration, performance, resiliency,
and the development of best practices. Sizing the solution for your Oracle ecosystem is a
streamlined process because extensive testing provides an accurate foundation for
meeting database requirements. As part of the validation process, this guide documents
the best practices that we used to configure and accelerate Oracle databases on an
Oracle Real Application Clusters (RAC) production OLTP, snapshot, and virtualized (both
11g and 12c) database configurations along with the backup and recovery of the RAC
production configurations.
We tested the physical environment or the physical production database setup by running
an Oracle RAC 12cR2 database across two physical servers running Red Hat Enterprise
Linux 7.4.
We created and validated the Oracle 12c production OLTP database on the XtremIO X2
storage with just a few clicks. The key results for this use case are as follows:
DBAs can plan and provision the XtremIO storage simply by using the XtremIO UI
with 3-step provisioning.
The Ready Solution for Oracle sustained 383,000 Transactions per Minute (TPM)
using PowerEdge R940 servers and an XtremIO X2 all-flash array.
Two-node Oracle 12c RAC generated over 120,000 IOPS at sub 0.85 millisecond
storage latencies for the production OLTP database.
Dell EMC achieved a 54% space savings with an Oracle 12c RAC database on the
new Ready Solution with XtremIO. Inline deduplication and compression reduced a
1.16 TB database to 629 GB on XtremIO X2.
Physical
production
database setup
results (use case
2: test 1)
Chapter 1: Executive Summary
9
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
We tested the physical XtremIO Virtual Copy (XVC) with Oracle databases that were
repurposed from the production database by running the following Oracle databases in
parallel on a single dedicated server:
One production RAC DB (two nodes)
One Online Analytical Processing (OLAP)-like XVC database repurposed from the
OLTP production database
One OLTP XVC developmental database repurposed from the OLTP production
database
The key results for this use case are as follows:
Over 1.4 GB/s was generated by the OLAP workload running in parallel with the
Oracle RAC OLTP database.
The OLAP workload did not significantly impact average storage latencies of the
Oracle RAC OLTP database.
Over 389,000 TPM were achieved by the Oracle RAC database running in parallel
with the OLAP database.
XtremIO overall copy efficiency of 2.9 to 1 shows that two copies of an Oracle
database used a small amount of space. For example, 3.5 TB of Oracle databases
occupied just 1.2 TB on XtremIO X2 due to advanced data reduction features.
We tested the virtualized environment by running the following Oracle databases in
parallel on a single dedicated VMware ESXi host:
One production RAC DB (two nodes)
One virtual machine (VM) running Oracle Database 11g Release 2
A second VM running Oracle Database 12c Release 2
The key results for this use case are as follows:
We generated over 451,000 TPM across five OLTP databases on XtremIO X2
XtremIO scaled from 383,500 to 451,000 TPM, demonstrating the capability to
address rapid application growth
XtremIO X2 sustained over 131,000 IOPS across five Oracle database
Average storage latencies remained under 1 millisecond with five Oracle database
workloads running in parallel.
XtremIO scaled from 120,000 to over 131,000 IOPS using 36 SSD drives in
XtremIO X2 with with sub-millisecond latencies
In this use case, we observed a complete database consolidation with sub 0.85
millisecond latencies, and with increased IOPS, network throughput, and transactions per
minute.
Physical XVC
databases setup
results (use case
2: tests 2 and 3)
Virtualized
databases setup
results (use case
3)
Chapter 1: Executive Summary
10
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
We validated and tested the backup and recovery for this configuration of the Ready
Solutions for Oracle by using a Dell EMC Data Domain DD6300 system with Data Domain
Boost (DD Boost) software. We used the following test cases:
Full backup of a 1 TB production OLTP database
Full recovery of a 1 TB production OLTP database
Dell EMC engineering test results show the following performance outcome by using a
Data Domain DD6300 system:
A full backup of a 1 TB Oracle database completed in 36 minutes. The database
size was reduced to 756.7 GB (total compression size) in the Data Domain system
with 28 percent compression (a compression factor of 1.4X) and storage throughput
of 482 MB/sec.
The full recovery of a 1 TB Oracle database completed in 32 minutes with storage
throughput of 571 MB/sec.
Scope
This validation guide describes the Ready Solutions for Oracle with Data Protection on
XtremIO X2 storage, including Data Domain storage. We tested and validated Ready
Solutions for Oracle with various types and sizes of database workloads to ensure
maximum flexibility. This guide discusses the methodology of the testing that we
conducted on the solution and the results of the testing.
Audience
This guide is intended for IT administrators, storage administrators, virtualization
administrators, system administrators, IT managers, and personnel who evaluate, acquire,
manage, maintain, or operate Oracle database environments.
Data protection
test results
(Chapter 6)
Chapter 1: Executive Summary
11
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
We value your feedback
Dell EMC and the authors of this document welcome your feedback on the solution and
the solution documentation. Contact The Dell EMC Solutions team with your comments.
Authors: Oracle Ready Solutions Engineering team, Indranil Chakrabarti, Sam Lucido,
Reed Tucker
Chapter 2: Technology Overview
12
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Chapter 2 Technology Overview
This chapter presents the following topics:
Solution overview ............................................................................................. 13
Solution architecture ........................................................................................ 14
Key components ............................................................................................... 15
Chapter 2: Technology Overview
13
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Solution overview
The ability to run OLAP, OLTP, and test/dev in a single virtualized environment while
safeguarding availability, lowering costs, and increasing productivity yields significant
advantages. This Ready Solution for Oracle was designed to support production OLTP
and OLAP applications and test/dev environments simultaneously without sacrificing
performance or storage space. With these Oracle databases and other applications co-
existing and functioning optimally, you gain a host of benefits, including the following:
Enhanced IOPS with sub-0.85 millisecond response times
Efficient operation of multiple applications in the same rack while saving on
licensing costs
Breakthrough simplicity for deployment, management, and support
Management by DBAs of their own backup and recovery processes without the
assistance of backup administrators
Exceptional performance, reducing backup times maintenance windows
Minimal impact on production workloads while performing backups or recovery,
enabling DBAs to run production simultaneously with backup windows
Integration of the hardware stack, drivers, and other components by the same
common vendor
Reduction of the customer’s TCO due to purchase of an optimal hardware stack
with this solution
Chapter 2: Technology Overview
14
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Solution architecture
The following figure shows the solution architecture.
Figure 1. Ready Solutions for Oracle with XtremIO X2 storage and data protection: Architecture overview
Chapter 2: Technology Overview
15
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Key components
Dell EMC PowerEdge 14G servers are built to accommodate databases, storage arrays,
data protection, hyper-converged appliances and racks. They are frequently used in
Ready Solutions and other Dell EMC solutions. These servers are part of a secure,
scalable compute platform that is the ideal foundation for cloud, analytics, and software-
defined data center initiatives.
The PowerEdge R740 server was designed to accelerate application performance by
using accelerator cards and storage scalability. The 2-socket, 2U platform has the
optimum balance of resources to power the most demanding environments.
The PowerEdge R940 server delivers consistent, high performance results for data-
intensive applications and data analytic workloads. With powerful Intel Xeon Scalable
processors and up 112 cores, the PowerEdge R940 server can quickly turn analytics into
insights to drive your business forward faster. Create an optimal configuration of NVMe,
SSD, HDD, and GPU resources to address your most demanding workloads–all in a 2U
chassis.
The Dell EMC XtremIO array is a purpose-built, all-flash array that delivers an innovative
metadata-centric architecture and unprecedented management simplicity. It is designed
with an elegant software-driven architecture that not only provides high performance with
consistently low latency for production volumes, but also creates thousands of copies on
which it runs workloads without impacting performance. The XtremIO X2 array provides
all-the-time, inline data services: data reduction, thin provisioning, flash-optimized data
protection, encryption, metadata-aware replication, and in-memory virtual data copies.
XtremIO Virtual Copy (XVC) helps provision and deploy space-efficient, instant virtual
data copies without impacting system performance. XVCs are created by capturing the
state of data in volumes at a particular point in time and allowing users to access that data
when needed, even when the source volume has been changed or deleted. XVCs are
inherently writeable, but can be created or changed to be read-only to maintain
immutability. Virtual copies can be taken from either the source or any virtual copy of the
source volume, which makes it easy to protect and recover from any operational and
logical corruption. XVCs enable the creation of frequent point-in-time copies (according to
RPO intervals – seconds, minutes, hours) and use them to recover from any data
corruption. An XVC can be kept in the system for as long as needed. Recovery by using
XVC is instantaneous and does not impact system performance.
XVC is an integral part of the XtremIO array’s integrated Copy Data Management (iCDM)
capabilities. It enables administrators to create, refresh, and restore thousands of
production copies, and run workloads on them without any storage overhead, accelerating
business agility. iCDM enables instant XVC creation from the production system or from a
gold master copy with no performance impact. Administrators can provision copies that
are immediately usable by the application, enabling quick deployments and ensuring that
those deployed copies are fully functional. These copies can be repurposed for near real-
time analytics, test/dev, patching, sandbox testing, and the ability to refresh and restore in
all directions, resulting in complete space efficiency. XVCs instantly refresh virtual copies
such as production to any copy, any copy to any copy, and any copy to production.
Dell EMC
PowerEdge 14G
servers
Dell EMC
XtremIO X2
storage
XtremIO Virtual
Copy (XVC)
Chapter 2: Technology Overview
16
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Data Domain systems are disk-based inline deduplication appliances and gateways that
provide data protection and disaster recovery (DR) in the enterprise environment. All
systems run the Dell EMC Data Domain Operating System (DD OS), which provides a
command-line interface (CLI) to perform all system operations. They also run the Dell
EMC Data Domain System Manager (DD System Manager UI) to configure, manage, and
monitor. The Data Domain storage system offers a cost-effective alternative to tape. Data
Domain systems reduce the amount of disk storage to retain and protect data by 10 to 30
times. Because data on disk is available online and onsite for longer retention periods,
restoration is fast and reliable.
DD Boost technology for Recovery Manager (RMAN) optimizes communication between
the database servers and the Data Domain system. It improves backup performance by
reducing the amount of data that is transferred over the network between database
servers and the Data Domain system, as well as the amount of data stored by Data
Domain. By working with RMAN, the DD Boost components consists of:
DD Boost server software that runs on the Data Domain system.
DD Boost database application agents, which are installed on database servers.
The agent works as a plug-in for Oracle RMAN to provide database backup. It has
a DD Boost library for communicating with the DD Boost server running on a Data
Domain system.
DD Boost software extends the Data Domain Data Invulnerability Architecture by
generating checksums on the Oracle database server before RMAN sends the data to the
Data Domain system. The Data Domain system receives the data, computes the new
checksum that is based on the incoming data, and compares the new checksum with the
old checksum sent from the Oracle database server. This process ensures inline
verification of data.
DD Boost distributed segment processing
DD Boost technology includes distributed segment processing (DSP). When this feature is
enabled, the deduplication process is distributed between the DD Boost database
application agent on the database server and the DD Boost server on the Data Domain
system. Because parts of the deduplication process run on the database servers, the DD
Boost library sends only the unique data to the Data Domain system over the network.
With this DD Boost feature enabled, the backup with deduplication process follows these
steps:
1. The backup data stream is broken into variable-length segments and each
segment is identified.
2. The system determines if each segment is unique or if it is already stored in the
Data Domain system.
3. If the segment is unique (not stored in the Data Domain system), it is compressed
and sent over the network to the Data Domain system and written to the disks.
Distributed segment processing provides significant benefits for the Oracle database
backup:
Improves backup throughput because the DD Boost library sends only the unique
data to the Data Domain system. The more deduplicated data that there is in the
Dell EMC Data
Domain systems
Dell EMC DD
Boost technology
for Recovery
Manager
Chapter 2: Technology Overview
17
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
backup dataset, the higher the backup throughput, which in turn reduces the
backup time.
Reduces the network bandwidth requirement. Because only unique data is sent to
the Data Domain system through the network, less network bandwidth is used.
Reduces the storage capacity required to store database backup images, and
increases the retention period for the database backups.
VMware vSphere is a complete and robust virtualization platform that uses dynamic
resource pools to virtualize business-critical applications with flexibility and reliability. It
transforms a computer's physical resources by virtualizing the CPU, RAM, hard disk, and
network controller. This transformation creates fully functional VMs that run isolated and
encapsulated operating systems and applications.
The vSphere virtualization layer decouples the application from the underlying physical
resources. This decoupling enables greater flexibility in the application layer by eliminating
hardware downtime for maintenance and changes to the physical system without affecting
the hosted applications. In a server-virtualization use case, this layer enables multiple
independent VMs to share the same physical hardware.
Red Hat Enterprise Linux (RHEL) 7.4 offers automation capabilities designed to limit IT
complexity while enhancing workload security and performance for traditional and cloud-
native applications. For more information about Red Hat Enterprise Linux 7.4, go to the
Red Hat website.
Oracle Database 12c delivers industry-leading performance, scalability, security, and
reliability on a choice of clustered or single servers running Microsoft Windows, Linux, or
UNIX. It introduces a new architecture, Oracle Multitenant, where one or more pluggable
databases (PDBs) are created inside a container database (CDB).
The multitenant architecture supports the following configurations:
A single-tenant configuration, with one PDB plugged into a CDB, which is available
for no extra cost in all editions
A multitenant option for up to 252 PDBs per CDB, which is an extra-cost option of
Oracle 12c Enterprise Edition
Oracle Database 11g Release 2 provides a foundation from which IT can successfully
deliver more information with a higher quality of service, making more efficient use of your
IT budget.
Oracle RMAN is a database backup and recovery tool that is built into the Oracle
database server. With Oracle RMAN, the Oracle Database Administrator (DBA) schedules
database backup jobs to back up database files and archive logs to a backup system
routinely. DBAs also use RMAN to restore and recover database files and archive logs
from the backup system.The DD Boost database agent works with RMAN to send the
database backup images to the Data Domain backup system.
VMware
vSphere 6.5
Red Hat
Enterprise
Linux 7.4
Oracle Database
12c
Oracle Database
11g
Oracle RMAN for
database backup
and recovery
Chapter 2: Technology Overview
18
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Dell EMC Connectrix switches and directors bring high bandwidth and zero downtime to
your storage network. Connectrix offers a range of enterprise-class directors, medium
density departmental switches, and edge switches for small to large enterprise
environments or applications. Key features include:
Fibre Channel (FC) connectivity of up to 32 Gb/s and speeds of up to 40 GbE
NVMe-ready
Departmental switch scaling from 8 to 96 ports per switch
Redundant components and multipath deployments to ensure high availability and
automated failover
Advanced management tools to simplify the deployment and management of your
storage networking environment automatically
This solution was built with the cost-effective Connectrix B-Series DS-6510B and DS-
6610B switches. Most of the switches in the Connectrix portfolio can be used to build a
storage network for Oracle solutions, as long as the SAN speeds match or exceed the
speed of the devices in the SAN.
The DS-6510B is a 16 Gb switch that scales from 24 ports to 48 ports. The DS-6610B is a
32 Gb switch that scales from 8 ports to 24 ports. By default, the DS-6610B has 16 Gb
SFPs but can be upgraded with 32 Gb SFPs.
S4048-ON ToR switch
The Networking S4048-ON switch is an ultra-low-latency 10/40 GbE top of the rack (ToR)
switch that is built for applications in high-performance data center and computing
environments. By using a non-blocking switching architecture, the S4048-ON switch
delivers line-rate L2 and L3 forwarding capacity with ultra-low-latency to maximize
network performance.
The compact design provides a density of 48 dual-speed 1/10 GbE (SFP+) ports and six
40 GbE QSFP+ uplinks to conserve valuable rack space and simplify migration to 40 Gb/s
in the data center core. Each 40 GbE QSFP+ uplink can also support four 10 GbE ports
with a breakout cable.
In addition, the S4048-ON switch incorporates multiple architectural features that
optimize:
Data center network flexibility
Efficiency and availability
PSU to I/O panel airflow for hot/cold aisle environments
Redundant, hot-swappable power supplies and fans
Networking S3048-ON management switch
The Networking S3048-ON management switch is a low-latency switch that features forty-
eight 1 GbE and four 10 GbE ports, a dense 1U design, and up to 260 Gb/s performance.
Dell EMC
Connectrix
switches
Dell EMC
Networking
switches
Chapter 3: Architecture Overview
19
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Chapter 3 Architecture Overview
This chapter presents the following topics:
Logical architecture overview .......................................................................... 20
Physical architecture overview ........................................................................ 24
Data Domain DD6300 systems for database backup ...................................... 29
Chapter 3: Architecture Overview
20
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Logical architecture overview
The XtremIO X2-based Ready Solution for Oracle is designed to consolidate multiple
types of mixed-workload Oracle databases in a single system. The following types of
Oracle databases have been tested and validated for Ready Solutions for Oracle:
A production database in a physical environment (two nodes)
Two-nodes running Oracle RAC OLTP production database (also referred to as
PROD, OLTP Prod, or Prod database)
XtremIO XVC databases in a physical environment (single node)
One XtremIO XVC database repurposed from the production database for a
OLAP workload (also referred to as OLAP XVC, snpolap, or OLAP Snap)
One OLTP XVC database repurposed from the production database for
development (also referred to as DEV XVC or OLTP Snap)
Oracle databases in a virtual environment (single node)
One VM running Oracle Database 11g Release 2 (also referred to as v11gDB)
A second VM running Oracle Database 12c Release 2 (also referred to as
v12cDB)
The following figure shows the logical architecture of consolidated mixed-workload
databases. It includes the multiple layers of infrastructure components of the Ready
Solutions for Oracle along with Data Protection by using a Data Domain system as the
backup appliance.
Chapter 3: Architecture Overview
21
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 2. Ready Solutions for Oracle with XtremIO X2 Storage and data protection: Architecture overview
Chapter 3: Architecture Overview
22
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The following figure shows the logical layout of the Ready Solution for Oracle with the
XtremIO X2 platform.
Figure 3. Ready Solution for Oracle with XtremIO X2 storage: Logical layout
The server layer consists of:
Two PowerEdge R940 servers host a two-node Oracle 12cR2 RAC database
(PROD ) with one Oracle RAC database instance on each server. Both of these
PowerEdge R940 servers have four 18-core CPUs and 1.5 TB RAM. The software
stack includes Red Hat Linux (RHEL) 7.4 and the Oracle 12cR2 RAC database
environment, which includes Oracle 12cR2 Grid Infrastructure and Oracle 12cR2
RAC.
One PowerEdge R940 server is the ESXi host with ESXI 6.5 U2 hosting two virtual
machines, VM1 and VM2. These machines host one single-node 12c OLTP
database (v12cDB) and one single-node 11gR2 OLTP database (v11gDB). The
PowerEdge R940 server has four 18-core CPUs and 1.5 TB RAM. The virtual
machine VM1 uses RHEL 7.4 as the guest operating system that runs Oracle
12cR2 Automatic Storage Management (ASM) and an Oracle 12cR2 standalone
database. The virtual machine VM2 uses RHEL 6.9 as the guest operating system
that runs Oracle 11gR2 ASM and an Oracle 11gR2 standalone database.
One PowerEdge R740 server hosts the DEV XVC and OLAP XVC snapshot
databases. The PowerEdge R740 server has two 12-core CPUs and 768 GB RAM.
The PowerEdge R740 server stack consists of RHEL 7.4 and Oracle 12cR2 ASM
with Oracle 12cR2 standalone database software. These snapshot databases are
based on an XVC virtual copy of the database volumes of the Oracle 12cR2 OLTP
RAC database (PROD). The DEV XVC database is a snapshot copy of the PROD
database for development and test purposes, while OLAP XVC is for OLAP
reporting. These XVC copies are mounted to the ASM instance in this server to
form corresponding diskgroups where the two snapshot databases are stored. Both
XVC snapshot databases are writable snapshot copies of the PROD database.
Logical layout
Chapter 3: Architecture Overview
23
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The storage layer consists of an XtremIO X2 two-X-Brick cluster that hosts the storage
volumes’ XVC snapshots of all the databases and Oracle 12cR2 clusterware, as well as
the VM operating system disks. The following table describes the storage layout.
Table 1. Storage size for databases
Database Storage sizes (GB)
DATA REDO TEMP FRA Clusterware1 VM OS Total
OLTP 12cR2 PROD
4x300 2x100 500 200 3x50 N/A 2,250
v12cDB 4x600 2x100 500 2x100 3x100 600 4,200
v11gDB
DEV XVC2 4x300 2x100 100 200 3x50 N/A 250
OLAP XVC2 4x300
Total 3,600 400 1,100 600 600 600 6,700
Notes:
1 Clusterware storage size includes space for the Oracle Cluster Registry (OCR), voting disk, and
Grid Infrastructure Management Repository (GIMR). The VM operating system volume consists of
the virtual disks of the VMM operating system. 2 Snapshot volumes for both XVC databases - 4x300 GB for DATA, 2x100 GB for REDO, ASM and 1x200 GB for FRA - are repurposed copies of the corresponding volumes of OLTP 12cR2 PROD database. These volumes are not counted as extra space beyond the original storage volumes.
ASM provides storage management for all databases. For 12cR2 RAC PROD databases:
4 x 300 GB volumes are grouped to form a 1,200 GB DATA diskgroup for database
files
2 x 100 GB volumes form REDO1 and REDO2 diskgroups for REDO logs
1 x 200 GB volume forms a FRA diskgroup for FRA (Flash Recovery Area)
1 x 500 GB volume forms a TEMP diskgroup for TEMP files
For virtual databases, each volume is divided into two equal VMDKs, each of which is
presented to each of the two VMs to form the corresponding diskgroups. For the XVC
snapshot databases, the XVC snapshots are mounted into the ASM instance of the XVC
database server to form the corresponding diskgroups.
Chapter 3: Architecture Overview
24
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Physical architecture overview
The following figure shows the physical architectural layout of the Ready Solution for
Oracle with an XtremIO X2 storage array.
Figure 4. Physical architecture and connectivity overview
Physical layout
Chapter 3: Architecture Overview
25
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The figure also shows the high level connectivity between the various major components
in the Ready Solutions for Oracle. A single point of failure is eliminated for all the critical
hardware components, as follows:
Each of the database servers consists of highly available LAN and SAN
connectivity with multiple network interface cards (NICs) and multiple host bus
adapters (HBAs).
Two dual-port 10 GbE NICs and two dual-port 16 Gb/s HBAs in each database
server provide sufficient bandwidth and high availability for Oracle database
LAN and SAN traffic.
Redundant network ports are connected to separate 10 GbE switches for HA
and bandwidth in case of failure.
HBA ports on the same HBA adapters are connected to separate FC switches
for HA and bandwidth in case of failure.
Redundant ToR S4048-ON 10 Gb Ethernet switches provide sufficient 10 Gb ports
and bandwidth to share both Oracle database public and private interconnect traffic.
These switches also provide sufficient 40 GbE uplink ports to connect the Ready
Solution for Oracle to the datacenter core switches.
Redundant DS-6510B 16 Gb/s Fibre Channel (FC) switches provide sufficient ports
and bandwidth to share Oracle database SAN traffic between the database servers
and the XtremIO X2 storage controllers.
A dual X-Bricks XtremIO X2 cluster storage array with redundant and active-active
storage controllers provide sufficient capacity, bandwidth, and high availability for all
the Oracle databases tested in the Ready Solution for Oracle.
Each FC port in each XtremIO X2 storage controller is connected to separate
FC switches for high bandwidth and high availability (HA).
A management port from each major hardware component is connected to S3048-
ON 1 GbE management switch.
On each of the database servers:
A dedicated iDRAC network port is connected to the management switch
for out-of-band, bare-metal management of the server
The 1 GbE LAN on motherboard (LOM) network ports is connected to the
management switch for in-band management of the server from within the
operating system
Chapter 3: Architecture Overview
26
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The following table lists the major hardware components in the Ready Solution for Oracle
with XtremIO X2 storage.
Table 2. Ready Solution for Oracle with XtremIO X2 storage: Components overview
Component Description
Database servers 2 x Dell EMC 4S PowerEdge R940 servers for physical production databases
1 x Dell EMC 4S PowerEdge R940 server for virtual databases
1 x Dell EMC 2S PowerEdge R740 server for physical XVC databases
LAN switches 2 x Dell EMC Networking S4048-ON 10 Gb Ethernet switches
SAN switches 2 x Dell EMC Connectrix DS6510-B 16 Gb/s FC switches
Management switch 1 x S3048-ON 1 Gb Ethernet switch
Storage array Dual X-Bricks XtremIO X2 cluster
Compute
The following table lists the hardware details of the database servers used for physical
production databases in the Ready Solution for Oracle with XtremeIO X2 storage.
Table 3. Ready Solution for Oracle with XtremIO X2 storage: Physical production database server components
Component Description
Servers 2 x PowerEdge R940 servers
Chassis 2.5” chassis with up to 8 hard drives
Processor per server 4 x Intel Xeon Gold 6150 18c 2.7 GHz
Memory per server 1,536 GB (24 x 64 GB QR DDR4 2666MT/s LRDIMMs)
Local disks per server 3 x 1.2 TB 10 K SAS 12 Gb/s 2.5 in. HDDs (includes 1 hot spare)
RAID controller PERC H740P/H730P
iDRAC iDRAC9 Enterprise
rNDC Broadcom 5720 QP 1 Gb Base-T rNDC
Add-on NICs per server 2 x Broadcom 57412 DP 10 Gb SFP+ PCIe adapter
HBAs per server 2 x Emulex LPe31002-M6-D DP 16 Gb/s FC HBAs
Power supplies per server 2 x 1,600 W
Hardware
components and
sizing
Chapter 3: Architecture Overview
27
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Table 4. Ready Solution for Oracle with XtremIO X2 storage: Physical XVC databases server components
Component Description
Server 1 x PowerEdge R740 server
Chassis 8 x 2.5 in. SAS/SATA hard disk drives (HDDs) for 2 CPU configuration
Processor 2 x Intel Xeon Gold 6136 12c 3.0 GHz
Memory 768 GB (24 x 32 GB DR DDR4 2667 MT/s RDIMMs)
Local disks 3 x 1.2 TB 10 K SAS 12 Gb/s 2.5 in. HDDs (includes 1 hot spare)
RAID controller PERC H740P Adapter
iDRAC iDRAC9 Enterprise
rNDC Broadcom 5720 DP 1Gb + 57412 DP 10Gb NetXtreme- E rNDC
Add-on NICs None
HBAs 2 x Emulex LPe16002B-M6-D DP 16 Gb/s FC HBAs
Power supply 2 x 1,100 W
Table 5. Ready Solution for Oracle with XtremIO X2 storage: Virtualized databases server components
Component Description
Servers 1 x PowerEdge R940 server
Chassis 2.5” chassis with up to 8 hard drives
Processor 4 x Intel Xeon Gold 6150 18c 2.7 GHz
Memory 1,536 GB (48 x 32 GB DR DDR4 2667 MT/s RDIMMs)
Local disks 8 x 1.6 TB SAS 12 Gb/s 2.5 in. SSDs
RAID controller PERC H740P
iDRAC iDRAC9 Enterprise
rNDC Broadcom 5720 QP 1 Gb Base-T rNDC
Add-on NICs 2 x Intel X710 DP 10 Gb SFP+ NICs
HBAs 2 x QLogic QLE2692 DP 16 Gb/s FC HBAs
Power supply 2 x 2,000 W
Chapter 3: Architecture Overview
28
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Storage
The following table lists the hardware details of the storage array used in this Ready
Solution for Oracle with XtremIO X2 storage.
Table 6. Ready Solution for Oracle with XtremIO X2 storage: Storage components
Storage array Dell EMC XtremIO X2 storage
System specification Two X2-S X-Bricks Cluster
Operating system version 6.1.0-99_X2
Active-Active Controllers 4
Front-end FC ports 8
SSD enclosures 2
Number of SSDs 36
Raw/Usable capacity 13.1 TiB / 10 TiB
Infiniband switches 2
Network
The following table lists the network switches used in this Ready Solution for Oracle with
XtremIO X2 storage.
Table 7. Ready Solution for Oracle with XtremIO X2 storage: Network switches
Switch function Switch type
LAN 2 x Dell EMC Networking S4048-ON 10 GbE switches
SAN 2 x Dell EMC Connectrix DS-6510B 48-ports 16 Gb/s switches
Management 1 x Dell EMC Networking S3048-ON 1 GbE switch
Chapter 3: Architecture Overview
29
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Data Domain DD6300 systems for database backup
We tested the Data Domain DD6300 system as a backup configuration to back up and
restore the databases in this Ready Solution for Oracle. The following table lists the
hardware and software stack of the Data Domain DD6300 system. Chapter 6 provides the
details of our test methodology, tools, and results.
Table 8. Data Domain DD6300 system for database backup
Category Components
Processor 2 x Intel Xeon CPU E5-2680 v3, 2,501 MHz
Memory 96 GB (12 x 8 GB 1,866 MHz)
Number of network ports (in use) 2 x 10 GbE
Number of enclosures (DS60) 1
Active tier disks (in use) 11 x 3.64 TiB (40.03 TiB) SAS HDDs
Active tier disks (spare) 1 x 3.64 TiB SAS HDD
Cache tier disks 2 x 0.728 TiB (1.45 TiB) SAS SSDs
Expandable storage disks1 60 x 2.73 TiB (163.8 TiB) SAS HDDs—56 active (152.9 TiB) + 4 spare (10.9 TiB)
1 The expandable storage disks in DS60 were available but were not used in the testing.
This DD6300-based backup system was configured and tested as the database backup
and recovery solution for this Ready Solution for Oracle.
The following figure shows the high-level backup solution architecture. The Data Domain
DD6300 backup appliance is connected to the public network of this Ready Solution for
Oracle to back up the databases.
Figure 5. Backup solution high-level architecture
Chapter 4: Design Considerations
30
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Chapter 4 Design Considerations
This chapter presents the following topics:
Compute design ................................................................................................ 31
Network design ................................................................................................. 37
Storage design .................................................................................................. 45
XtremIO Virtual Copy (XVC) database design ................................................. 51
Data Domain backup system design ............................................................... 54
Chapter 4: Design Considerations
31
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Compute design
The database servers tested in this Ready Solution for Oracle—PowerEdge R940-based
physical production database servers, a PowerEdge R740-based physical XVC
databases server, and a PowerEdge R940-based virtual databases server—were
designed and configured with the following best practices:
The PCIe network adapters and HBAs that are used for Oracle database public,
private interconnect, and SAN traffic were populated based on the recommended
PCIe slot priority for optimal power, bandwidth, and thermal performance of the
adapters and the system. The populated and the recommended PCIe slots are
wired to two separate CPUs in their respective servers, allowing for load balancing
of I/Os across the two CPUs.
For optimal performance, in the PowerEdge R940-based databases servers,
the HBAs were populated in PCIe slots 2 and 5 and the NICs were populated in
PCIe slots 3 and 6.
For optimal performance, in the PowerEdge R740-based database server, the
HBAs were populated in PCIe slots 1 and 7 and the NICs were populated in
PCIe slot 3 and the rack Network Daughter Card (rNDC) slot.
The BIOS System Profile was set to Performance.
Memory DIMMs were populated with at least one DIMM per memory channel
across all CPU sockets to maximize the memory throughput with the CPU sockets.
The following table shows the capacity and quantity of memory DIMMs populated in
each of the database servers that are based on the recommended best practices.
Table 9. Database servers: Memory DIMMs capacity and quantities
Use case Server type
Number of CPU sockets
DIMMs per channel populated
DIMMs per socket populated
Per DIMM capacity
Total physical DRAM
OLTP PROD Database
R940 4 1 6 64 GB 4 x 1 x 6 x 64 = 1,536 GB
XVC Databases
R740 2 1 6 32 GB 2 x 1 x 6 x 32 = 768 GB
Virtual Databases
R940 4 2 12 32 GB 4 x 2 x 12 x 32 = 1,536 GB
For additional recommended best practices that were implemented on the physical
database servers see RHEL 7.4 as bare-metal operating system.
Physical servers
Chapter 4: Design Considerations
32
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The virtualized environment in this Ready Solution for Oracle was tested with two virtual
databases deployed on a single R940-based VMware ESXi host.
Figure 6. Single ESXi host with two VMs running two virtual databases
As shown in the figure above, the ESXi host contains:
One virtual machine (VM1) running:
RHEL 6.9 as the guest operating system
Oracle 11gR2 grid infrastructure software
Standalone Oracle Database 11gR2
A second virtual machine (VM2) running:
RHEL 7.4 as the guest operating system
Oracle 12cR2 grid infrastructure software
Standalone Oracle Database 12cR2
The ESXi host and the VMs were configured, monitored, and maintained using VMware
vSphere Web Client and VMware vCenter Server Appliance (VCSA), which was deployed
as a VM on the management server.
ESXi host
Chapter 4: Design Considerations
33
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
In the PowerEdge R940-based virtual databases server, we deployed ESXi 6.4 U2 as the
hypervisor. We applied best practices in our test environment, as described in the
following sections.
Note: We used the XtremIO Host Configuration Guide to apply best practices. See this guide for
the complete list of best practices for XtremIO storage in a VMware ESXi host environment.
HBA queue depth settings
The following table lists the default and the recommended HBA queue depth settings in
ESXi 6.5 hosts connecting to XtremIO X2 storage arrays:
Table 10. HBA queue depth settings in ESXi based hosts
Parameter Default value Recommended value
LUN Queue Depth QLogic: 64
Emulex: 30
QLogic: 256
Emulex: 128
HBA Queue Depth QLogic: N/A
Emulex: 8192
QLogic: N/A
Emulex: 8192 (maximum)
We set the LUN queue depth to the recommended value of 256 for the QLogic HBAs used
in our virtualized databases server. This setting ensures that the XtremIO X2 storage
arrays handle an optimal number of SCSI commands (including I/O requests).
Note: The QLogic HBA Queue Depth setting is no longer read by vSphere, therefore, it is not
relevant when configuring a vSphere host with QLogic HBAs.
Multipath configuration
The virtual databases ESXi host was configured by using vSphere Native Multipathing
(NMP). The following parameter values are recommended on the host for optimal
performance with XtremIO X2 storage:
Set the native path selection policy on the XtremIO volumes presented to the ESXi
hosts to round-robin.
Set the vSphere NMP round-robin path switching frequency to XtremIO volumes
from the default value (1000 I/O packets) to 1.
Note: In ESXi 6.5, the default path selection policy is round-robin and the default path switching
frequency is 1. Therefore, no change was needed in our virtualized databases server.
Host parameter settings
The following ESXi host parameters were configured:
Disk.SchedNumReqOutstanding—Determines the maximum number of active
storage commands (I/O) allowed at any time at the VMkernel. This parameter value
for each XtremIO volume presented to the ESXi host was set to the recommended
value of 256 by running the following CLI commands on the ESXi 6.5 host:
Chapter 4: Design Considerations
34
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
To get the list of all XtremIO volumes, type:
$> esxcli storage nmp path list | grep XtremIO -B1 | grep "\
naa" | sort | uniq
To set the value for each volume, type:
$> esxcli storage core device set -d <naa.xxx> -O 256
where <naa.xxx> is the XtremIO volume obtained from the previous command.
Disk.SchedQuantum—Determines the maximum number of consecutive
“sequential” I/Os allowed from one VM before switching to another VM. This value
was set from its default value of 8 to the recommended value of 64.
We used the following design principles and best practices to create the VMs in this
Ready Solution for Oracle:
SCSI controllers—We created multiple SCSI controllers to optimize and balance
the I/O for the different database disks, as shown in the following table.
Table 11. SCSI controller properties set in VMs
Controller Purpose SCSI bus sharing
Change type
SCSI 0 Guest OS disk None VMware Paravirtual
SCSI 1 Oracle DATA disks Physical VMware Paravirtual
SCSI 2 Oracle REDO disks Physical VMware Paravirtual
SCSI 3 Oracle OCR, GIMR, FRA, TEMP Physical VMware Paravirtual
Hard disk drives—All database-related virtual disks—for example, DATA, REDO,
FRA, OCR/VD, and TEMP—were assigned the following properties:
Type: ‘Thick provisioned eager zeroed’—Which ensures that the space required
for the virtual disks are allocated at creation time and the data on the physical
device on the storage is zeroed out
Sharing: ‘No sharing’—Selected because the deployed databases are
standalone databases that do not require sharing database virtual disks with
another VM, unlike RAC VMs that share virtual disks between two or more VMs
in the database cluster.
VM vCPU and vMem—The following table lists the distribution of virtual CPU
(vCPU) and virtual memory (vMem) to the two database VMs.
Table 12. VM configuration: vCPU and vMem details
VM Number of vCPUs
vMem
Reservation (GB) Total (GB)
VM1 18 120 140
VM2 18 120 140
Virtual machines
Chapter 4: Design Considerations
35
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Enable disk UUID—In each of the VM options, we added and set the configuration
disk.enableUUID parameter to TRUE. This setting ensures that the VMDK
always presents a consistent UUID to the VM.
RHEL 7.4 as bare-metal operating system
In the PowerEdge R940-based production database servers and the PowerEdge R740-
based XVC databases server, we deployed RHEL 7.4 as the bare-metal operating
system.
Note: We used the XtremIO Host Configuration Guide to apply best practices. See this guide for
the complete list of best practices for XtremIO storage in a Linux environment.
HBA queue depth settings
The following table lists the default and recommended HBA queue depth settings for a
Linux environment.
Table 13. HBA queue depth settings in Linux-based servers
Parameter Default value Recommended value
LUN Queue Depth QLogic: 32
Emulex: 30
QLogic: Keep default value
Emulex: Keep default value
HBA Queue Depth QLogic: 32
Emulex: 8192
QLogic: 65535 (maximum)
Emulex: 8192 (maximum)
Note: We kept the default queue depth values in our physical database servers because we used
Emulex HBAs.
I/O elevator settings
The recommended I/O elevator setting in the RHEL operating system running in the
database servers connecting to XtremIO X2 storage arrays is either deadline or noop.
The cfg I/O elevator setting is not recommended. In our physical database servers, we
used deadline, which is the default I/O elevator setting in RHEL 7.4.
Multipath configuration
We configured the physical database servers using Linux Native Multipathing available in
the RHEL 7.4 operating system. We created the configuration file for the
/etc/multipath.conf multipath daemon with the following recommended settings:
devices {
device {
vendor XtremIO
product XtremApp
path_grouping_policy multibus
path_checker tur
path_selector "queue-length 0"
rr_min_io_rq 1
user_friendly_names yes
fast_io_fail_tmo 15
failback immediate
}
RHEL operating
system for
Oracle
databases
Chapter 4: Design Considerations
36
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Partition alignment in Linux
We partitioned the database disks or XtremIO volumes presented to the Linux-based
physical database servers by using fdisk with the default starting sector value of 2,048.
This setting ensures that the starting sector number is a multiple of 16 (16 sectors, at 512
bytes each, is 8 KB). Therefore, each database disk is correctly aligned with the XtremIO
storage LUN striping.
RHEL as VM guest operating system
In this Ready Solution for Oracle, we deployed RHEL 6.9 as the guest operating system in
VM1 running Oracle 11g R2 database and RHEL 7.4 as the guest operating system in
VM2 running Oracle 12c R2 database.
In each guest operating system , we adhered to the following recommended best
practices.
PVSCSI driver
For optimal XtremIO X2 storage performance in a VMware environment, we recommend
PVSCSI controllers and driver in the guest VMs. In this Ready Solution for Oracle, we
ensured that the inbox RHEL vmw_pvscsi driver module was loaded and used in the
guest operating systems.
Note: The PVSCSI driver is used only when the SCSI controller type is set to VMware
Paravirtual in the VM settings.
PVSCSI LUN Queue Depth and ring-pages settings
The following table shows the default and the recommended vmw_pvscsi parameter
settings.
Table 14. PVSCSI parameter settings in guest operating systems
Parameter Default value Recommended value
vmw_pvscsi.cmd_per_lun RHEL 6: 64
RHEL 7: 254
RHEL 6: 254
RHEL 7: 254
vmw_pvscsi.ring_pages RHEL 6: 8
RHEL 7: 8
RHEL 6: 32
RHEL 7: 32
The parameters and their respective recommended values in this table were appended to
the kernel boot arguments:
In the /boot/grub.conf file for the RHEL 6-based guest operating system
In the /etc/default/grub file for the RHEL 7-based guest operating system
Other guest operating system configurations
Other guest operating systems were configured as follows:
Multipath was not configured because it is handled at the ESXi host level.
In the 12c database VM, all database storage disks were set up by using Oracle
ASM Filter Driver (ASMFD).
Chapter 4: Design Considerations
37
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
In the 11g database VM, all database storage disks were set up by using udev
rules.
Network design
Physical network design and connectivity
The following figure shows the solution’s redundant and highly available physical LAN
design and connectivity.
Figure 7. Physical network design and connectivity
LAN setup
Chapter 4: Design Considerations
38
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Note: The ports on the switches shown in the preceding figure, to which the database server
network ports are connected, are for illustration purposes only. Network administrators can choose
any available ports on the switches, as appropriate.
The four database servers tested in this solution were configured with a highly available
network:
In the two physical production RAC database servers represented by R940-Prod-
DB-Node1 and R940-Prod-DB-Node2 in Figure 7.
Each RAC node has two 10 GbE network ports for Oracle public traffic
connected to two separate ToR switches.
Each RAC node has two 10 GbE network ports for Oracle private interconnect
traffic connected to two separate ToR switches.
Each RAC node has two 1 GbE network ports connected to the management
switch. One port is for server management from within the operating system
and one port is for out-of-band management of the server by using iDRAC.
In the single physical XVC database server represented by R740-XVC-DB-Server
in Figure 7:
Two 10 GbE network ports for Oracle public traffic are configured and
connected to two separate ToR switches.
Two 1 GbE network ports are configured and connected to the management
switch. One port is for in-band server management from within the operating
system and one port is for out-of-band management of the server by using
iDRAC.
In the single virtualized database server represented by R940-Virtual-DB-Server in
Figure 7:
Two 10 GbE network ports for Oracle public traffic are configured and
connected to two separate ToR switches.
Two 1 GbE network ports are configured and connected to the management
switch. One port is for in-band server management using the VMware vCenter
Server Appliance and one port is for out-of-band management of the server by
using iDRAC.
Oracle public networks from all database servers and private interconnect networks from
the two Oracle RAC nodes are configured on the same redundant ToR 10 GbE S4048-ON
switches. However, the network traffic is segregated by using VLANs as shown in the
following table.
Table 15. Sample VLAN configuration on S4048-ON 10 GbE ToR switches
Traffic type VLAN ID
All Oracle public 16
Oracle private interconnect 100
Chapter 4: Design Considerations
39
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Note: VLAN IDs used in the table are examples. Network administrators can use VLAN IDs that
conform to their network policies and standards only if the Oracle public and private networks are
on two separate VLANs.
Virtual network design
The virtual environment in this Ready Solution for Oracle was tested by using a single
VMware ESXi host running two VMs. One VM for running a standalone Oracle 11gR2
database and the other VM for running a standalone Oracle 12c R2 database.
Figure 8. Virtual network design on the VMware ESXi database host
The preceding figure shows the virtual or VM network topology implemented in the
virtualized databases ESXi host. It also shows how the virtual layer maps to the physical
layer.
As shown in the figure, the VMware-based virtual network design in the Ready Solution
for Oracle consists of the following virtual switches and port groups:
Public distributed switch—This switch is implemented as a distributed virtual
switch for Oracle public traffic, which is generated by the two virtual databases
running inside two separate VMs. In this distributed switch, we created one
distributed port group and two uplink ports:
Public distributed port group—This group provides the virtual interfaces for
Oracle public traffic for the two Oracle database VMs, which are represented by
the VM names xorb-virt-11g and xorb-virt-12c in the figure. This port
group is tagged with VLAN ID 16, which is the same as the VLAN ID that is
configured on the S4048-ON ToR switch for public traffic.
Chapter 4: Design Considerations
40
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Two physical uplink ports—Two 10 GbE physical ports from two separate
NICs serve as uplink ports to the public distributed switch to provide sufficient
bandwidth and redundancy.
NOTE: Typically, for single ESXi host implementations, standard switches and port groups are
sufficient. Distributed switches and port groups configured in this solution provide easy expansion
to virtual Oracle RAC environments by using multiple ESXi hosts, if needed.
Management standard switch—This switch is implemented as a standard switch
for management traffic. The default VMKernel port in the Management Network port
group, vmk0, is used to manage the ESXi host from VMware vCenter. The VM
Network port group provides the virtual interfaces to access and manage the Guest
VMs. Both these management traffic interfaces use one 1 GbE LAN On
Motherboard (LOM) port in the ESXi host that is connected externally to the S3048-
ON management switch.
NOTE: Though not shown in Figure 8 or implemented during testing of this Ready Solution for
Oracle, we recommend that you configure an additional 1 GbE management network port for
redundancy.
We used the following recommended SAN connectivity and zoning best practices to
configure all the database servers with the XtremIO X2 storage arrays:
Use at least two initiators per ESXi host or physical database servers for load
balance and bandwidth. For high availability, place the two initiators on separate
HBAs.
Implement redundant FC switches for high availability.
Ensure that one FC zone set includes one HBA port or initiator and at least one FC
target port on each XtremIO storage controller.
The following figure shows the recommended physical SAN connections between the
database servers, the FC switches, and the dual X-Brick XtremIO X2 cluster storage
arrays in this Ready Solution for Oracle.
SAN setup
Chapter 4: Design Considerations
41
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 9. SAN setup: Physical design and connectivity
Note: The ports on the FC switches shown in the figure to which the database server HBA ports
and the XtremIO FC front-end ports are connected are for illustration purposes only. SAN
administrators can choose any available ports on the switches, as appropriate.
The SAN connectivity and redundancy of the components in the solution ensure that no
single point of failure exists and provides the necessary bandwidth. As shown in the
figure, the SAN setup in the Ready Solution for Oracle with XtremIO X2 storage consists
of:
Chapter 4: Design Considerations
42
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
HBAs in the database servers:
Two dual-port 16 Gb/s HBAs per database server with a total bandwidth of 8
GB/s per server. Two dual-port HBAs per node provide HA and bandwidth if
one HBA fails.
Each port on the same HBA adapter is connected to two separate FC switches
to provide redundant paths and HA if one FC port or FC switch fails.
Redundant FC switches:
Two 16 Gb/s FC switches provide redundant paths for bandwidth and
connectivity between the database servers and the storage controllers if one
FC switch fails.
Front-end FC ports in the XtremIO storage controllers:
Two 16 Gb/s FC front-end ports per XtremIO I/O storage controller with a total
bandwidth of 8 GB/s per controller.
Each FC front-end port per controller is connected to two separate FC switches
to provide redundant paths and HA if one or more FC front-end ports, one or
more storage controllers, or one FC switch fails.
Zoning overview
The following figure shows the logical view of the Oracle RAC production database
servers after the recommended zoning configurations are created on the redundant FC
switches. The same design and configuration is used for the XVC database and the
virtualized database servers. Zoning is configured so that each host initiator in the
database server is zoned to four target front-end ports that are located on four separate
XtremIO storage controllers. This configuration provides a total of 16 paths per database
server and ensures sufficient bandwidth and availability for the Oracle database servers to
reach the storage if one or more ports or HBAs, a switch, or storage FC ports or
controllers fail.
Chapter 4: Design Considerations
43
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 10. SAN setup: Zoning logical view
To test the backup and recovery solution, we connected the DD6300 system as the
backup appliance to the Ready Solution for Oracle with XtremIO X2 storage. As shown in
the preceding figure, we connected two 10 GbE ports from two separate NICs on the
DD6300 system to two separate S4048-ON 10 GbE switches. S4048-ON switches serve
as the ToR Ethernet switches for Oracle database public and private interconnect network
traffic in this Ready Solution for Oracle.
DD6300 network
design
Chapter 4: Design Considerations
44
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Figure 11. Data Domain DD6300 backup and recovery solution: IP network connections
For the databases in this Ready Solution for Oracle to communicate with the DD6300
backup appliance, the two DD6300 network ports on the S4048-ON switches were added
as untagged 10 GbE ports to VLAN (configuration) ID 16. This VLAN serves as the public
VLAN in this Ready Solution for Oracle. We connected the management port on the
Chapter 4: Design Considerations
45
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
DD6300 system to the S3048-ON 1 GbE switch that serves as the management switch in
the Ready Solution for Oracle with XtremIO X2 storage.
In the DD6300 system, we configured the two network ports with static IP addresses that
belong to the same subnet as the Oracle database public network in this Ready Solution
for Oracle. These two network interfaces were added to the default interface group under
the DD Boost protocols configuration.
XtremIO storage volume design
In this solution, we used the following design principles to implement the storage volumes
for the Oracle databases:
Create three volumes for OCR
Create at least four volumes for DATA
Create two volumes for REDO
Create one volume each for FRA and TEMP
The following figure shows the storage design of these five databases.
Figure 12. Storage design and database consolidation on XtremIO X2 storage
In this solution, the Oracle RAC production database is running across two physical nodes
or servers. We created separate volumes for OCR, DATA, REDO, FRA, and TEMP in the
XtremIO system and mapped them to both physical hosts. The following table shows the
volume design for the production database
Storage design
overview
Production
database storage
design
Chapter 4: Design Considerations
46
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Table 16. Production database volume design
Volume name
Number of volumes
Size per volume (GB)
Total size (GB)
C2-OCR 3 50 150
C2-DATA 4 300 1,200
C2-REDO 2 100 200
C2-FRA 1 200 200
C2-TEMP 1 500 500
Total 11 2,250
Two snapshot databases, DEV XVC and OLAP XVC, are hosted in a single R740-
XVC_DB_Server. As shown in Figure 12, these databases use the common OCR
volumes for the clusterware and use a single TEMP volume for the temporary tablespace.
Each snapshot database has its own DATA volumes, REDO volumes, and FRA volume.
While OCR volumes and the TEMP volume are the storage volumes, the four DATA
volumes, two REDO volumes, and FRA volume are the XVC snapshot copies of the
corresponding volumes of the OLTP RAC database (PROD). The following table lists the
XVC snapshot copies and storage volumes for the two XVC databases.
Table 17. List of storage volumes
Name Number of volumes
Snapshot/volume
Snapshot database
Source volumes
Size per volume (GB)
Total size (GB)
DATA_DEV 4 snapshots DEV XVC DATA 300 1,200
REDO_DEV 2 snapshots DEV XVC REDO 100 200
FRA_DEV 1 snapshot DEV XVC FRA 200 200
DATA_OLAP 4 snapshots OLAP XVC DATA 300 1,200
REDO_OLAP 2 snapshots OLAP XVC REDO 100 200
FRA_OLAP 1 snapshot OLAP XVC FRA 200 200
TEMP 1 volume Shared both databases
N/A 500 500
OCR 3 volumes Clusterware N/A 50 150
In the virtualized environment, we have two single-instance databases in two separate
VMs. We created shared volumes for OS, OCR, DATA, REDO, FRA and TEMP. The
following table shows the volume design for the virtualized databases.
XVC databases
storage design
Virtualized
databases
storage and ESXi
datastore design
Chapter 4: Design Considerations
47
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Table 18. Virtualized database volume design
Volume Name
Number of volumes
Size per volume (GB)
Total size (GB)
C3-VM-OS 1 600 600
C3-OCR 3 100 300
C3-DATA 4 600 2,400
C3-REDO 2 100 200
C3-FRA 1 200 200
C3-TEMP 1 500 500
Total 12 4,200
VM datatore design
The VM datastore design on vSphere follows the XtremIO storage volume design.
Therefore, for each volume created on the XtremIO storage, we create an equivalent
vSphere datastore formatted with the VMFS 6 file system with a single GPT partition that
spans the entire disk.
Similar to the XtremIO storage volume design, the VM datastore design for the virtualized
database configuration involves a separate datastore for each of the volumes as shown in the
following table.
The table shows the datastore design for the virtualized databases. During the VM
configuration of each database, we selected these dedicated datastores as the location of
each of the HDDs that we created for each of the database volume types.
Table 19. vSphere VM datastore XtremIO X2 storage volume design for OLTP 11gR2 and 12cR2 virtualized databases
Datastore name Datastore size (GB)
Purpose
C3-VM-OS 600 1 x operating system datastore for the two guest
operating systems (xorb-virt-11g and
xorb-virt-12c). Each guest operating system
Virtual Machine Disk (VMDK) is 250 GB.
C3-OCR1 100 3 x OCR datastores for normal redundancy OCR/voting disk in virtualized database.
Each OCR VMDK is 48 GB. C3-OCR2 100
C3-OCR3 100
C3-DATA1 600 4 x DATA datastores for Oracle DATA disks in the virtualized databases.
Each Data VMDK is 298 GB. C3-DATA2 600
C3-DATA3 600
C3-DATA4 600
C3-REDO1 100
Chapter 4: Design Considerations
48
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Datastore name Datastore size (GB)
Purpose
C3-REDO2 100 2 x REDO datastores for Oracle REDO disks in the virtualized databases.
Each REDO VMDK is 48 GB.
C3-FRA 200 1 x FRA datastore for Oracle FRA disks in the virtualized database
Each FRA VMDK is 98 GB
C3-TEMP
500
1 x TEMP datastore for Oracle TEMP disks in the virtualized databases
Each TEMP VMDK is 248 GB
Total datastores: 12 4,200
Database ASM storage design
The details of the storage design for Oracle RAC databases are based on the design
introduced in Production database storage design and Virtualized databases storage and
ESXi datastore design.
The following table provides the details of the storage volumes that are provisioned for the
production database in this Ready Solution for Oracle.
Table 20. Storage volumes configured for the production Oracle database
Size (GB)
Oracle ASM disk
Oracle ASM disk group
ASM striping
Oracle datafile
50 C2_OCR1 +OCR (normal redundancy)
Coarse striping
OCR files and voting disk files
50 C2_OCR2
50 C2_OCR3
300 C2_DATA11 +DATA (external redundancy)
Coarse striping
Data files, temp files, control files, undo tablespace 300 C2_DATA12
300 C2_DATA13
300 C2_DATA14
100 C2_REDO1 +REDO2 (external redundancy)
Fine-grain striping
Online redo log files.
100 C2_REDO2 +REDO 2 (external redundancy)
Fine-grain striping
Online redo log files.
200 C2_FRA +FRA (external redundancy)
Coarse striping
Archived redo logs
500 C2_TEMP +TEMP(external redundancy)
Fine-grain striping
Temp files
Chapter 4: Design Considerations
49
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The XVC database shares the same storage volumes as the production Oracle database
as shown in the following tables. The only difference is that, for an XVC database such as
XVC DEV, the four DATA volumes (DATA_11_DEV, DATA_12_DEV, DATA_13_DEV,
DATA_14_DEV), two REDO volumes (REDO1_DEV and REDO2_DEV), and one FRA
volume (FRA_DEV) are the XVC snapshot copies of the corresponding volumes of the
OLTP RAC database (PROD). OCR1-3 and TEMP are the regular volumes, not the
snapshot volumes. The same principal applies to the second snapshot database XVC
OLAP.
Table 21. Storage volumes configured for XVC DEV Oracle database
Size (GB)
Oracle ASM disk
Oracle ASM disk group
ASM striping
Oracle datafile
50 OCR1 +OCR (normal redundancy)
Coarse striping
OCR files and voting disk files
50 OCR2
50 OCR3
300 DATA11_DEV +DATA_DEV (external redundancy)
Coarse striping
Data files, temp files, control files, undo tablespace 300 DATA12_DEV
300 DATA13_DEV
300 DATA14_DEV
100 REDO1_DEV +REDO2_DEV (external redundancy)
Fine-grain striping
Online redo log files.
100 REDO2_DEV +REDO 2_DEV (external redundancy)
Fine-grain striping
Online redo log files.
200 FRA_DEV +FRA_DEV (external redundancy)
Coarse striping
Archived redo logs
500 TEMP +TEMP(external redundancy)
Fine-grain striping
Temp files
Table 22. Storage volumes configured for XVC OLAP Oracle database
Size (GB)
Oracle ASM disk Oracle ASM disk group
ASM striping
Oracle datafile
50 OCR1 +OCR (normal redundancy)
Coarse striping
OCR files and voting disk files
50 OCR2
50 OCR3
300 DATA11_OLAP +DATA_OLAP (external redundancy)
Coarse striping
Data files, temp files, control files, undo tablespace
300 DATA12_OLAP
300 DATA13_OLAP
300 DATA14_OLAP
Chapter 4: Design Considerations
50
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Size (GB)
Oracle ASM disk Oracle ASM disk group
ASM striping
Oracle datafile
100 REDO1_OLAP +REDO2_OLAP (external redundancy)
Fine-grain striping
Online redo log files.
100 REDO2_OLAP +REDO1_OLAP (external redundancy)
Fine-grain striping
Online redo log files.
200 FRA_OLAP +FRA_OLAP (external redundancy)
Coarse striping
Archived redo logs
500 TEMP +TEMP(external redundancy)
Fine-grain striping
Temp files
The following table provides the details of the storage volumes that are provisioned for
each of the virtualized databases in this Ready Solution for Oracle.
Table 23. Storage volumes configured for virtualized Oracle databases
VMware virtual disk
Size (GB)
Oracle ASM disk
Oracle ASM disk group
ASM striping Oracle datafile
Disk2 298 DATA1 +DATA (external redundancy)
Coarse striping Data files, control files, undo tablespace
Disk3 298 DATA2
Disk4 298 DATA3
Disk5 298 DATA4
Disk6 48 OCR1 +OCR (normal redundancy)
Coarse striping OCR files and voting disk files
Disk7 48 OCR2
Disk8 48 OCR3
Disk9 48 REDO1 +REDO1 (external redundancy)
Fine-grain striping
Online redo log files
Disk10 48 REDO2 +REDO2 (external redundancy)
Fine-grain striping
Online redo log files
Disk11 98 FRA +FRA (external redundancy)
Coarse striping Archived redo logs
Disk12 248 TEMP +TEMP1 (external redundancy)
Fine-grain striping
Temp files
Chapter 4: Design Considerations
51
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
XtremIO Virtual Copy (XVC) database design
XVC copies from the production database
XtremIO pioneered the concept of integrated Copy Data Management (iCDM) which
allows consolidation of both the primary database and its associated snapshot copies on
the same scale-out all-flash array. With this feature, we can group snapshots of the
volumes of the production database to form multiple copies of the production database.
XtremIO Virtual Copies (XVCs) is XtremIO’s implementation of the snapshot copy
functionality. XVCs are created by capturing the state of data in volumes at a particular
point in time. An XVC snapshot provides the users to access the data in the volume at the
time when the XVC snapshot was taken. To take the cross-consistent XVC copy of the
volumes of the PROD database, we placed all four DATA volumes, two REDO volumes,
and one FRA volume of the PROD database in a consistency group called C2_CAP as
shown in the following figure.
Figure 13. Consistency Group C2_CAP
In this architecture, we took two XVC snapshots of the consistency group C2_CAP: DEV
XVC and OLAP XVC. Creating an XVC snapshot is quick, has no impact on the PROD
database, and imposes no PROD database downtime. The snapshots do not consume
any capacity at the point when created. We create these two XVC copies for different
purposes. DEV XVC forms a development database and OLAP XVC forms an analytics
application database. We created an XVC snapshot copy of the C2_CAP consistency
group by using the Create Repurpose Copy process in the XtremIO Manager UI, as
shown in the following two figures.
Chapter 4: Design Considerations
52
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Figure 14. Create an XVC copy of consistency group GC_CAP
Figure 15. Consistency group C2_CAP and its XVC copies C2_CAP_DEV and C2_CAP_OAP
Snapshots database based on XVC copies of the production database
When the C2_CAP DEV or C2_CAP_OLAP XVC copies are created, we use them to
create the DEV_XVC and DEV_OLAP snapshot databases by mounting these XVC
copies to a database server. The following procedure provides the major steps to create
snapshot databases with the C2_CAP_DEV or C2_CAP_OLAP XVC snapshot copies.
1. Present the XVC snapshot copies to the XVC server and mount them as ASM
disks, as shown in the following table. TEMP, OCR1, OCR2 and OCR3 are the
original volumes, not the snapshots.
Table 24. Volumes and snapshots presented to the XVC server
Snapshot or volume name
Snapshot source volume
Snapshot
/volume Diskgroup Database/clusterware
DATA11_DEV C2_DATA11 Snapshots DATA DEV XVC Database
DATA12_DEV C2_DATA12
DATA13_DEV C2_DATA13
DATA14_DEV C2_DATA14
REDO1_DEV C2_REDO1 REDO1
REDO2_DEV C2_REDO2 REDO2
FRA_DEV C2_FRA FRA
DATA11_OLAP C2_DATA11 Snapshots DATA OLAP XVC Database
DATA12_OLAP C2_DATA12
DATA13_OLAP C2_DATA13
Chapter 4: Design Considerations
53
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Snapshot or volume name
Snapshot source volume
Snapshot
/volume Diskgroup Database/clusterware
DATA14_OLAP C2_DATA14
REDO1_OLAP C2_REDO1 REDO1
REDO2_OLAP C2_REDO2 REDO2
FRA_OLAP C2_FRA FRA
TEMP N/A Volume TEMP Shared by DEV XVC and OLAP XVC
OCR1 N/A Volume OCR Clusterware
OCR2 N/A Volume
OCR3 N/A Volume
Note: The entries marked as “Snapshots” are the XVC copies of the PROD database and are
mapped to the XVC server. The entries marked as “Volume” are the actual storage volumes
mapped to the XVC servers. These volumes are TEMP and OCR1, OCR2, OCR3, which are not
based on snapshot copies.
2. Provide new diskgroup names. When the XVC copy takes an XVC snapshot, the
XVC snapshot has the ASM disk name and the ASM diskgroup name as its
ancestor volume. When mounting an XVC snapshot to form a new database,
change its ASM diskgroup name and ASM disk name to avoid duplicating ASM
diskgroup names. The following table shows the new ASM diskgroup names for
these XVC snapshots.
Table 25. ASM diskgroup renaming map
Volume name Original diskgroup name New diskgroup name Snapshot database
DATA11_DEV DATA DATA_DEV DEV XVC
DATA12_DEV
DATA13_DEV
DATA14_DEV
REDO1_DEV REDO1 REDO1_DEV
REDO2_DEV REDO1 REDO1_DEV
FRA_DEV FRA FRA_DEV
DATA11_OLAP DATA DATA_OLAP OLAP XVC
DATA12_OLAP
DATA13_OLAP
DATA14_OLAP
REDO1_OLAP REDO1 REDO1_OLAP
REDO2_OLAP REDO1 REDO1_OLAP
Chapter 4: Design Considerations
54
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Volume name Original diskgroup name New diskgroup name Snapshot database
FRA_OLAP FRA FRA_OLAP
3. Change all the ASM diskgroup names in all the file paths for all the database files,
control files,and redo logs. Also change the ASM diskgroup names in all
references to file paths and in the destinations setting in the spfile. This
modification is necessary because all these file paths refer to the old diskgroup
names that are no longer valid because the ASM diskgroup name has changed..
For example, the following SQL command renames the database file:
SQL>alter database rename file
'+DATA/DBCAP/DATAFILE/users.261.986035293' to
'+DATA_DEV/DBCAP/DATAFILE/users.261.986035293';
4. Change the database name and the DBID with the DBNEWID utility. The
DBNEWID utility enables you to assign the new database name and the new
database ID to replace the original production database name from the primary
database (PROD). Then, change the dbname in the spfile.
5. To open the new database, use the alter database reset logs command.
Disable the archive log mode if the new database is for development and an
archive log is not required.
For detailed steps about how to create the snapshot databases by using XVC snapshot
copies from the PROD database, see the Dell EMC Ready Solutions for Oracle with Data
Protection Deployment Guide.
Data Domain backup system design
During the database backup operation with Oracle RMAN, the Oracle database sends
backups to the Data Domain system through the network, which can be Fibre Channel or
Ethernet. We selected DD Boost over Ethernet protocol to take advantage of the proven
performance and deduplication features of DD Boost technology. In this configuration,
both the DD Boost feature and the distributed segment processing (DSP) are enabled. DD
Boost software runs on both the Oracle database server and the Data Domain system. As
shown in the following figure, for each backed-up segment, the DD Boost software
determines if the segment is unique (has not been previously stored in the Data Domain
system). When DD Boost confirms that the segment is unique, the segment is
compressed and transferred over the network and stored on the Data Domain system.
The deduplication and compression processes ensure that only unique data is
compressed and sent over the network and stored in the Data Domain system.
DD Boost
technology
Chapter 4: Design Considerations
55
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 16. Oracle RMAN backup to the Data Domain system with DD Boost software
During the first full database backup, because no data from this database was stored in
the Data Domain system, all the data segments from the backup are unique. Therefore,
each data segment from the first full backup is compressed, sent over the network, and
stored in the Data Domain system. Starting with the second full backup, the DD Boost
software backs up only those unique data segments that were not previously stored in the
Data Domain system.
Figure 17. Show compression output
Two related factors are used to measure the effectiveness of the DD Boost deduplication
and compression features:
Total-Comp Factor = 1054.5/784.2 = 1.3x
Chapter 4: Design Considerations
56
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Reduction % = ((Pre-Comp-Post-Comp)/Pre-Comp)* 100 = (1054.5-784.2)/
1054.5 = 25.63%,
The first result, 1.3x, is the total compression factor achieved by DD Boost deduplication
and compression. The second result, 25.63%, shows that DD Boost technology reduced
the storage usage and network bandwidth by 25.63 percent during the subsequent
backup.
The Data Domain system includes a set of disks that store database backups. During the
initial Data Domain configuration, these disks are assigned to disk groups to be used to
create file systems for storing database backups. For example, the DD6300 system has
one head unit with 14 disks plus one additional disk enclosure (DS60) with 60 disks. As a
default configuration, the disk group dg0 as a base unit is created with 12 disks from the
head unit. That is, 11 disks (1.1–1.10, 1.12) plus one disk (1.11) as a spare disk, for a
total of 40 TiB usable storage capacity that can be used to store the database backup
images.
For more storage, 60 additional storage disks in the disk enclosure DS60 can be added
into DD6300.
During Data Domain system initialization, a file-system-enabling command on the Data
Domain system command line enables the file system. The following command shows
the current space usage of the file system in a DD6300.
Figure 18. Data Domain DD6300 file system space status
When the file system is enabled, the storage space from the disks becomes available. To
perform operations such as backup, restore, or remote replication, create logical volumes
in the Data domain system.
The logical disk volumes created in the Data domain system are called as storage units.
The Data Domain system exposes volumes called as storage units to a backup server
enabled with the DD Boost software.
Create one or more storage units on the Data Domain system to use with the database
application agent on the database server to back up the database files, as shown in the
following example:
Storage and file
system
Mtree and
storage unit
Chapter 4: Design Considerations
57
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 19. Storage units
These storage units are shown as a logical partition of the Mtree file system:
Figure 20. Mtrees for the storage units
To implement the Oracle optimized deduplication feature in a Data Domain system, use
the following command to set the value of the app_optimized-compression option to
oracle1 on the Mtree:
mtree option set app-optimized-compression oracle1 mtree
<storage_unit_name>
For example, run these commands in the command line on the Data Domain system for
storage unit slob_unit_7 and slob_unit_10:
mtree option set app-optimized-compression oracle1 mtree
/data/col1/slob_unit_7
mtree option set app-optimized-compression oracle1 mtree
/data/col1/slob_unit_10
A Data Domain system connects to the Oracle Ready Solution configuration through an
Ethernet network as a backup appliance. The physical connectivity between the Data
Domain system and the Oracle Ready Solution configuration is based on two ports of 10
GbE network interface controllers (NICs) that are installed on the DD6300 system and
four ports of two 10 GbE NICs that are installed on the DD9300 system. For a detailed
description about the connectivity design, see Network design.
We created a network interface group on the Data Domain system by adding these
interfaces to this group.
The following figure below shows that two network interfaces, 172.16.191.30 and
172.16.191.31, were added to the default interface group in the DD6300 system.
IP network
design
Chapter 4: Design Considerations
58
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Figure 21. Interface group in the Data Domain DD6300 system
To register and connect the database server as a client with the Data Domain system,
select the static IP address assigned to one of the interfaces on the Data Domain. By
internally enabling load balance and failover capability among the network interfaces
configured in a group, the interface group configuration provides a high network
bandwidth and a highly available backup network between the database servers and the
Data Domain system.
To increase the RMAN backup/restore throughput, establish a number of parallel backup
channels with the RMAN backup or restore. On an Oracle RAC database system, we can
take advantage of the multiple-instances architecture of the RAC database to scale the
RMAN backup workload by distributing multiple parallel backup channels over multiple
RAC database instances. Multiple backup channels direct the connections to each
instance of an Oracle RAC database.
The PARALLELISM setting in the RMAN backup and restore script defines the total
number of parallel RMAN backup or restore channels. The setting varies depending on
the database size, CPU utilization, backup throughput, and backup time. In general, more
parallel channels can lead to a higher backup throughput with a shorter backup time, but
also require a higher CPU utilization and more network bandwidth.
For example, we used a total of four channels in the first full backup performance test with
two channels connecting to each RAC database instance, as shown in the following
figure.
Multiple channel
backup and
restore with
Oracle RAC
Chapter 4: Design Considerations
59
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 22. Multiple channels for Oracle RAC Database backup and restore
Configure Oracle RMAN backup and restore by providing specific parameter settings. For
commercial backup with the DD6300 system, we used the following settings in our backup
and restore tests for all the databases.
Table 26. RMAN backup and restore parameter settings
Operation Parameter Setting
Prod database backup PARALLELISM 4
Prod database backup SECTION SIZE 4 G
Prod database backup BLKSIZE 1048576
Prod database restore PARALLELISM 4
Prod database restore BLKSIZE 1048576
RMAN backup
and restore
parameters
Chapter 5: Test Methodology and Results
60
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Chapter 5 Test Methodology and Results
This chapter presents the following topics:
Test objective .................................................................................................... 61
Test tools and methods .................................................................................... 62
Use case 1: Ease of storage provisioning with XtremIO storage .................. 62
Use case 2: Baseline—One production OLTP RAC database ........................ 66
Use case 3: Impact of one 11gR2 and one 12cR2 virtualized database on the baseline ........................................................................... 78
Results Summary: Mixed physical and virtual environments running mixed OLTP and OLAP workloads on Ready Solution for Oracle with XtremIO X2 ......................................................... 83
Chapter 5: Test Methodology and Results
61
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Test objective
This Ready Solution for Oracle on XtremIO X2 storage provides simple, easy-to-use
management through an interface that allows storage administrators to provision storage
with little setup or planning. Also, it provides predictable and consistent low-latency
performance with sub-0.85 millisecond response times regardless of the workload and
environment that can include production, QA, test, or development in typical enterprise
applications. Multiple copies might be required for test/development, reporting, or online
analytics.
DBAs and test/dev engineers often spend hours managing database creation and
refreshing the environments while often being limited by capacity, performance, and
number of copies. XtremIO's Integrated Copy Data Management (iCDM) enables you to
create XtremIO Virtual Copies (XVCs) from production instantly with no performance
impact. These copies can be repurposed for near-real-time analytics, test/dev, or any
other use case—all with complete space efficiency.
Protecting the database is easy with XtremIO. There is no need to consider the
complexities of RAID type, data file capacity, or load balancing. The data is protected with
the XtremIO Data Protection (XDP) proprietary flash-optimized algorithm. XDP is different
from RAID in several ways. Because XDP is always working in an all-flash storage array,
several criteria were important in the design of this protection scheme. XDP benefits
include ultra-low capacity overhead, high levels of data protection in case of double SSD
failure, rapid rebuild times, flash endurance, and extreme performance.
In this Ready Solution for Oracle, we use Data Domain to back up and recover an Oracle
database. Data Domain benefits are the same as XDP benefits. XtremIO virtual copies
provide easy protection and recovery from any operational and logical corruption. XVCs
enable the creation of frequent point-in-time copies (according to RPO intervals –
seconds, minutes, hours) and use them to recover from any data corruption. An XVC can
be kept in the system for as long as it is needed. Recovery using an XtremIO virtual copy
is instantaneous and does not impact system performance.
To demonstrate the benefits of this solution, Dell EMC has designed a series of
performance tests using OLTP and OLAP databases. OLTP database usage is
characterized by small requests for information, such as looking up an inventory item or
checking a customer account, and supporting mission-critical back-office applications.
ERP and CRM systems can support thousands of users who generate millions of
database transactions and require fast response times. In this case, response time is the
total amount for the database to respond to a request. For our OLTP tests, we used an
aggressive response time goal of under 0.85 milliseconds to measure success.
Chapter 5: Test Methodology and Results
62
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Test tools and methods
To simulate both OLTP and OLAP database workloads, we used the HammerDB tool
version 3.1. OLTP workload was generated through standard TPC-C like queries from
HammerDB, and the OLAP workload was generated through a custom query run on the
OLAP XVC database. Our tests featured 70 virtual users on a production database and 10
virtual users each on OLTP XVC databases and virtualized databases. We created a
HammerDB dataset with 9,000 warehouses. The OLTP workloads were made up of 65
percent reads and 35 percent writes. The OLAP custom reporting query performs read-
only operations. For detailed HammerDB parameter settings, refer to the HammerDB
configuration parameters section in Appendix A.
Use case 1: Ease of storage provisioning with XtremIO storage
This use case describes the ease of configuring XtremIO X2 storage for Oracle Database.
We created required volumes from XtremIO storage for the Oracle 12c standalone
database installation by considering these design principles:
Created four volumes for DATA to optimize queuing
Created three volumes for OCR for redundancy
Created separate volumes for REDOLOGS for ease in monitoring performance
Created separate volume for FRA because it is not needed for repurposing a
database
Created separate volume for TEMP because it is not needed for repurposing a
database
All these volumes were created from the XtremIO X2 management console. Volume
provisioning in XtremIO X2 storage is simple and straightforward. You can create volumes
and present them to the Oracle database servers with just a few clicks from the XtremIO
X2 management console. To create volumes and map them to the servers:
1. Log in to the XtremIO X2 web management console by providing credentials:
Chapter 5: Test Methodology and Results
63
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 23. XtremIO Management login screen
2. In the XtremIO X2 management console, click Configuration, click Volumes,
and then click Create New Volume.
Figure 24. Creating storage configuration volumes
3. Enter the number of volumes, the volume name prefix, the size of the volumes,
and then click APPLY.
Chapter 5: Test Methodology and Results
64
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Figure 25. Enter new storage volume parameters
4. Repeat step 1 through step 3 to create volumes for the Oracle 12c database, as
listed in the following table.
Table 27. Storage volume parameters
Volume Name
Number of volumes
Size per volume (GB)
Total size (GB)
C1-OCR 3 50 150
C1-DATA 4 250 1,000
C1-REDO 2 100 200
C1-FRA 1 100 100
C1-TEMP 1 100 100
Total 11 1,550
5. Select all the newly created volumes and click Mapping.
Chapter 5: Test Methodology and Results
65
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 26. Select volumes for mapping
6. Select the Initiator Group (Server) to which the volumes will be mapped and click
NEXT.
Figure 27. Select initiator group
Chapter 5: Test Methodology and Results
66
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
7. Verify mapping confirmation and click APPLY.
Figure 28. Mapping confirmation
8. After mapping newly created volumes that are mapped to the initiators, execute
rescan-scsi-bus.sh on all hosts to display volumes.
This use case demonstrates the ease of provisioning XtremIO volumes for the Oracle 12c
database and explains step-by-step provisioning of volumes to Oracle 12c database
servers. It takes just a few mouse clicks to provision a volume from XtremIO storage to
the Oracle 12c database.
Use case 2: Baseline—One production OLTP RAC database
In the second use case, we created two Oracle 12c Release 2 RAC databases across two
PowerEdge 940 servers, as shown in the figure below. We used the HammerDB test tool
to generate an OLTP workload with a 65/35 read/write mixture. We created the databases
with an 8 KB block size and with ASM in a coarse-striped and externally redundant
configuration. In this use case, we performed multiple stress tests to understand the
performance and other capacity numbers that are generated under different workloads
such as:
2 nodes RAC Production DB (Test 1)
Test 1 + two SNAP DBs generated from RAC DBs (Test 2)
Test 1 + Test 2 + one 11g Virtualized DB + 12c Virtualized DB (Test 3)
Chapter 5: Test Methodology and Results
67
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Test 1
XtremIO X2 storage is designed to unlock flash technology's instant performance potential
by uniquely leveraging the characteristics of SSDs. It uses advanced inline data reduction
methods to reduce the physical data that must be stored. XtremIO's storage system uses
industry-standard components and custom-designed intelligent software to deliver
unparalleled levels of performance, achieving consistently low latency for up to millions of
IOPS.
The XtremIO Management Server (XMS) delivers an HTML5 user interface that is simple
and easy to use for storage administrators. XMS enables storage administrators to
provision storage with little setup or planning as shown in use case 1. In this use case we
created two Oracle 12c Release 2 RAC databases across two PowerEdge 940 servers,
as shown in the following figure. We used HammerDB to create an OLTP workload with a
65/35 read/write mixture. We created the databases with an 8 KB block size and with
ASM in a coarse-striped and externally redundant configuration. The detailed
configuration of the two 12c RAC OLTP databases are documented in use case 1. In this
use case, we performed a series of tests and we will discuss the performance results with
RAC, SNAP, and virtualized databases that are either repurposed or newly created. In
test 1, we already created the RAC OLTP DB as shown in the following figure.
Figure 29. Test 1 architecture
The following table shows the high-level configuration of the two production Oracle RAC
databases.
Table 28. Production Oracle RAC database configuration
Category Specification/setting PROD configuration
Operating system VM guest OS RHEL 7.4
Database configuration Database version 12c R2 RAC
Database size 1 TB
Performance
impact of two
snapshot
databases
Chapter 5: Test Methodology and Results
68
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Category Specification/setting PROD configuration
db_block_size 8 KB
db_file_multiblock_read_count 4
sga_max_size 64 GB
pga_aggregate_target 200 GB
SLOB I/O configuration Read/write ratio 65/35
We ran the two production Oracle RAC databases on dedicated PowerEdge R940 servers
and a dedicated XtremIO X2 array. The goal of this test was to develop and validate
implementation best practices for running Oracle databases on this platform. We
monitored performance. Because the two Oracle 12c RAC databases had dedicated
servers and storage, performance measurements do not reflect the consolidation
capabilities of the database platform. Most customers consolidate databases to achieve
greater capital and operation expenditure savings and gain more value from their
investment in licensing the databases. By generating an OLTP workload, the goal was not
to maximize performance but to create a realistic production workload. Results of the
testing are shown in the following figure.
The average CPU utilization across this test is minimal, which provides significant room
for growth. The Oracle RAC database generated over 120,828 IOPS, demonstrating a
significant production workload.
120,828
Total IOPS
Test 1 - IOPS
Chapter 5: Test Methodology and Results
69
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The average latency for the Oracle 12c RAC was .39 for reads and .66 for writes and well
under our target threshold of .85 milliseconds. Results show the XtremIO X2 array
delivered strong storage performance for a production-like workload of 120,828 IOPS.
Transactions per Minute (TPM) measures the number of completed transactions the SQL
Server database is able to complete in one minute and is an indicator that DBAs can use
to evaluate database activity. The two-node Oracle 12c RAC database was able to
process 383,591 TPM in this test, showing the capability to support a large OLTP
workload.
0.39
0.66
Average read latency Average write latency
Test 1 - Latency
383,591
TEST 1 - TPM
Chapter 5: Test Methodology and Results
70
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The following table shows how space savings work. The production database was
provisioned with 1,200 GB of disk volumes on the XtremIO X2 array, and the database
used 628.98 GB physical space after compression. The XtremIO XVC processed the
production copies for OLTP. XtremIO compression in this test has generated the Data
Reduction Ratio of 1.8:1. The overall efficiency ratio is 1.85:1.
Table 29. Different XtremIO ratios depicting the power of deduplication and compression
Parameter Value
Test case number 2
Description Production database with 720 GB data
Number of volumes 4
Databases Production DB
Total volume size (GB) 1,200
Host accessible size (GB) 1,166
Logical used (GB) 1,165.35
Unique physical space (GB) 628.98
Data reduction ratio 1.8:1
Copy efficiency 1.0:1
Thin provisioning ratio 0.97:1
Overall efficiency ratio 1.85:1
Test 2
IT organizations are under increased pressure to update and add new features to
applications quicker and more frequently. To address business demands for faster
updates and new features, many IT organizations look for opportunities to increase
efficiencies by using automation. In this use case, we used the XtremIO XVC feature to
create production copies and repurpose them for development. XtremIO's Integrated
Copy Data Management (iCDM) allows for instant XtremIO Virtual Copies (XVCs) to be
created from production with no performance impact. These copies can be repurposed for
near real-time analytics, test/dev, and any other use case—all with complete space
efficiency, reducing the time it takes to provision a test or development database.
Application and database administrators can repurpose a production copy on demand.
Using XtremIO Virtual Copies (XVCs) accelerates repurposing production copies and
enables the IT organization to fully automate the complex and time-consuming process.
The following figure depicts the architecture used for test 2 of use case 2.
Chapter 5: Test Methodology and Results
71
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 30. Architecture of test 2 of use case 2
For a detailed description of snapshot databases and their parameters, see
Chapter 5: Test Methodology and Results
72
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
XtremIO Virtual Copy (XVC) database design. Using the XtremIO XVC to create snapshot
databases, our testing yielded the performance numbers shown in the following figure.
OLAP workloads are used for data mining in which a DBA or developer examines large
data sets to generate new informational insights that assist the business with decisions.
OLAP workloads are characterized by reading large sets of historical data that makes
storage throughput the key metric for OLAP workloads. Both the production OLTP Oracle
RAC database and the Oracle OLAP database where running in parallel. Even with the
increase in workload, the XtremIO X2 array was able drive 1.43 GB/s of throughput.
The production OLTP RAC database performance experienced a minor loss in IOPS.
However, the key metric of latency was consistently strong.
Chapter 5: Test Methodology and Results
73
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The overall average storage latency was .44 ms for reads and .65 ms for writes. The
minor increase in read latency is due to the large database table scales in the OLAP
database. The write latency is interesting, as the OLAP workload did not perform many
writes. So the write latency was actually better than in test 1, our baseline. The consistent
latency performance means that application users continue to have good application
response times.
Test 3
In this test, we created one OLTP snapshot database from the existing OLTP production
database. Details of the snapshot OLAP and OLTP databases architecture are shown in
the following figure.
0.44
0.65
Read Latency Write Latency
Overall Average Storage Latencies
Chapter 5: Test Methodology and Results
74
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Figure 31. Architecture of running an OLAP and OLTP snapshot with the PROD RAC database
Based on the architecture shown in the figure, we derive the following performance
statistics, as shown in the following figure.
Chapter 5: Test Methodology and Results
75
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
In test 3, we added another OLTP workload by repurposing a production snapshot. Our
original production OLTP, OLAP, and a second OLTP workload were running in parallel,
which is a demanding workload for an XtremIO X2 array with 36 flash drives.
120,828
125,305
Test 1 Test 3
Total IOPS
120,828
125,305
Test 1 Test 3
TOTAL IOPS
Chapter 5: Test Methodology and Results
76
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Overall, the XtremIO X2 array delivered 6.4 percent more IOPS from test 1 to test 2.
However, IOPS alone do not provide the full performance picture.
Overall average storage latency was sub-millisecond with .62 ms for reads and .81 ms for
writes. Test results for average write latency slightly exceeded our goal of .85
milliseconds, which was well under the gold standard of 1 ms for all-flash arrays. The
difference in .06 milliseconds (.81 - .85 milliseconds ) most likely does not impact the
application experience for the end user. Therefore, it is our opinion that these latencies
are good considering the incremental increase in workloads and the XtremIO X2
configuration.
0.66
0.81
1 2
Overall Average Storage Latency
Average OLTP Write Latency
383,591
389,341
Test 1 Test 3
Total TPM
Chapter 5: Test Methodology and Results
77
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The total Transactions per Minute (TPM) also increased by 1.50 percent from test 1 to test
3. In test 3, the XtremIO X2 array supported 389,341 TPM, showing the capability of the
array to drive greater workloads with a minimal configuration.
These incremental workload tests are designed to push the small XtremIO X2 array to its
limits, while challenging the array to meet or exceed the aggressive goal of .85
milliseconds of latency or less while supporting a mixed workload of OLTP databases and
one OLAP workload. Test results show that the XtremIO array has continually delivered
.85 sub-millisecond latencies with the exception of test 3.
Results show that IOPS and TPM have increased, meaning that the XtremIO array can
consolidate challenging workloads and deliver on performance. Customers can
standardize their database ecosystems on XtremIO X2 storage starting with a small
configuration as a means of minimizing their initial investment. Then, over time, they can
add to the XtremIO configuration to address the continued growth of the database
ecosystem.
This following table shows that when the two snapshots are repurposed, the overall
efficiency, copy efficiency, and data reduction ratio improved considerably. We can see
that by adding two snapshots, the unique physical space usage increased by a small
amount (631.27-628.98 = 2.29 GB). This space is mainly used for storing the Oracle ASM
diskgroup metadata, new control files, and spfiles for the snapshot databases. This result
confirms that creating XtremIO snapshots does not take additional physical space in the
XtremIO storage.
Chapter 5: Test Methodology and Results
78
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Table 30. Improvement in the XtremIO ratios when two snapshots are added to the production RAC database
Parameter Value Value
Test case number 2 3
Description Production database with 720 GB data
Production database + two snapshot databases with 720 GB data
Number of volumes 4 12
Databases Production DB Production database and two snapshot databases
Total volume size (GB) 1,200 3,600
Host accessible size (GB) 1,166 3,535
Logical used (GB) 1,165.35 1,206.48
Unique physical space (GB) 628.98 631.27
Data reduction ratio 1.8:1 1.9:1
Copy efficiency 1.0:1 2.9:1
Thin provisioning ratio 0.97:1 0.335:1
Overall efficiency 1.85:1 5.6:1
Chapter 5: Test Methodology and Results
79
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Use case 3: Impact of one 11gR2 and one 12cR2 virtualized database on the baseline
In this use case, all the OLTP workloads were combined across two PowerEdge R940
servers and one PowerEdge R740 server. The advantages of combining all OLTP
workloads include greater capital and operation expenditure savings, consolidation, and
ease of management.
As shown in the following figure, we ran one production RAC database in parallel with six
snapshot development and OLAP databases plus two virtual 11g and 12c databases for a
total of five mixed workload databases. We used HammerDB to create an OLTP workload
with a 65/35 read/write mixture. We created all the OLTP databases with an 8 KB block
size and with ASM in a coarse-striped and externally redundant configuration. The
following figure shows the details of the architecture of use case 3.
Figure 32. Architecture of use case 3
The OLTP and OLAP VMs were similar in configuration, with 18 more vCPUs and 72
pCPU. At the database configuration level, each development database was configured
with an sga_max_size of 12 GB and the production database was configured with a larger
sga_max_size of 32 GB. The following figure shows the performance of this use case.
For the final and most challenging incremental workload test, we added two virtualized
Oracle databases. One virtualized database was 11gR2 and the other was 12cR2. Both
virtualized databases generated an OLTP workload. Adding virtualized databases to the
workload extends our mixed workload testing to include a mixed platform that includes
both physical and virtual workloads.
Chapter 5: Test Methodology and Results
80
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The test results show a steady increase in IOPS with test 4 generating 131,764 IOPS.
This result is a 5.15 percent increase in IOPS from test 3 to test 4. Recall that in test 3,
our latency for writes exceeded the goal of .85 milliseconds .
125,305
131,764
Test 3 Test 4
Total IOPS
Chapter 5: Test Methodology and Results
81
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
The overall average read latency was .66 ms and the write was .77 which is slightly higher
than our goal of .85 milliseconds . Both DBAs and storage administrators understand that
workloads fluctuate over time and there is an exceptional range that meets business
SLAs. In this case, there is a minor difference of .02 ms (.77 - .85 milliseconds ) for our
write latency. Latency results across all our tests show consistently low latency times.
0.66
0.77
Average OLTP Read Latency Average OLTP Write Latency
Overall Average Storage Latency
389,341
451,561
Test 3 Test 4
Total TPM
Chapter 5: Test Methodology and Results
82
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The total TPM increased most significantly in performance from test 3 to test 4. In test 4,
we achieved 451,561 TPM, which is a 15 percent increase from the prior test. This test
result shows that XtremIO X2 storage is a strong platform for consolidating OLTP
workloads. Performance trends across all our tests show that IOPS consistently increased
as we added more OLTP workload.
IOPS consistently increased as we added more OLTP workload. The XtremIO X2 array
was able to scale with our database workloads, showing its capability as a database
consolidation platform.
In the following chart, the blue bars indicate test 1, the green bars indicate test 3, and the
purple bars indicate test 4. Overall average storage latency across the three tests was
under .66 ms for reads and .81 milliseconds for writes. More importantly, the XtremIO X2
array delivered consistent sub-millisecond latencies under increasing workloads,
strengthening the conclusion that this array is ideal for database consolidation.
120,828
125,305
131,764
Test 1 Test 3 Test 4
Total IOPS
Chapter 5: Test Methodology and Results
83
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
0.39
0.660.62
0.81
0.66
0.77
Average OLTP Read Latency Average OLTP Write Latency
Overall Average Storage Latency
383,591
389,341
451,561
Test 1 Test 3 Test 4
Total TPM
Chapter 5: Test Methodology and Results
84
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Finally, the TPM results show the most positive gain for supporting increasing volumes of
transaction. The XtremIO X2 array delivered a significant increase in TPM from test 1
through test 4.
All the results show how XtremIO X2 storage has been designed to support demanding
database workloads. In particular, this XtremIO configuration had 36 flash drives but
supported over 130,000 IOPS at sub-millisecond latencies. From a transactional workload
view, this small all-flash configuration drove over 450,000 TPM.
Results Summary: Mixed physical and virtual environments running mixed OLTP and OLAP workloads on Ready Solution for Oracle with XtremIO X2
We tested mixed physical and virtual environments running mixed OLTP and OLAP
workloads on Ready Solution for Oracle with XtremIO X2 storage. The tests performed in
these use cases demonstrate that you can consolidate all types of Oracle databases with
sub-millisecond latencies and strong throughput. Virtualization combined with XtremIO X2
inline deduplication and compression enable greater consolidation and disk space saving
on this database platform.
Chapter 6: Test Methodology and Results: Data Protection
85
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Chapter 6 Test Methodology and Results: Data Protection
This chapter presents the following topics:
Test objective .................................................................................................... 85
Use case 1: Full backup of one OLTP RAC database ..................................... 85
Use case 2: Restore and recovery of one OLTP RAC database from full backup ......................................................................................... 88
Data protection testing summary .................................................................... 88
Chapter 6: Test Methodology and Results: Data Protection
86
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Test objective
Traditional ERP and CRM business applications are under constant pressure to protect
and recover data in the shortest possible time. Organizations are greatly interested in
achieving high throughput, low CPU utilization, fast backup and recovery time, low storage
I/O response time, and so on. However, Oracle DBAs and other data center staff are also
interested in achieving high data compression for Oracle backup sets, high network and
storage throughputs, and high storage IOPS during the backup and restore times.
Dell EMC has designed a series of backup and recovery tests by using RAC OLTP
databases on a Data Domain appliance. The backup and recovery of OLTP databases is
of great importance as data stored in OLTP is loaded with a huge volume of organization-
wide transactional or inventory data that supports mission-critical back-office applications.
Faster response and recovery of this data is of particular importance, especially during
longer database downtime.
During this testing we used the Data Domain DD6300 system with DD Boost. This
configuration has yielded superior performance for over 10 years in many challenging
situations.
The Data Domain system with DD Boost software enhances the speed of backup and
recovery with high levels of deduplication and compression. This method accelerates
speed and offers space savings with increased reliability of data restoration and recovery.
In our test environment, we performed multiple tests in different use cases and achieved
impressive results.
Our benchmark for success requires that the backup and recovery time is below 40
minutes for 1 TB of data with higher storage throughput under all circumstances, that
space savings is 28 percent or more, and that the storage throughput is 450 MB/sec or
more.
Use case 1: Full backup of one OLTP RAC database
We performed a full backup of a 1 TB Oracle database by using DD Boost software. DD
Boost software integrates with RMAN and enables host-based deduplication of database
backups to the Data Domain appliance. A full backup eliminates reliance on other
backups, simplifying the management of backups and simplifying restoration after an
unplanned failure.
In this use case, we used the DD Boost appliance to perform the full backup of the
production database. In the tested configuration, we used a LAN connection to the Data
Domain appliance, as shown in the following figure.
Chapter 6: Test Methodology and Results: Data Protection
87
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Figure 33. Use case 1 architecture diagram
The first full backup of an Oracle database is entirely unique; thus, all the data is protected
on the Data Domain system. Therefore, the DD Boost software only sends a small subset
of information to the Data Domain system for protection. Although the first full backup is
unique, when the data has been protected on the Data Domain system, it then is
compressed, as shown in the following figure.
Figure 34. Data compression statistics after the backup in Data Domain Appliance
A Data Domain system uses a local compression algorithm developed specifically to
maximize throughput as data is written to disk. The default algorithm (lz) allows shorter
backup windows for backup jobs but uses more space. Two other types of local
compression are available, gzfast and gz. Both provide increased compression over lz,
but at the cost of additional CPU load. Local compression options provide a trade-off
between slower performance and space usage. It is also possible to disable local
compression.
Chapter 6: Test Methodology and Results: Data Protection
88
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
The following figure demonstrates a local compression factor savings that is based on the
default algorithm (maximized throughput) on the Data Domain system. There is a
relationship between the amount of unique data and the local compression factor: the
greater the amount of unique data, the more opportunity for compression and the higher
the compression factor. For example, the first backup consists of entirely unique data and
has the largest compression factor. The following figure illustrates the compression.
Figure 35. Database server size and Data Domain backup comparison
This compression saves significant space on the Data Domain system. Dell EMC
engineering test results show that the compression factor was 1.4X: that is a 40 percent
space savings for the full backup. The 1 TB Oracle RAC database was backed up in 36
minutes to the Data Domain system. Because this backup was the first full backup and all
the data is considered unique, the backup time shows the capability of the business to
protect databases quickly with the Data Domain system.
Chapter 6: Test Methodology and Results: Data Protection
89
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Use case 2: Restore and recovery of one OLTP RAC database from full backup
Backing up and protecting databases enables recovery from an unplanned failure.
Unplanned failures can represent significant risk to the business by stopping back-office
operations, thus impacting revenue. In this test, we performed a restore from the Data
Domain system backed up to the PowerEdge R740 servers. The goal of this test is to
show a fast restore time of a 1 TB Oracle RAC database that has been protected in the
Data Domain system.
Figure 36. Use case 2 architecture diagram
In this use case, the total recovery time includes restoring the database from Data Domain
using RMAN and opening the database for processing. Restore time alone does not
represent that the database is open and available to the business. In this test, we showed
that a 1 TB Oracle RAC database can be fully recovered from backup in less than 33
minutes.
Data protection testing summary
In summary, this final test shows that the data protection benefits Oracle DBAs by freeing
up CPU resources. In the two use cases described in this chapter, we see that the time of
backup and restore are 36 and 32 minutes and 40 seconds respectively, which are
reasonably low. Also, we see that the storage throughput in these use cases is 482
MB/sec and 571 MB/sec respectively. The following table shows the details.
Table 31. Comparative analysis of backup and restore of Oracle database on XtremIO storage
Parameter name Backup Restore
Size (TB) 1 1
Time (Mins) 36 32.67
Chapter 6: Test Methodology and Results: Data Protection
90
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Parameter name Backup Restore
IOPS 8,060 9,149
Storage throughput (MB/sec) 482 571
Compression factor 1.4X NA
Chapter 7: Conclusion
91
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Chapter 7 Conclusion
This chapter presents the following topics:
Conclusion ........................................................................................................ 91
Benefits .............................................................................................................. 91
Summary ........................................................................................................... 92
Chapter 7: Conclusion
92
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Conclusion
Database systems contain the most critical data for companies, therefore, these complex
systems remain in the data center. However, enterprises want solutions with cloud
characteristics such as scalability, performance, consolidation, automation, centralized
management, and protection. Extensive testing of Ready Solutions for Oracle has stress-
tested every component of the system to validate that this database platform delivers
value. From an owner or administrator perspective, management is simplified because
Dell EMC delivers and supports the entire stack. This solution works for every Oracle
ecosystem, and those IT organizations using VMware virtualization or Dell EMC
infrastructure will find this to be a complementary solution that integrates quickly into the
existing data center.
Benefits
Testing of the Ready Solutions for Oracle with XtremIO X2 and Data Domain shows that
the database solution scales well, supports multiple workload types, and enables
aggressive consolidation of the enterprise’s ecosystem. Scalability is essential as
databases grow in size and number over time. In all use cases, the testing shows that
systems can start with just a few databases and support multiple workloads without
impacting the overall database performance.
Today’s consolidated data centers must demonstrate the ability to support multiple types
of workloads. The capability to consolidate types of workloads enables the business to
remove dedicated silos that increase complexity and costs. Testing of the Ready
Solutions for Oracle has proven that mixed workloads can be easily consolidated on this
solution.
Consolidation of databases to fewer servers results in significant savings for the business.
Lower operating and capital expenditures are two possible savings vehicles. Ready
Solutions for Oracle have been verified to support RAC, Snap, and Virtualized databases
in a heterogeneous workload environment. Even the most aggressive testing resulted in
unused resources that could host even more databases.
The use cases included two PowerEdge R940 servers, one PowerEdge R740 server and
an XtremIO X2 array. The following is a review of the results of use cases, where the
workload spans from one production OLTP RAC database to OLTP RAC database, one
each SNAP development OLTP and OLAP standalone databases, and two OLTP
standalone virtualized databases running in parallel:
Simplified and intuitive volume provisioning on the XtremIO X2 array with:
Over 450,000 Transactions per Minute (TPM)
Over 130,000 IOPS with sub-millisecond latencies
Because of the processing power of the PowerEdge servers, CPU utilization was
minimal, leaving room for more databases or for failover of VMs from one ESXi host
to another.
The XtremIO X2 array with inline deduplication and compression saved 1.8X the
flash space, using only 629 GB of capacity for 1.2 GB of logical usable data.
Chapter 7: Conclusion
93
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
A snapshot database was created by using just a few clicks with minimal storage
addition and no performance impact. Greater space savings of 1.9X were achieved
through the use of the inline deduplication and compression features of the XtremIO
X2 array.
A full backup of a 1 TB Oracle database took 36 minutes to complete by using a
Data Domain 6300 system. The database size was reduced to 756.7 GB (total
compression size) in the Data Domain system with 28 percent compression (a
compression factor of 1.4X) and storage throughput of 482 MB/sec.
The full recovery of a 1 TB Oracle database took 32 minutes to complete by using
the Data Domain 6300 system with storage throughput of 571 MB/sec.
Summary
This solution is an integrated, validated, and tested database solution. Guesswork,
complexity, and risk are exchanged for faster time-to-value, ease of management and
support, and an engineered system that is specifically designed for Oracle databases. The
solution’s PowerEdge R740 and R940 servers support large database workloads and still
have more than 75 percent unused capacity.
Solution test results show that the XtremIO X2 storage array delivers fast response times,
with latencies under 0.85 milliseconds and throughput to satisfy demanding OLAP and
OLTP databases. Repurposing copies of production to development by using the XtremIO
XVC inline deduplication and compression features delivered 1.8X flash space savings
with the overall efficiency of 5.6. In addition, features such as replication, which are not
discussed in this guide, can provide protection from all types of disasters for Oracle
databases.
Automation is the key to reducing the time devoted to routine database provisioning tasks.
With AppSync software, you can automate the repurposing and protection of databases.
You can repurpose databases on-demand or on a schedule. Either way, the time that is
saved by automating the work can then be invested in more valuable activities.
For data protection of the RAC databases running on XtremIO X2 storage, this solution
attains goals in terms of CPU utilization, database backup/recovery time, and network
throughput. The solution uses inline deduplication and compression to accelerate backup
and recovery activity while reducing bandwidth utilization and backup/recovery time, and
increasing storage/network throughput. The DD Boost software prevents duplicating
backups of similar data, thus reducing the load on the database, storage, and backup
host. The DD Boost software also reduces the frequency of full backups, improves RPO
and RTO, and reduces load on the data center infrastructure.
Dell EMC has an Oracle team that is devoted to customers who are interested in Ready
Solutions for Oracle. Many of these database experts have been working with Oracle for
more than 10 years and understand all the dependencies to ensure your success. Dell
EMC’s Oracle Specialists can size and configure Ready Solutions for Oracle to meet the
needs of your business.
Chapter 8: References
94
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Chapter 8 References
This chapter presents the following topics:
Dell EMC documentation .................................................................................. 94
VMware documentation .................................................................................... 94
Oracle documentation ...................................................................................... 94
HammerDB documentation .............................................................................. 94
Chapter 8: References
95
Dell EMC Reay Solution for Oracle with Data Protection Enterprise-Class Protection and Provisioning using Dell EMC XtremIO X2 and Data Domain Storage
Validation Guide
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Dell EMC documentation
The following documentation on DellEMC.com or Dell EMC Online Support provides
additional and relevant information. Access to these documents depends on your login
credentials. If you do not have access to a document, contact your Dell EMC
representative.
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain
Deployment Guide
Introduction to Dell EMC XtremIO X2 Storage Array
Best Practices for Running Oracle on Dell EMC XtremIO X2
Dell EMC Data Domain Deduplication Storage Systems Spec Sheet
XtremIO Snapshot Refresh for Oracle Database Production, Development and
TEST
VMware documentation
The following documentation on the VMware website provides additional and relevant
information:
VMware ESXi 6.5 Installation and Setup
VMware vSphere 6.5 Installation and Setup
Oracle Databases on VMware Best Practices Guide
Oracle documentation
The following documentation on the Oracle website provides additional and relevant
information:
Oracle Database 12c Release 2 Installation Guide
Oracle Real Application Clusters 12c Release 2 Installation Guide
Oracle Grid Infrastructure Installation and Upgrade Guide
HammerDB documentation
The following documentation on the Hammer website provides additional and relevant
information
Installing and Starting HammerDB on Linux
Configuring Schema Build Options
How to Run an OLTP Workload
Appendix A: Configuration Details
96
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Appendix A Configuration Details
This appendix presents the following topics:
Database performance data collection ............................................................ 96
Database parameters ........................................................................................ 98
HammerDB configuration parameters ............................................................. 98
OLAP query customization ............................................................................ 100
Appendix A: Configuration Details
97
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Database performance data collection
We collected the following database performance data through the Oracle database AWR
report.
Based on the 20-minute AWR reports of a test case, the IOPS is the sum of physical
read total I/O requests, per Second, and physical write total I/O requests, per
Second, as shown in the following figure.
Figure 37. Sample of IOPS measurement from AWR report
For an Oracle OLTP-style I/O workload, db file sequential read, the User I/O class wait
is always the top wait event, accounting for most of the wait time. In this example, wait
time averaged 0.443 milliseconds, as shown in the preceding figure.
Note: The db file sequential read events account for single block random I/O calls to the
operating system.
In addition to db file sequential read wait event, transaction redo logging write is another
key performance indicator for Oracle OLTP-style I/O workloads. The following figure
shows the top five timed events section of the AWR report from one of the OLTP
production databases while the workload ran. In this example, wait time averaged 0.244
milliseconds.
IOPS
I/O latency
Appendix A: Configuration Details
98
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Figure 38. Example I/O Latency measurement from AWR report
For an Oracle OLAP-style I/O workload, the I/O MB/s throughput can be calculated as the
physical read total I/O bytes per second, as shown in the following figure. In this example,
the I/O throughput is 591,947,981.72 bytes per second, or 564.52 MB/s.
Figure 39. Sample of I/O throughput in MB/s from AWR report
The CPU utilization of the database nodes is shown in the OS Statistics By Instance
field of the AWR report, as shown in the following figure.
Figure 40. Example CPU Utilization measurement from AWR report
I/O MB/s
throughput
CPU utilization
Appendix A: Configuration Details
99
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
Database parameters
To perform an ideal performance comparison, we used the following database
configuration on different types of databases. We used these database parameter settings
to configure production, XVC, and virtualized databases before running the test
workloads.
Table 32. Database parameter settings for OLTP PROD, DEV XVC, OLAP XVC, and virtual databases
Parameter Production DEV XVC OLAP XVC Virtualized (11gR2 and 12cR2)
Database block size 8KB 8KB 8KB 8KB
sga_target 16GB 16GB 16GB 64GB
sga_max_size 32GB 32GB 32GB 64GB
pga_aggregate_target 200GB 200GB 200GB 16GB
open_cursors 1000 1000 1000 1000
filesystemio_options setall setall setall setall
use_large_pages TRUE TRUE TRUE TRUE
resource_manager_plan null null null null
db_file_multiblock_read_count 4 4 16 4
HammerDB configuration parameters
During each test, we kicked off as many HammerDB UI instances as the number of OLTP
databases that were running during that test. During test 1 which involved only the OLTP
database (production), we ran only one HammerDB workload generator UI. During test 3,
when we tested one OLAP database (OLAP XVC) and two OLTP databases (one OLTP
production and one OLTP DEV XVC) in parallel, we ran two separate instances of the
HammerDB workload generator UI to generate load on the two OLTP databases in
parallel.
Note: The OLAP workload on the OLAP XVC database was generated by using a custom OLAP
query that is run directly on the XVC database server. HammerDB was not used to generate the
OLAP workload.
The number of concurrent HammerDB virtual users for each test is shown in the following
table:
Appendix A: Configuration Details
100
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection Validation Guide
Table 33. HammerDB Virtual User settings for each OLTP database during each test case
Test case #
OLTP workload OLAP workload
Databases running (names)
Number of HammerDB Virtual Users
Database running (name)
Number of OLAP users
1 db2tp 70 None -
2 db2tp 70 snpolap 1
3 db2tp, snpdev 70, 10 snpolap 1
4 db2tp, snpdev, v11gdb, v12cdb
70, 10, 10, 10 snpolap 1
During each test case, all other HammerDB driver script and Virtual User options were set
as shown in the following table:
Table 34. Common HammerDB parameter settings for all OLTP databases
Parameter Value
Oracle TPC-C Driver Script Options
Total Transactions per User 100,000
TPC-C Driver Script Timed Driver Script (checked)
Exit on Oracle Error TRUE (checked)
Keying and Thinking Time FALSE (unchecked)
Checkpoint when complete TRUE (checked)
Minutes of Rampup Time 5
Minutes for Test Duration 15
Use All Warehouses TRUE (checked)
Time Profile TRUE (checked)
Virtual User Options
User Delay(ms) 500
Repeat Delay(ms) 500
Iterations 1
Show Output TRUE (checked)
Log Output to Temp TRUE (checked)
Use Unique Log Name TRUE (checked)
No Log Buffer TRUE (checked)
Log Timestamps TRUE (checked)
Appendix A: Configuration Details
101
Dell EMC Ready Solutions for Oracle with Dell EMC XtremIO X2 and Data Domain Enterprise-Class Storage Provisioning and Data Protection
Validation Guide
OLAP query customization
To generate an OLAP workload, we used the following query that was run directly on the
XVC database server:
select max(H_AMOUNT),min(H_AMOUNT) from HISTORY where H_DATE >=
'01-JUN-81'
Table 35. Configuration parameters for the OLAP workload
Parameter Value
User Count 1
Run Time 20 min
Note: The custom query did not take 20 minutess to finish. The query was run in a loop for 20
minutes.