Design and Architecture of Dell Acceleration Appliances for Database (DAAD): A Practical Approach with High Availability Guaranteed

Kai Yu , Yuxiang Gao

Dell Global Solutions Engineering Group

Peng Zhang, Stony Brook University

Meikang Qiu, Pace University

Agenda

• Database as a Service (DBaaS) Cloud Model

• Existing Solutions

• High Performance HA Storage Solutions

• Dell Accelerations Appliances for Database (DAAD)

• Experimental Evaluation: Performance Benchmarks

• Conclusions and Next Version of DAAD

About Authors • Kai Yu, Senior Principal Engineer, Dell Enterprise Solutions Engineering

20 years Database and Infrastructure Solutions Engineering, Oracle ACE Director.

Over 140 conference presentations and whitepapers, 2 US Patents 2012 Oracle Excellence Award: Technologist of the Year: Cloud Architect by

Oracle Magazine, OAUG Innovator of Year; Co-author Apress Book “Expert Oracle RAC 12c” My Oracle Blog: http://kyuoracleblog.wordpress.com/

• Yuxiang Gao, Senior HPC Consultant, Dell Enterprise Solutions Engineering (until 2014)

PhD degree in Applied Mathematics from Stony Brook University Focuses on accelerator-based cluster design and performance

analysis

About Authors • Peng Zhang, senior research associate , Stony Brook University

• PhD degree in Applied Mathematics from Stony Brook University • Working on the multidisciplinary programs for simulating complex

biological systems on modern high performance computing (HPC) • Focuses on optimizing large-scale parallel applications on modern

HPC systems, for offering computationally affordable solutions.

• Meikang Qiu, Associate Professor of Computer Science at Pace University • PhD degree of computer science from University of Texas at Dallas • ACM Senior member, Senior Member of the IEEE. • Research interests include cyber security, embedded systems,

cloud computing, smart grid, microprocessor, data analytics • Published 3 books, 170þ peer-reviewed journal and conference

papers (including 70 journal articles, 100 conference papers), and 3 patents

• Won Naval Summer Faculty Award in 2012 and Air Force Summer Faculty Award in 2009; ACM Transactions on Design Automation of Electrical Systems (TODAES) 2011 Best Paper Award; won another 4 Conference Best Paper Awards (IEEE/ACM ICESS’12, IEEE GreenCom’10, IEEE EUC’10, IEEE CSE’09)

Database as a Service Cloud Model • Database as a Service cloud model

– Consolidate multiples databases : multitenant architecture – Provide Database as a Service (DBaaS)/ Platform as a Service (PaaS) – Great benefits: – Reduce cost by maximizing the utilization of system resources – Allow to provisioning on demand: dynamically provisioning to meet the

workload needs

Infrastructure Challenges on DBaaS

– High Availability Requirement • Great business impact caused by infrastructure downtime • Different service level(SLA) for different business

– Highly scalable architecture to meet the performance requirements • Handle the workloads from multiple databases • Bottlenecks from processor, memory, network, storage

Storage frequently presents critical database performance bottlenecks

– Processor, memory and network performance improved significantly – Storage performance bottlenecks • Database workloads usually requires very high IO requirements throughputs( IOPs, MB/s), IO latency. • Hard disk performance limited by mechanical speed. • As results, Processor, memory and network resources underutilized • Software license also underutilized (usually linked to # of processors)

Database as a Service Cloud Model

Oracle Real Application Clusters: active-active cluster database

• Based on share everything clustering architecture • Protect database availability against up to N-1 server failure • Reduce planned downtime for hardware, OS, software upgrade • Add node or remove node based on demand of capacity • Application load balancing • Provide high availability and scalability architecture • Require shared storage to store database files. • Require high speed interconnect network across database nodes

Existing Solutions

User connectionsUser connections

RAC Instance1

RAC Database

RAC Instance2

RAC Instance3

Cluster Interconnect

Node1 Node2 Node3

HA Storage Solutions

– Redundancy Array of Inexpensive Disks (RAID) to ensure storage HA – Network-Attached Storage (NAS) and Storage Area Network (SAN)

• Multiple IO paths • Redundant Network Switches •

• Redundant storage Controllers

• R Re Redundant Backend cables D Disk Array RAID configuration • Perform Performance bottlenecks

Existing Solutions

Flash based SSD for low latency and high IOPs

–Store data in flash base PCIe SSDs that are plugged into PCI backplane Dell Express Flash PCIe SSD SanDisk ioMemory Card Non Volatile Memory Express (NVM Express) – Dell Acceleration Appliances for Database (DAAD) based on ioMemory

High Performance HA Storage Solutions

High Availability enabled DAAD

– A HA cluster with Two DAAD nodes: 40 Gb point-to-point network between two DAAD nodes

Volume configuration for storage high availability

Dell Acceleration Appliances for Database (DAAD)

High Availability Database Solution Architecture

– Architecture of Oracle Database on DAAD

IO Multipath to the storage volumes

Dell Acceleration Appliances for Database (DAAD)

Iometer to measure the pure IO performance

Calibrate_IO to simulate Oracle Database IO: Compare with baseline MB/s scalability IOPs scalability

Experimental Evaluation: Performance Benchmarks Test Name DAAD vs 96-disk Baseline

IOPS MB/s speedup

4k Random Read 1,010,13/ 36,771

3,946 / 144 27x

8K Random 70/30 R/W

346,120/ 37,723

2,704 / 295 9.2x

1MB Seq Read 7,453/ 2,907

7,453 / 2,907 2.6x

1MB Seq Write 4,132 / 940 4,132 / 940 4.4x

Test Name DAAD vs 96-disk Baseline

Latency(ms) Reduction 4KB Random Read

0.51 / 13.92 96%

8KB Random 70/30 R/W

1.48 / 13.57 89%

TPCC-Like benchmark by HammerDB Simulating application workloads, using 10ms response time limit Measuring :Transaction Per Minute (TPM): New Order Per Minute (NOPM) Average Response time (ART); Scalability by # of DB nodes

1 2 3 4

TPM 970,107 1,714,048 2,257,219 2,542,395

NOPM 326,839 582,967 796,161 959,274

ART 5.97 6.75 7.52 7.98

0.00

10.00

20.00

30.00

40.00

50.00

60.00

0

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

Re

spo

nse

Tim

e (

ms)

Tra

nsa

cti

on

s p

er

Min

ute

Number of Database Nodes

Performance Scaling for DAAD

Experimental Evaluation: Performance Benchmarks

Dell Acceleration Appliances for Database (DAAD)

– High Performance storage solutions for Oracle Database – Highly Available Architecture: Storage, Network and Database Nodes.

Next Version of DAAD – Introducing InfiniBand Network for Storage and DB node Interconnenct – Highly Available Architecture: Storage, Network and Database Nodes

Conclusion and Next Version of DAAD