Enabling Local and Remote High Performance Backups Shyamsundar R Software Engineer, SMS Novell, Inc....

transcript

Enabling Local and Remote High Performance Backups

Shyamsundar RSoftware Engineer, SMSNovell, Inc.

Vijai Babu MadhavanSoftware Engineer, SMSNovell, Inc

one Net: Information without boundaries…where the right people are connected with the right information at the right time to make the right decisions.

The one Net vision

Novell exteNd™

Novell Nsure™

Novell Nterprise™

Novell NgageSM

The one Net vision

Novell Nterprise is an innovative family of products which gives you the power to enable and manage the constant interaction of people with your business systems — regardless of who they are or where they are.

Novell Nterprise™

Novell exteNd™

Novell Nsure™

Novell Nterprise™

Novell NgageSM

Agenda

Technology overview

SMS technology advancements

Troubleshooting backup on NetWare

Making the most out of your storage

Related technology

Backups

Backup is a second class citizen…

Backup data flow

Backup Engine

File Systems

Local backup data flow

Backup data flow

Backup Engine

File Systems*

Backup data flow

Backup Engine

File Systems

Backup data flow

Backup Engine

File Systems

Backup data flow

Backup Engine

File Systems

Backup data flow

SMDRBackup Engine SMDR

Network

Backup Engine

File Systems

Remote backup data flow

File Systems

Backup data flow

Network

*Remote backup data flow

File Systems

Backup Engine

File Systems

Backup data flow

Network

File Systems

Backup Engine

File Systems

Backup data flow

Network

File Systems

Backup Engine

File Systems

Backup data flow

Network

File Systems

Backup Engine

File Systems

Local backup data flow: Systemic view

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory*

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

CPU-memory bus

Bridge

Storage (SCSI) Bus

I/O (PCI) bus

Memory

Remote backup data flow: Systemic view

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory*

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

*Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

*Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

Network

Network Controller

Storage (SCSI) Bus

CPU-memory bus

Bridge

I/O (PCI) bus

Memory

*Network

Network Controller

Storage (SCSI) Bus

Available network bandwidth

Ethernet wire capacity in theory• 100 Mbps = 94.44 Mb/s ~= 676 MB/min

– Accommodating space loss due to protocol headers

– Accommodating time loss due to inter-frame gap• 1000 Mbps ~= 6755 MB/min

Practical dedicated Ethernet wire capacity• 100 Mbps – ~656 MB/min – 97%• 1000 Mbps – ~6417 MB/min – 95%

“Ideal bulk transfer applications should be bandwidth bound”

Available network bandwidth

Ethernet wire capacity in theory• 100 Mbps = 94.44 Mb/s ~= 676 MB/min

– Accommodating space loss due to protocol headers

– Accommodating time loss due to interframegap• 1000 Mbps ~= 6755 MB/min

Practical dedicated Ethernet wire capacity• 100 Mbps – ~656 MB/min – 97%• 1000 Mbps – ~6417 MB/min – 95%

“Ideal bulk transfer applications should be bandwidth bound”

Agenda

Technology overview

Related technology

How fast can remote backups go?

We can do 1TB in 3.5 Hours on the wire at 4.8GB/Min, with the streaming SMDR prototype

Streaming SMDR prototype results

“Remote matching local performance...”

Setup#1 Setup#2 Setup#3

470 1080 2050

510 1100 2100 Streaming PrototypeLocal

05/24/1901 01/04/1903 09/30/190504/14/1901 12/15/1902 08/11/1905

8.5 1.9 2.4

Streaming PrototypeLocal% Difference

Setup #1: 2GB SYS volume with an average file size of 27KB

Setup #3: 6GB volume with 6000 files each being 1MB in size

470 1080 2050

05/24/1901 01/04/1903 09/30/190504/14/1901 12/15/1902 08/11/1905

8.5 1.9 2.4

470 1080 2050

05/24/1901 01/04/1903 09/30/190504/14/1901 12/15/1902 08/11/1905

8.5 1.9 2.4

Improvements over existing SMDR

min Existing SMDR

Prototype 1

Prototype 2

Existing SMDR 212 305 560

Prototype 1 216 440 1550

Prototype 2 470 1080 2050

Setup 1 Setup 2 Setup 3

“Improved remote restore performance”

min Existing SMDR

Prototype 1

Prototype 2

Existing SMDR 600

Prototype 1 1800

Prototype 2 2600

How Streaming SMDR works

Alleviates latency effects on throughput by anticipation

Keeps the network pipe full

Better network bandwidth utilization

Read ahead, lazy write

Decouples the interface from the implementation

Who gets the benefit

Applications that use SMDR for remote backups

Data movement applications• Novell migration wizard• Novell server consolidation utility• Novell DFS volume move/volume split• Commercial copy applications

SMDR: Features

Existing Features• Provides location transparency• Abstracts communication details, supports both

TCP/IP and IPX/SPX• Advertisement/Name resolution• Synchronous RPC model

Recent Features• Multiple IP address support• DNS support for name resolution• Policy based discovery/name resolution mechanism

order• iManager and server console based configuration

SMDR: Features

Existing Features• Provides location transparency• Abstracts communication details, supports both

TCP/IP and IPX/SPX• Advertisement/Name resolution• Synchronous RPC model

Recent Features• Multiple IP address support• DNS support for name resolution• Policy based discovery/name resolution mechanism

order• iManager and server console based configuration

TSAFS: Features

Scalability• Serial nature of backup does not limit scalability• Re-architected TSA to scale with disk configurations

Manageability• Improved manageability, helps identify performance sweet

spots• iManager and server console based configuration

Availability• Cluster enabled, supports continuation of backup after a

fail-over/fail-back• Not just with NCS, even with server restarts

Ships with NetWare® 6.5, NetWare® 6.0 SP4

TSAFS: Features

Sample TSAFS test results

8000Th

TSA600 1380 1400

TSAFS 3300 6120

Tweaked 5880 6120

Setup #1 Setup #2

Setup #1: Ultra 160 controller with 4 disks using RAID 5, with PCI-X 64bit 100 MHzSetup #2: Ultra 160 controller with 6 disks using RAID 0, with PCI-X 64bit 100 MHz

TSAFS improvements and tweaking effects

Sample TSAFS test results

TSAFS improvements and tweaking effects

Setup #1: Ultra 160 controller with 4 disks using RAID 5, with PCI-X 64bit 100 MHzSetup #2: Ultra 160 controller with 6 disks using RAID 0, with PCI-X 64bit 100 MHz

8000Th

TSA600 1380 1400

TSAFS 3300 6120

Tweaked 5880 6120

Setup 1 Setup 2

Agenda

Technology overview

Related technology

General Guidelines

Rules of Thumb• Eliminate one component at a time• Try and eliminate broader sub systems first

– For e.g., target the disk or network first

Use tools• DPMETERM• TSATEST• NetWare Remote Manager (NRM)• iManager and Server Console

General Guidelines

Rules of Thumb• Eliminate one component at a time• Try and eliminate broader sub systems first

– For e.g., target the disk or network first

Use tools• DPMETERM• TSATEST• NetWare Remote Manager (NRM)• iManager and Server Console

Troubleshooting demo

Establish a baseline• TSATEST• Backup application

Can it be improved?• DPMETERM

Can we go beyond this?• Using NRM• Using TSATEST• Tweaking TSAFS, ReadThreadsPerJob

and ReadBufferSize

Troubleshooting SMDR issues

Use tools for performance issues• PERFTEST• NETMON• Other packet capture/analysis utilities

Discovery/Name Resolution issues• SLP• SMDR NEW• SMDR CONN

“Ensure simultaneous remote backups do not flood your network”

Troubleshooting TSAFS issues

TSAFS debug log• TSAFS /SmsDebug=<Value> /SmsDebug2=<Value>

Server memory usage• TSAFS /CacheMemoryThreshold=<Percentage>

Other factors• Competing software (Compression/Anti-Virus)• CPU usage/Busy threads, NRM I/O statistics

Agenda

Technology overview

Related technology

Power of aggregation

CPU-memory bus

PCI bus

D D D D

D D D D D

SCSI/RAID Controller

SCSI Controller

PCI bus...

PCI busPCI busPCI bus ......

Fibre Channel HBA

Suggestions on exploiting bandwidth

Create parallel I/O paths for high performance• Load balance peer-to-peer PCI buses

– Distribute your storage and network controllers

• Use multiple channels per HBA and stripe across them

Use technology to suit performance needs• 1Gbps networks are becoming common, 4Gbps cards are out

in the market, 10Gbps is close to reality

Mix and match with care• Do not use Ultra 320 disks with an Ultra 160 SCSI controller• Performance is limited by the lowest bandwidth component in

your data pipe

Design your storage configuration for high availability as

your data pipe

Agenda

Technology overview

Related technology

Related Technology

File Copy On Write• Permits open file backup

Pool Snapshots• Point in time data snapshots

Freeze/Thaw• Permits data consistency across compliant

applications

Versioning• User managed versioning of files, easing restores

Related Technology

applications

Related Technology

applications

Related Technology

applications

SMS on Linux

Location Transparency

Enterprise scale, logical backup

Multiple file system support

Rich meta-data support

SMS: In Summary

One-stop Network wide Storage Management Solution

• High Performance, Scalable

• Extensible solution

• Multi-Latency/Bandwidth

environments

• True backup virtualization

Backups

Backup is a second class citizen…

Or is it?

General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. Novell, Inc., makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Novell, Inc., reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.

No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.

Enabling Local and Remote High Performance Backups Shyamsundar R Software Engineer, SMS Novell, Inc....

Documents