Date post: | 03-Dec-2014 |
Category: |
Documents |
Upload: | david-deng |
View: | 110 times |
Download: | 0 times |
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation
DS8000 Technical Overview
Paul SpagnoloIBM Advanced Technical SkillsApril 9, 2023
R6.2
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation2
Agenda
Design Principles
DS8000 Highlights
Storage Architecture and Components
Caching Algorithms
Storage Virtualization
DS8000 Family Comparison
Functions
– Easy Tier
– I/O Priority Manager
– Thin Provisioning
– Resource Groups/Multi-Tenancy
– System z Synergy Functions
– Power Systems Synergy Functions
– IBM i and DS8000 Synergy
Encryption
Copy Services
– FlashCopy
– Metro Mirror
– Global Copy
– Global Mirror
– Metro/Global Mirror
– z/OS Global Mirror (XRC)
DS8000 Performance
Logical Configuration
Click on to return to this agenda slideReturn
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation3
Design Principles
Deliver high-end block-access disk storage systems to satisfy customer requirements
Examples:
– Maintain access to data• Goal: > five 9s availability• Redundant hardware, extensive error recovery, release-to-release code reuse• Nondisruptive changes: repairs, hardware upgrades, microcode upgrades, logical
configuration– Deliver high performance
• High random IOPS, low random I/O response time, high sequential throughput• Support efficiently running a mix of workloads concurrently (e.g., random and sequential)
– Support flexible customization• Separately scale cache, host ports, disk drives• Scale from the smallest to the largest configuration nondisruptively
– Support extensive functionality• Internal copy, 2-3 site remote mirroring, thin provisioning, sub-volume tiering, wide-striping,
Quality of Service• Extensive synergy with IBM i, p and z servers
– Provide ease-of-management• Friendly GUI, easy volume management• Extensive self-tuning reduces manual tuning time and effort Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation4
DS8000 Highlights
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation5
4th-generation DS8000 enterprise disk system
Binary Compatibility
2004 2009
POWER5 POWER6
DS8800 builds on a market-proven, reliable code base!
2010
POWER6+
DS8800 DS8800DS8100/DS8300
DS8100/DS8300 DS8700 DS8700
2006
POWER5+
DS8100/DS8300
Turbo
DS8100/DS8300
Turbo
The IBM POWER processor has been behind the success of IBM enterprise storage beginning with the Enterprise Storage Server in 1999
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation6
Current DS8000 Family Models
DS8700• POWER6 controllers (2-way and 4-way)• 4 Gb/s and 8 Gb/s host adapters• 2 Gb/s device adapters• 3.5” Enterprise Fibre Channel, SSD and SATA drives
DS8700• POWER6 controllers (2-way and 4-way)• 4 Gb/s and 8 Gb/s host adapters• 2 Gb/s device adapters• 3.5” Enterprise Fibre Channel, SSD and SATA drives
DS8800• POWER6+ controllers (2-way and 4-way)• 8 Gb/s host adapters• 8 Gb/s device adapters• 2.5” Enterprise SAS-2 and SSD drives ,3.5” Nearline drives
DS8800• POWER6+ controllers (2-way and 4-way)• 8 Gb/s host adapters• 8 Gb/s device adapters• 2.5” Enterprise SAS-2 and SSD drives ,3.5” Nearline drives
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation77
DS8000 – Hardware Structure
RAIDAdapters
RAIDAdapters
RAIDAdapters
HostAdapters
HostAdapters
HostAdapters
HostAdapters
HostAdapters
HostAdapters
HostAdapters
PersistentMemory
Volatile Memory
N-waySMP
N-waySMP
PersistentMemory
Volatile Memory
Switched Fabric
Higher Bandwidth Fault Tolerant Fabric
RAIDAdapters
RAIDAdapters
RAIDAdapters
Three-Tier Cooperative Processing Architecture
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation8
DS8700 Hardware
Model 941 base and model 94E expansion– 941 with up to 4 x 94E expansion frames
• Enterprise fibre channel, SSD and SATA drive options• UP to five frames• UP to 1,024 drives
Power 6 (P6 4.7Ghz) processors– 2-way and 4-way options
Host adapters– Up to 128 ports– Each port supports FCP, FC-AL and FICON at the port level– Base frame and first expansion frame allows 16 adapters per
frame– Both 4 Gb/s and 8Gb/s Host Adapters available
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation9
DS8800 Hardware
Model 951 base and model 95E expansion– 951 with up to 3 x 95E expansion frames
• 2.5”, small-form-factor drives• 6 Gb/s SAS (SAS-2)
– Enclosures support 50% more drives • Maximum of four frames• Maximum of 1,536 drives
– Top of rack exit option for power and cabling
Power 6+ (P6+ 5Ghz) processors– 2-way and 4-way options
Host adapters– Up to 128 8 Gb/second ports– Each port supports FCP, FC-AL and FICON at the port level– Base frame and first expansion frame allows 16 adapters per
frame– Both 4 and 8 port host adapter cards available
Efficient front-to-back cooling (cold aisle/hot aisle)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation10
DS8000 Business Class OfferingSmallest hardware configuration possible
DS8800 Business Class cabling option
Single frame only (not upgradeable to multi-frame)– 2 way CEC
– Cache size: 16GB – 64 GB
– 2 I/O enclosures
Drive Options – 146GB/15k, 300GB/15k +FDE encryption options
– 450GB/10k, 600/10k, 900GB/10k + FDE encryption options
– 300 GB SSD
– 3 TB/7.2k nearline
No. of HDD: 16 ~ 240 (increment of 16)
Single or Three phase power
Number of Adapters– Device: 2 Pairs only w/full 240 drive support for best scalability and cost/TB
– Host Adapters: 2 ~ 4 (32 ports – FCP/FICON)
Logical configuration:– Intermix of CKD or Open
– Recommend extent rotation to balance performanceDS8800 BC
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation11
Disk enclosure comparison
Disk Technology– 3.5” (LFF) Fibre Channel
Throughput– 2Gbps FC interconnect backbone
– 2Gbps FC to disks
Density– Supports 16 disks per enclosure
– 3.5U of vertical rack space
Cabling– Passive copper interconnect
Modularity– Rack level power
– Rack level cooling
DS8700 Enclosure DS8800 Enclosures
Disk Technology– 2.5” (SFF) SAS
Throughput– 8Gbps FC interconnect backbone
– 6Gbps SAS-2 to disks
Density– Supports 12/24 disks per enclosure
– 2U of vertical rack space
Cabling– Optical short wave multimode interconnect
Modularity– Integrated power
– Integrated coolingThe DS8800 uses the 3.5” enclosure for the larger sized 3 TB nearline drives
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation12
Storage Architecture and Components
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation13
DS8700 Base Frame (Model 941)
Two processor complexes based on System p Power 6 4.7 GHz– 2-way or 4-way (total of 4 or 8 processors)
– 2-way up to 128 GB of processor memory
– 4-way up to 384 GB of processor memory
Supports up to 128 DDMs in base frame
4 Gb and 8 Gb host adapters– FCP for open systems servers
– FICON for System z servers
– Each port on card can be independently set to support FCP, FC-AL or FICON
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation14
DS8700 Expansion Frames (Model 94E)
Zero to 4 optional expansion frames (installed to the right of the base frame– First expansion frame may contain additional host adapters
– Each frame has its own power supplies and power line cords (2 for redundancy)
– Concurrent upgrade to add expansion frames and disk capacity
– All additional frames can be concurrently added
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation15
DS8700 Expansion Frame Options
Base Model
Expansion Model
Processor Type
Maximum DDMs
Maximum Processor Memory
Maximum Host
Adapters*
941 None 2-way 4.7 GHz
128 128 GB 16
941
None
4-way 4.7 GHz
128
384 GB
16
1 x 94E 384 32
2 x 94E 640
3 x 94E 896
4 x 94E 1024
* The maximum number of 8 Gb host adapters is 8 in the base frame and 8 in the expansion frame for a total of 16
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation16
DS8800 Base Frame (Model 951)
Two processor complexes based on System p Power 6+ 5.0 GHz– 2-way or 4-way (total of 4 or 8 processors)
– 2-way up to 128 GB of processor memory
– 4-way up to 384 GB of processor memory
Supports up to 240 DDMs in base frame
8 Gb host adapters– FCP for open systems servers
– FICON for System z servers
– Each port on card can be independently set to support FCP, FC-AL or FICON
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation17
DS8800 Expansion Frames (Model 95E)
Zero to 3 optional expansion frames (installed to the right of the base frame– First expansion frame may contain additional host adapters
– Each frame has its own power supplies and power line cords (2 for redundancy)
– Concurrent upgrade to add expansion frames and disk capacity
– All additional frames can be concurrently added
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation18
DS8800 Expansion Frame Options
Base Model
Cabling Class
Expansion Model
Processor Type
Maximum DDMs
Maximum Processor Memory
Maximum Host
Adapters
951 Business None 2-way 5.0 GHz
240 64 4
951 Standard None 2-way 5.0 GHz
144 128 4
951 Standard None 4-way 5.0 GHz
240 384 8
951 Standard 1 x 95E 4-way 5.0 GHz
576 384 16
951 Standard 2 x 95E 4-way 5.0 GHz
1056 384 16
951 Standard 3 x 95E 4-way 5.0 GHz
1,536 384 16
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation19
Power Processors
Based on Power6 server technology
– DS8700 uses Power 6 at 4.7 GHz
– DS8800 uses Power 6+ at 5.0 GHz
Two System p servers installed in the base frame
– 2-way or 4-way option yield 4 or 8 processors per machine
– I/O connectivity is via PCI Express
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation20
Processor Memory Options
Both the DS8700 and DS8800 offer up to 384 GB of total processor memory– Each processor has half of the total system memory
– Memory functions as cache for reads and writes
– Writes are written to both servers, with one stored in non-volatile memory and the other stored in cache
– Concurrent upgrade to all processor memory sizes• Options for processor memory
– 32 GB memory / 1 GB NVS– 64 GB memory / 2 GB NVS– 128 GB memory / 4 GB NVS– 256 GB memory / 8 GB NVS– 384 GB memory / 12 GB NVS– Business Class system supports from 16 GB – 64 GB / 1 GB – 2 GB NVS
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation21
Host Adapters
Host adapters connect via FICON or Fibre Channel to attached servers– Host adapters also are used for replication technologies
– Up to 128 ports
– Host adapters can be a mix of long wave and short wave (all ports on card are same type)
– Any port can be configured independently to support either FICON or FCP
Concurrent upgrade/MES to add additional host adapters
DS8700 host adapter cards are all 4-port (either LW or SW)– 4 Gb/second (maximum of 32 cards or 128 4 Gb/second ports)
– 8 Gb/second (maximum of 16 cards or 64 8 Gb/second ports)
DS8800 host adapter cards can be 4-port or 8-port (either LW or SW)– 8 Gb/second (maximum of 16 cards – 128 ports using 16 8-port cards)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation22
Disk Drive Options
DS8700
– 600 GB SSD
– 300 GB/15,000 RPM FC + FDE option
– 450 GB/15,000 RPM FC + FDE option
– 600 GB/15,000 RPM FC
– 2 TB/7,200 RPM SATA
DS8800
– All SAS-2
– 300 GB SSD
– 146 GB/15,000 RPM + FDE option
– 300 GB/15,000 RPM + FDE option
– 450 GB/10,000 RPM + FDE option
– 600 GB/10,000 RPM + FDE option
– 900 GB/10,000 RPM + FDE option
– 3 TB/7,200 RPM
Maximum of 1,536 drives
Drives install in groups of 16– SSD drives available in groups of 8 or
16
– Nearline drives available in groups of 8
Maximum of 1,024 drives
Drives install in groups of 16– SSD drives available in groups of 8 or 16
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation23
Disk Drive Installation Location
Model Type Base Frame
Drive Sets(DDMs)
Expansion 1
Drive Sets(DDMs)
Expansion 2
Drive Sets(DDMs)
Expansion 3
Drive Sets(DDMs)
Expansion 4
Drive Sets(DDMs)
Total Possible Drives
DS8700 8 (128) 16 (256) 16 (256) 16 (256) 8 (128) 64 (1024)
DS8800 15 (240) 21 (336) 30 (480) 30 (480) N/A 96 (1056)
Each full drive set consists of 16 DDMs. With the exception of Solid State Drives and nearline options, drives are always ordered in full sets of 16. SSD drives may be ordered in a single group of 8 if desired. Nearline drives on the DS8800 are available in groups of 8
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation24
DS8800 Drive Summary
Drive Set Standby CoD Drive
Set
Encryption Drive Set
Encryption Standby
CoD Drive Set
300 GB SSD eight drive set
300 GB SSD sixteen drive set
146 GB/15K RPM
300 GB/15K RPM
450 GB/10K RPM
600 GB/10K RPM
900 GB/10K RPM
3 TB/7200 RPM eight drive set
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation2525
DS8700 Drive Summary
Drive Set Standby CoD Drive
Set
Encryption Drive Set
Encryption Standby
CoD Drive Set
600 GB SSD eight drive set
600 GB SSD sixteen drive set
300 GB/15K RPM
450 GB/15K RPM
600 GB/15K RPM
2 TB/7200 RPM
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation26
Why Solid State Drives
Tier 0 storage class for high priority, time-sensitive applications
Market drivers– Pervasive computing
– Green IT & data center power consumption
– Spinning drives have not kept up with processor performance improvements
SSD Drive benefits– More transactions in less time
– Addresses cache unfriendly workloads – yielding better response times
– Reduced energy utilization
– Improve availability – lower component failures
– Enabling new functions
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation27
SSD Latency with DS8700/DS8800
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation28
Device Adapters Device adapters connect the I/O
enclosures in the processors to the disk drives
– Device adapters perform all RAID functions and rebuilds in the event of a drive failure
– Device Adapters are configured in active/active pairs that provide redundant access to drives
– RAID levels supported by these device adapters include RAID-5, RAID-6 and RAID-10
Device adapters on DS8700 are 2 Gb/second (Fibre Channel)
Device adapters on DS8800 are 8 Gb/second Fibre Channel
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation29
Caching Algorithms
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation30
Superior Caching Efficiency and Performance
DS8000 uses 4 KB cache slot size – Superior efficiency versus 16 or 64 KB slots
used by some vendors
– Benefits include larger “effective” cache due to smaller cache slot size
Self learning algorithms– Adaptively and dynamically learns what data
should be stored in cache based upon recent access and frequency of access
Open Systems LUN
OR
System z 3390 Device
Experience has taught me Should I keep this data? What data is needed next by
the host?
DS8000 Intelligent Caching
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation31
Cache Algorithms
Sequential Adaptive Replacement Cache (SARC)
– Self tuning algorithm to support a mix of sequential and random I/O streams
– SARC determines• When data is copied into cache• Which data is copied into the cache• Which data is evicted when the cache becomes full• How to adapt the algorithm to differing workloads
– Not just least recently used, but also how frequently referenced
– Resists tendency to store one time use sequential data in cache
Adaptive Multi-Stream Pre-fetching (AMP)
– Complements SARC by managing sequential read pre-fetch cache
– Determines when and what should be pre-fetched
– Addresses pre-fetch waste• When pre-fetched data is evicted from cache before it is used
– Avoids cache pollution• When less useful data is pre-fetched instead of more useful data
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation32
Intelligent Write Caching
Intelligent Write Caching (IWC)– Improves performance through better write cache
management and better destage order of writes
– Based on two well known algorithms• CLOCK and CSCAN
How IWC Works:– Organize write groups into a sorted order forming
a clock
– Clock hand moves clockwise destaging write groups in order
– Write groups are created with bit initialized to 0
– Clock hand can only destage groups with bit 0
– When clock hand encounters a write group with bit 1, it resets bit to zero and skips it
– On a write to an existing group (write hit), set the bit to 1
Head and Tail are glued to form aSORTED CLOCK
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation33
Storage Virtualization
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation34
Storage Hierarchy Disk
– Individual DDMs
Array Sites– Logical Grouping of 8 DDMs of same speed and
capacity Arrays
Array– One 8-DDM Array Sites used to construct one RAID5,
RAID-6 or RAID10 array
Ranks– One Array becomes one CKD or FB Rank – Available space in rank divided into extents
• An extent is the minimum allocation unit when a LUN or CKD volume is created (FB = 1GB, CKD = 1113 cylinders)
Extent Pools– 1-N Ranks form an Extent Pool
• Min of 2 pools—1 each for server0 and server1• Max of 1 pool for each rank
– All Extents in a Pool are same storage type (CKD/FB); same RAID recommended
– Associated with server0 or server1
Extent Pool
Extent Pool
RAID5 or RAID-6 or RAID10
CKD or FB
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation35
RAID-5 Array
Extent Pool / Volume Creation
3390-1
RAID-5 Array RAID-10 Array RAID-10 Array
CKD Extent Pool FB Extent Pool
3390-3 3390-9 3390-27 12 GB 50 GB 101 GB 40 GB
Rank 1 Rank 2 Rank 3 Rank 4
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation36
Logical Volumes
Fixed Block LUNs
– Composed of one or more 1 GB extents from an extent pool• LUNs cannot span multiple extent pools• LUNs can have extents from different ranks within the same extent pool• LUNs can have a maximum size of 16 TB
– Can contain up to 64K FB LUNs
CKD Volumes
– Composed of 3390 model 1, which has 1113 cylinders• When defining, specify the number of cylinders not extents
– Standard CKD volumes up to 65,520 cylinders (55.6 GB) and with EAV, up to 1,182,006 cylinders (1 TB)
– Can contain up to 65,280 volumes
Considerations
– When creating FB LUNs, create LUNs that are a multiple of 1 GB to avoid spatial waste
– When creating CKD volumes, create volumes that are a multiple of 1113 cylinders to avoid spatial waste
– Total of 64K volumes (CKD + FB)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation37
Storage Pool Striping is an algorithm choice for volume creation which allows for better backend disk utilization
Volumes are created by allocating one Extent from available Ranks in an Extent Pool, in a round-robin fashion
– At right - 7 GB Volume showing the order of Extent allocation
CKD and Fixed Block exists in separate extent pools
The next volume will be started from an extent on the next rank in the round-robin rotation
Additional ranks can be added to the extent pool as needed
Storage Pool Striping (aka Rotate Extents)
Extent Pool with 3 Ranks
1
2
3
4
7
5
6
Rank 9
Rank 10
Rank 11
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation38
Volume Groups and Hosts
Volumes are grouped as necessary to create a volume group.
The same volume can be present in multiple volume groups as needed to support application requirements
– Common to share volumes among clustered servers
A host is defined by its world-wide port name (WWPN)
Hosts must contain at least one WWPN and many servers attached to DS8000s have multiple hosts
– Multiple host ports are balanced using MPIO or Subsystem Device Driver (SDD).
Volumes are assigned to the host by connecting the volume group to the host
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation39
Example Host Attachment & Volume Groups
WWPN-1 WWPN-2 WWPN-3 WWPN-4
Host Attachment:AIXprod1
Host Attachment:AIXprod2
WWPN-5 WWPN-6
Host Attachment:Test
WWPN-7
WWPN-8
Host Attachment:Prog
Volume Group:DB2-1
Volume Group:DB2-2
Volume Group:DB2-Test
Volume Group:Docs
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation40
Benefits of Storage Virtualization
Flexible LSS definition allows maximization and optimization of the number of devices per LSS
No strict relationship between RAID ranks and LSSs– Up to 255 LSSs available in DS8000
No connection of LSS performance to underlying storage
Number of LSS can be defined based on device number requirements
Increased number of logical volumes up to 65,280 (FB and CKD)
Increased logical volume size – CKD up to 1 TB
– FB up to 16 TB
Flexible RAID options (RAID-5, RAID-6 and RAID-10)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation41
Benefits of Storage Virtualization (continued)
Extent pools used to create FB and CKD volumes
Virtualization provides the capability to– Enable storage pool striping
– Dynamically add and remove volumes
– Dynamic volume expansion
– Dynamic extent pool merging using Easy Tier
– Dynamic volume relocation using Easy Tier
– Dynamically rebalance hot spots within homogeneous pools
Thin provisioning – allocations from extent pool in 1 GB increments
Track space efficient volumes for FlashCopy SE – repository exists in extent pools
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation42
DS8000 Family Comparison
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation43
DS8000 Hardware SummaryDS8100 DS8300 DS8700 DS8800
DDMs 16-384 16-1,024 16-1,024 16-1,536
DDM Interface 2Gbps FC-AL 2Gbps FC-AL 2Gbps FC-AL 6Gbps SAS-2
Enterprise (FC/SAS) DDM Types FC – 73, 146, 300, 450 GB FC – 73, 146, 300, 450 GB FC – 300, 450, 600 GB SAS - 146, 300, 450, 600, 900 GB
Nearline DDM Types 1 TB 1 TB 2 TB 3 TB
SSD DDM Types 73, 146 GB 73, 146 GB 600 GB 300 GB
RAID Types RAID 5, 6, 10 RAID 5, 6, 10 RAID 5, 6, 10 RAID 5, 6, 10
Max Usable Capacity 216 TiB 586 TiB 1158 TiB 1408 TiB
Max Sequential Bandwidth (MB/s) 2GB/s 3.9GB/s 9.7GB/s 11.8GB/s
Max Number of LUNs / CKD volumes 64K total 64K total 64K total 64K total
Max N-Port Logins/Port 510 510 510 510
Max Process Logins 2K 2K 2K 2K
Max Logical Paths / CU 512 512 512 512
Max LUN Size 2 TB 2 TB 16 TB 16 TB
Dynamic Provisioning Add / Del Add / Del Add / Del / Depopulate rank Add / Del / Depopulate rank
Processor Memory / NVS 16-128GB / 1-4GB 32-256GB / 1-8GB 32-384GB / 1-12 GB 16-384GB / 1-12GB
Processor P5+ 2.2GHz 2-way P5+ 2.2GHz 4-way P6 4.7Ghz 2 or 4-way P6+ 5.0GHz 2 or 4-way
Host Adapters ESCON x 2 ports 4 Gb FC x 4 ports
ESCON x 2 ports 4 Gb FC x 4 ports
4 Gb FC x 4 ports 8 Gb FC x 4 ports
8 Gb FC x 4 or 8 ports per adapter
Host Adapter Slots 16 32 32 16
Max Host Adapter Ports 64 128 128 128
Single DA Throughput 600MB/s 600MB/s 600MB/s 1,600MB/s+
DA Slots 8 16 16 16
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation44
DS8000 Performance Summary
CKD (System z Environment) - DS8800 Full Box Results– RAID5384x 15K RPM HDDs, 48x 10K RPM HDDs, 8x DA Pair, 16x HA w/ 32x 8Gb ports
DS8300 DS8700 DS8800 % increase vs. DS8300
FICON Seq Read GBps 4.1 9.4 10.0 144%
FICON Seq Write GBps 2.1 5.6 5.7 171%
zHPF 4K Write Hits 4KB K IOps 124 159 175 41%
zHPF 4K Read Hits 4KB K IOps 344 423 440 28%
zHPF DB zOS 4KB K IOps 165 201 204 24%
FICON DB zOS 4KB K IOps 124 174 181 46%
FB (Distributed Environment) - DS8800 Full Box Results– 96x RAID5 Arrays768x 15K RPM HDDs, 16x SSDs, 8x DA Pair, 16x HA w/ 32x 8Gb ports
DS8300 DS8700 DS8800 % increase vs. DS8300
Seq Read GBps 3.9 9.7 11.8 203%
Seq Write GBps 2.2 4.7 6.7 205%
Database Open 4KB K IOps 165 191 196 19%
4K Read Miss 4KB K IOps 111 137 160 44%
4K Read Hits 4KB K IOps 425 523 530 25%
4K Write Hits 4KB K IOps 164 203 222 35%
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation45
DS8000 Functional Summary by Release
2004• 255 LCUs Supported• RAID5/RAID10• RMC/zGM/PTC/PAV• 64K Logical Volumes• 2GB FCP/FICON• 73/146/300GB DDs
10-2006• HyperPAV• HMC CIM Agent• 3rd & 4th expansion
frame2-2008• SSPC Support (upgr)• DS8000 M/T intermix • FICON Ext Distance
10-2008 • zHPF• zGM Incremental
Resync
7-2009• Thin Provisioning• Quick init• zHPF Multi-track
support
4-2010• Easy Tier• Thin Provisioning• Quick Init• 600GB 15K• 2TB SATA• Multi-GM• zHPF multi-trk
2006• Turbo Models• 500GB FATA• 4GB FCP/FICON• 242x Machine Types• Synergy Items
2007• SSPC Support (new)• Storage Pool Striping• FC Space Efficient• Dynamic Volume
Expansion
5-2008• Extended
Address Volumes• Variable LPAR• IP v6
2-2009• Solid State Drives• 1 TB SATA• Intelligent Write
Cache• Full Disk Encrypt• Remote pair FC
10-2009• DS8700
10-2010• DS8800
1.06.0
6.1
5-2011• Easy Tier2• IO Priority
Manager Open
• Resource Groups
• Ease of Mgmt
• 16TB LUN
DS8700 / DS8800 include all functional enhancements up to R6.2
DS8100 / DS8300 includes all functional enhancements up to R4.3
2.0 2.4 3.0 3.1 4.0 4.1 4.2 4.3 5.0 5.1 6.0 6.1 6.2
11-2011• DS8800 new
drives• DS8800 4th
frame• Easy Tier 3• I/O Priority
Manager CKD• zHPF QSAM
BSAM BPAM• 1 TB EAV• DB2 list pre-
fetch
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation46
DS8000 Power Consumption
kVA BTU
DS8100 Base Frame with 128 disks 5.8 19,800
DS8100 Exp Frame with 256 disks 6.5 22,200
Full DS8100 Configuration – 384 disks 12.3 42,000
DS8300 Base Frame with 128 disks 7 24,000
DS8300 Expansion Frame 5.5 18,900
Full DS8300 Configuration – 1024 disks 26.6 91,185
DS8700 Base Frame with 128 disks 6.8 23,000
DS8700 Expansion frame 7.2 18,900
Full DS8700 Configuration – 1024 disks 35.6 98,600
DS8800 Base Frame with 4-way, Standard Configuration with 240 disks
7.6 26,000
DS8800 Expansion Frame 7.3 25,000
Full DS8800 Configuration – 1056 disks 36.8 101,000
Power numbers from DS8000 Installation and Planning Guide GC27-2297Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation47
Functions
Easy Tier
I/O Priority Manager
Thin Provisioning
Resource Groups/Multi-Tenancy
System z Synergy Functions
Power Systems Synergy Functions
IBM i and DS8000 Synergy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation48
Easy Tier
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation49
Easy Tier - Overview
Easy Tier provides advanced volume management capabilities– Sub-volume drive tiering
– Command-based volume relocation
– Automated drive utilization balancing to remove any hot spots or populate new empty drives
Easy Tier provides– CLI/GUI setup and management
– Storage Tier Advisor Tool (STAT) for I/O analysis and projected benefit
Easy tier is a licensed advanced function– No charge
– Supported by all server platforms with no additional software
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation50
Easy Tier - Data Relocation OptionsManual Mode (Volume Level) and Automatic Mode (Extent Level)
SSD Rank Pools Enterprise or Nearline Pools
Merged Pool (Two or three tiers)
Automatic extent level data relocation enabled in a Merged Extent Pool
Manual ModeVolume Based Data Relocation
Automatic ModeExtent Level Data Relocation
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation51
Easy Tier - Workload Learning
Applications
VirtualDisk
Performance monitoring and reporting available to track
the I/O demand from application and I/O service time from storage device
Performance data is collected for multiple durations, hours,
days and weeks
Heatmap and I/O Density Report
SmartMonitoring
Data collection period = 5 minutes
For all active extents
Solid-state
Enterprise - FC / SAS
Nearline - SAS / SATA
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation52
Easy Tier – Automatic Mode Overview
Extent Migration Plan built based on I/O statistics
Dynamically relocates a logical volume’s extents
– Hot extents relocated to higher performance class of disk (enterprise SSD)
– Cold extents relocated to lower performance class of disk (enterprise Nearline)
Extent level relocation requires mixed technologies in a merged extent pool (between any two or three tiers), for example:
– SSD + Enterprise + Nearline
– SSD + Nearline or
– Enterprise + Nearline
DS8000 Extent Size
– 1GB for FB
– 3390 Mod 1 (0.94 GB) for CKD
Improves storage cost-performance
– Utilize Storage Tier Advisor tool to determine optimal SSD configuration and benefits
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation53
Easy Tier – Automatic Mode Extent Relocation
Logical Volume Mixed Technology Extent Pool
SSD Arrays
HDD Arrays
Extent Virtualization
Hot Extents Migrate Up
Cold Extents Migrate Down
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation54
Easy Tier – Automatic Mode Setup
Options for configuring extent pools for Automatic Mode– Create new extent pool with any two or three tiers of disk
– Merge existing tiers of disk extent pools
– Add SSD rank to existing HDD extent pool or Nearline
– Add HDD rank to existing SSD extent pool if no space efficient capacity (virtual capacity or repository) configured in the extent pool
Options for configure logical volumes for Automatic Mode– Create new logical volume in mixed technology extent pool
– Migrate standard logical volumes between homogeneous and mixed technology extent pool
– Merge existing homogeneous extent pool with existing logical volumes with another extent pool to create a mixed technology extent pool
Existing recommendations continue– Each extent pool utilize same RAID format, size and speed of HDDs
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation55
Easy Tier – Automatic Mode User Interfaces
Design point is for customer to not normally need or use any controls to manage Automatic Mode
Easy Tier Automatic Modes– Tiered pools – automatic mode extent migration between disk tiers – All pools – automatic extent migration between tiers and homogeneous pool
rebalancing– No pools - Stops automatic mode extent migration and prevents homogeneous pool
rebalancing
Easy Tier monitoring– Collects heat data for analysis in STAT tool but does no extent migration
Analysis Tools– Can offload reports on extent monitoring and obtain SSD capacity planning
recommendations– Can engage IBM for extended analysis and consulting
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation56
Easy Tier – Homogeneous Pool Rebalancing
Easy Tier will automatically rebalance within a homogeneous extent pool
– Reduces skew and hot spots within a rank and redistributes extents across ranks within the homogeneous pool
hothot coldcoldRank hothot coldcold
hothot coldcoldRank
hothot
coldcold
Rank
Rank
New rank added to pool
hothot coldcoldRank hothot coldcold
hothot coldcoldRank
hothot
coldcold
Rank
Rank coldcold
coldcold
Natural Performance Skew
coldcold coldcold coldcold
coldcoldcoldcold
coldcold
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation57
Easy Tier – Manual Mode Overview
Easy Tier Manual Mode allows a user to perform the following actions:
– Volume Migration• User can change a logical volume’s storage technology by dynamically
relocating between extent pools• User can change a logical volume’s extent allocation algorithm (EAM)
(e.g. can re-rotate extents within the target extent pool)
– Extent Pool Merge• User can merge two existing extent pools without moving data
– Consolidate extent pools with equivalent disks– Merge extent pools with to create a mixed technology extent pool for Automatic Mode
– Rank Depopulation• Storage Administrator can ask that a rank be removed from an extent
pool• Automatic, non-disruptive and transparent to host access, the used
extents will be reallocated to other ranks in the pool and rank freed
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation58
Easy Tier – Manual Mode Volume Migration
Solid State Disk146 GB / RAID 5
Solid State Disk146 GB / RAID 5
+Enterprise Disk
300 GB / 15K RPMRAID 5
Nearline Disk2 TB / 7.2K RPM
RAID 6
Enterprise Disk600 GB / 15K RPM
RAID 5
Change disk classChange RAID typeChange disk RPM
Re-stripe extents
Change between extent poolswith and without Easy Tier automatic mode
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation59
Easy Tier – Manual Mode User Interfaces
Migrate Volume (Target Extent Pool, Extent Allocation Method)– Can specify current extent pool or another extent pool– Can specify extent allocation method (rotate volumes or rotate extents)
Pause/Resume Volume Migration– Pause puts volume in migration paused configuration state and stops initiation of any
new extent migrations on a volume– Resume puts volume in migrating configured state and continues migration
Cancel Volume Migration– Nullifies volume migration if it has not started and puts volume in normal configuration
state– Stops volume migration if it has started and puts volume in migration cancelled
configuration state. Can request migrate volume to source or target extent pool to retry.
Merge Extent Pool– “Moves” all volumes in the source extent pool to the target extent pool– Deletes source extent pool if merge is successful
Rank Depopulation– Can use Easy Tier to depopulate a rank and remove from an extent pool– Automatic, non-disruptive and transparent to host access
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation60
Easy Tier – Supported Environment
Supported in DS8700 R5.1+ and DS8800 R6.1+
Automatic Mode and Volume migration supported on standard logical volumes
Track Space Efficient (TSE) volumes are not Easy Tier managed– Can reside in an extent pool managed by Automatic Mode
– Volume migration is not supported on space efficient volumes
Cannot migrate between extent pools on different storage images (0 / 1)
Copy services considerations– Easy Tier optimization of data on the primary system is not reflected at the secondary
Can merge any two extent pools except:– Both must be same extent type (CKD or Fixed Block)
– At most, one of two extent pools can have a space efficient repository
– At most, one of one of two extent pools can have virtual capacity
– Not allowed if one extent pool is homogeneous with SSD disks and additionally has space efficient repository or virtual capacity configured
Easy Tier automatic mode is not supported on encryption capable storage facilities – however Easy Tier manual mode is supported
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation61
Easy Tier – Copy Services
Copy services is not aware of Easy Tier data optimization
– Any manual volume level migrations should be performed on both the primary and secondary system• In the event of a failover, data optimized as primary and secondary
reflect the same data placement– In Automatic Mode, relocation of extents on the primary system is not
reflected at the secondary• I/O workload being collected at the primary and secondary are different
– Normal production workloads to the primary versus write only to the secondary– Easy Tier will make different optimization decisions for the different workload profiles
• Applies to Metro Mirror, Global Mirror and z/OS Global Mirror– Will take time to get to an optimized environment in the event of a failover
• Easy Tier will have to analyze the production workload, relearn and redistribute data based on this workload
Easy Tier will manage Thin Provisioned volumes
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation62
Easy Tier – Workload Considerations
Implementation Characteristics
– Extent size is 1 GB on FB, Mod 1 on CKD
– Automatic mode plan generation window is 24 hours
Some workloads may not benefit significantly from Automatic Mode
– Hot spots are small in size and uniformly distributed across extents such that all extents exhibit equal temperatures
– Hot spots vary over time such that they are uniformly distributed given a large enough monitoring period
– Critical workload to be performance optimized is intermixed with other workloads that result in a non-optimal extent placement• May be able to turn off monitoring in time windows where non-critical workloads are affecting
statistics in an undesirable manner (e.g. batch windows, off-shift or weekend workloads, month-end processing, etc.)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation63
Storage Tier Advisor Tool (STAT)
I/O workload information and disk tier recommendations– Pool, rank and volume information
Usage– Understand workload– Plan automated disk tiering– View results of automated disk tiering– Plan manual volume migration or extent pool merge
Reporting– System Summary - Tier status per pool (including rank IOPs/BW overloaded, rank IOPs skew)– System Recommendations – SSD, nearline or Enterprise1 recommendations– Extent Pool Reports – Tier status, (including rank utilization), SSD/ENT/Nearline recommendations,
Volume Heat Distribution
Requirements– Performance Monitoring (supporting code level (R6.1 for new support), Storage Image Monitor setting,
single tier extent pool or multi-tier extent pool)– Offload statistics via DS8000 Storage Manager GUI or DSCLI– Download STAT (free) and run on Windows– Easy Tier licensed feature (no charge) is required for monitoring
1Enterprise disk class includes both Fibre Channel and SAS drives.
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation64
STAT System Summary
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation65
STAT System Recommendation
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation66
STAT Storage Pool Status and Recommendations
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation6767
STAT Volume Heat Distribution
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation68
DS8000 Easy Tier Releases at a Glance
Easy Tier V1 (DS8700 R5.1)1. Automated cross-tier performance management for SSD/HDD hybrid pools
2. Manual mode management support for dynamic extent pool merge and dynamic volume relocation
Easy Tier V2 (DS8700/DS8800 R6.1)3. Automated cross-tier performance or storage economics management for
hybrid pools with any 2 tiers (SSD/ENT, SSD/NL or ENT/NL)
4. Automated intra-tier performance management (auto-rebalance) in hybrid pools
5. Manual mode management support for rank depopulation and optimized volume restriping within non-managed pools (manual volume rebalance)
Easy Tier V3 (DS8800/DS8700 R6.2)6. Automated cross-tier performance and storage economics management for hybrid pools
with 3 tiers (SSD/ENT/NL)
7. Automated intra-tier performance management in both hybrid (multi-tier) as well as homogenous (single tier) pools (auto-rebalance)
8. Thin Provisioning support for Extent Space Efficient (ESE) Volumes
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation69
Merged Pools (SSD+HDD)
SSD Pools Nearline Pools
Volume-based data relocation
Cross-tier data relocation
Manual volume migration Change Disk Class Change RAID Type Change RPM Change striping
Automated intra-tier rebalance
Enterprise Pools
Easy Tier managed pools
Summary of Easy Tier Migration Capabilities
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation70
I/O Priority Manager
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation71
DS8000 I/O Priority Manager
Application Level Quality of Service (QoS)– Provide mechanisms to manage quality of service for I/O
operations associated with critical workloads and to give them priority over other I/O operations associated with non-critical workloads
– Adaptive based on workload and contention for resources in storage subsystem
Monitoring functions do not require LIC feature key
• Can monitor workload without activating the LIC feature key and alert via SNMP
I/O Priority Manager volume level support:
– FB volumes introduced in Release 6.1
– CKD volumes introduced in Release 6.2
SLOW
PRIORITY
Help!
RAID
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation72
DS8000 I/O Priority Manager Controls
I/O Priority Manager Modes
– Disabled
– Monitor, Monitor+SNMP
– Manage, Manage+SNMP
I/O Priority Manager performance policies and groups
– There is a set of pre-defined performance policies, subset into performance groups for monitoring• One Default Performance Group – No Monitoring / No Management• 5 High Priority Performance Groups – Monitor/Manage• 5 Standard Performance Groups – Monitor/Manage• 5 Low Priority Performance Groups – Monitor/Manage
– All logical volumes are associated with a single performance group• Existing or newly created volumes default to the default performance group unless another performance group is
specified
– Performance statistics and SNMP traps can be obtained for a performance group that is monitored or managed
– I/O operations are managed in a performance group that is managed• Same policy applied to all I/O for a given volume based on performance policy for its associated performance group
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation73
DS8000 I/O Priority Manager – Principles
Certain resources affect I/O response time and/or throughput
– A given resource has a certain capacity to process I/O operations
– When the workload is below the maximum capacity of the resource, average I/O operation response times are typically in the nominal response range
– When the workload approaches the maximum capacity of the resource, I/Os end up getting queued and the average I/O response times can grow significantly above the nominal response range
– It is possible to proactively manage queuing of I/O operations on the resource to give priority to critical I/O operations at the expense of less critical I/O operations • Production versus development• Critical versus business as usual applications
I/O priority manager can add a delay to any given I/O
– By delaying some I/O, other I/Os get more throughput and better access to resources or available processing power
– By applying policies to determine which I/Os to consider delaying or how much to delay, the more important I/O operations are given appropriate consideration when there is resource contention
– I/Os are only considered for delay when there is contention for a resource that they require
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation74
DS8000 I/O Priority Manager – Operation
When monitoring or management is enabled – Behavior of disk rank stage/destage operations is monitored to detect when rank is running at saturation
(“Very High Usage”)
– If SNMP alerts are enabled, an alert is sent as resource enters saturation level
When management is enabled
– I/O operations in a performance group that is managed and that are associated with a rank that is not close to saturation are not delayed
– I/O operations in a performance group that is managed and that are associated with a rank that is close to saturation (“High Usage”) may be delayed based the performance policy• I/O with Low Priority performance policy is impacted as needed to improve QoS of Standard and
High Priority I/O• I/O with Standard performance policy is impacted less than Low Priority, and only if needed to improve
the QoS of High Priority• I/O with High Priority performance policy is not impacted• I/O with default priority performance policy is not impacted
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation75
CKD Rank Contention
Without z/OS software support
– On ranks with contention, I/O to a volume is managed according to the performance group of the volume
With z/OS software support
– User assigns application priorities via WLM
– z/OS assigns an ‘importance’ value to each I/O based on WLM inputs
– z/OS assigns an ‘achievement’ value to each I/O based on prior history of I/O response times for I/O with same ‘importance’ and based on WLM expectations for response time
– ‘Importance’ and ‘achievement’ value on I/O associates this I/O with a performance policy
– On ranks in contention, I/O is managed according to I/O’s performance policy
SLOW
PRIORITY
Help!
RAID
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation76
DS8000 I/O Priority Manager – Statistics
Each Performance Group has statistics for a set of volumes
Multiple performance groups per performance policy to manage different sets of volumes
Performance statistics samples are generated every 60 seconds
– The DS8000 maintains statistics for last • 60 1-minute intervals • 60 5-minute intervals • 60 15-minute intervals • 60 1-hour intervals • 60 4-hour intervals • 60 1-day intervals
Statistics can be offloaded DS/CLI or DS/GUI query
– Resource report
– Performance group report
– Offload request can specify a period of time and the sample interval
User may iterate setting volume performance groups and analyzing statistics to tune the result
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation77
DS8000 I/O Priority Manager – Example With I/O Performance Manager, the critical (favored) workloads are protected
from other non-critical (non-favored) workloads sharing the same resources.
DB like workload
0
10000
20000
30000
40000
50000
1 6 11 16 21 26 31
Time (min)
Th
rou
ghp
ut
(IO
/s)
Favored Non-favored
DB like workload
0
50
100
150
200
1 6 11 16 21 26 31
Time (min)
Res
po
nse
Tim
e (m
s)
Favored Non-favored
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation78
DS8000 I/O Priority Manager – Normal Operations
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation79
DS8000 I/O Priority Manager – With Resource Saturation
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation80
DS8000 I/O Priority Manager – Sample Report
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation81
DS8000 I/O Priority Manager – Create FB Volume Example
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation82
DS8000 I/O Priority Manager with Easy Tier
I/O Priority Manager and Easy Tier provide complementary benefits
I/O Priority Manager attempts to:
– make sure the most important I/O operations get serviced when a given rank is overloaded by the workload on the storage system
Easy Tier Automatic Mode attempts to:
– Locate allocated extents on a storage tier that is appropriate for the frequency of host access to maximize the throughput of the most active extents.
– Relocate extents between ranks within a storage tier to distribute the workload evenly the across available ranks and avoid rank overloading.
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation83
Thin Provisioning
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation84
Thin Provisioning
Thin provisioning allows a storage system to provide a volume to an application that is larger than the actual space consumed – When a thin provisioned volume is assigned to a host, the host sees the whole
(virtual) capacity of the volume, as if it were a fully-provisioned volume
As application data is written to the volume:– The disk subsystem then backs the volume with real capacity
– All I/O activities performed by the storage subsystem to allocate space when needed are fully transparent to the host
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation85
Full Provisioned Volume Versus Thin Provisioned Volumes
Full provisioned volumes– Consume the full amount of disk
space that is allocated in the lun regardless of the amount of data stored on the disk
Thin provisioned volumes– Only consume the amount of
space that the application has written to the disk
– As additional space is consumed, the DS8000 will provide additional space in 1 GB increments
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation86
Benefits of Thin Provisioning
Reduced storage management efforts– Exploiting over-provisioning in file systems, databases, logical volume
managers without the underlying physical capacity to be present
– Real capacity can be added automatically and transparently later as required
Improved capacity utilization– Physical capacity does not get allocated until actually used
– Unused capacity for a set of volumes resides in a shared pool of overall available capacity
– Contingency capacity can therefore be shared between this set of volumes
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation87
Thin Provisioning Supported Environments
Thin Provisioning currently supports FB luns
– No support for System i
– No support for CKD volumes on System z
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation88
Resource Groups / Multi-Tenancy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation89
DS8000 Resource Groups
Provides policy based limiting of:– Copy services relationships
– Pass-thru operations
– GM sessions/masters
– Copy Services operator access to copy services resources
Available for both CKD and FB volumes– No LIC feature required for Resource Groups functions
Meets requirements for managing copy services in a multi-tenancy environment
Can utilize resource groups to create resource domains to control copy services relationships and to prevent copy services operator errors from escaping a given domain
Introduced in Release 6.1
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation90
Multi-tenancy copy services environment
Site 1 Site 2
Hosts with LPARs
Sharks
Jets
Sharks
Jets
Sharks
Jets
Sharks
Jets
SSP
Switches Switches
Hosts with LPARs
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation91
System z Synergy Functions
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation92
IBM DS8000 System z Synergy Examples zHPF Enhancements (now includes all z/OS DB2 I/O)
Extended Distance FICON Caching Algorithms – AMP, ARC, WOW, 4K Cache Blocking
DFSMS Recognition of SSDs
Easy Tier
z/OS GM Enhanced Reader Support
SSDs + DFSMS + zHPF + HyperPAV + DB2
I/O Priority over FICON & within DS8K managed by zWLM Service Class
zWLM + DS8000 I/O Priority Manager
PAV, HyperPAV, MIDAWs
GDPS & GDOC Automation
HyperSwap Technology Improvements
Remote Pair FlashCopy & Enhancements
zCDP for DB2, zCDP for IMS – Eliminating Backup Windows
1TB EAVs
Quick Init for CKD Volumes
Dynamic Volume Expansion
FlashCopy and Dataset level FlashCopy
z/OS Distributed Data Backup
System z Discovery & Automatic Configuration (zDAC)
Alt Subchannel Exploitation
Disk Encryption
Performance
Availability
Management/Growth
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation93
Components of System Z I/O Response Times
IOSQTime
PendingTime
ConnectTime
DisconnectTime
Parallel Access Volumes (PAV)
Multiple Allegiance MIDAWs Adaptive Multi-Stream Pre-Fetching (AMP)
HyperPAV System z High Performance FICON (zHPF)
Intelligent Write Caching (IWC)
Sequential Adaptive Replacement Cache (SARC)
Solid State Drives
DS8000 functions and features to address response time components
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation94
Parallel Access Volumes (PAV)
PAVs were originally introduced in 1999 with the introduction of the Enterprise Storage Server (ESS)
Allows System z applications to share the same logical device by creating an alias for the base device
3 variations on PAVs– Static – Alias is always bound to same base device
– Dynamic – Uses Workload Manager to dynamically allocate aliases from a shared pool
– HyperPAVs – More efficient than dynamic PAVs. Each System z image has its own pool • Reduces the total number of aliases needed for a given workload• z/OS reacts quicker to I/O workload changes• Overhead of managing aliases is reduced as Workload Manager is not involved in
assigning/moving aliases
z/VSE supports PAVs
z/OS, z/VM and Linux for System Z support PAV and HyperPAV
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation95
Traditional z/OS Behavior – No PAVs
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation96
z/OS Behavior with PAVs
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation97
Dynamic PAVs
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation98
HyperPAV – More Than Just a Regular PAV
Reduce the number of aliases– Give back addressable device numbers
– Use additional device addresses to • Support more base addresses• Larger capacity devices
z/OS can react more quickly to I/O loads– React instantaneously to changing workloads
Overhead of managing alias exposures reduced– Workload Manager not involved in measuring and moving aliases
– Alias moves are not coordinated throughout the Sysplex
I/O reduction, no longer need to BIND/UNBIND to manage HyperPAV aliases
Increases I/O parallelism
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation99
HyperPAV
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation100
Multiple Allegiance and Priority I/O Queuing
System z does not normally permit I/O to a device concurrently while another system has an I/O active
Other system receives “device busy” – accumulates in Pending Time (Device Busy Delay)
Multiple Allegiance allows multiple I/Os to run concurrently if disk extents are different
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation101
Multiple Allegiance
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation102
MIDAWs
Originally Introduced by IBM on the System z9 processor
Improves FICON performance– Allows ECKD channel programs to read and write to many storage locations
using one channel command
– Improves the performance of sequential I/Os using 4K data sets, especially when using extended format data sets
– Eliminates the extended format penalty and shrinks the small DB2 page size performance penalty.
– MIDAWs implemented by Media Manager
– Noticeable improvements for:• Extended format data sets accessed through Media Manager• DB2• Extended format VSAM files
Benefit: Lower utilization of FICON channel, link and control unit
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation103
z/OS Extended Address Volumes
Problem: Running out of System z device addresses– 4 digit device number limit is fast approaching and not easily
bypassed
– Business Continuity solutions also push this limit
Solution: Extended Address Volumes– Continue the direction started with 3390-9 of defining larger
volumes by increasing the number of cylinders
– EAV volumes support up to 1,182,006 cylinders (1 TB)
– Updates to CCHHR in z/OS V1.10 allows shifts bits • 16 bits in CC grows to 28 bits• 16 bits in HH shrinks to 4 bits
– Supports on 3390 formatted volumes (not 3380 format)1 TB
1 GB
Mod 1
Mod 1062
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation104
Extended Address Volume (EAV) is the Next Step in Larger z/OS Volumes
3390-3 3390-93390-9
3390-9
3390-A “EAV”
3GBMax cyls: 3,339
9GBMax cyls: 10,017
27GBMax cyls: 32,760
54GBMax cyls: 65,520
100s of TBs
Maximum SizesSize limited to 1 TB (Max1,182,006 cylinders)
EAV: A volume with more than 65,520 cylinders
The HyperPAV function complements this design by scaling the I/O rates against a single volume
3390 Model A: A device configured to have 1 to many cylinders
Introduced in z/OS V1R10
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation105
High Performance FICON Introduction
High Performance FICON for System z (zHPF) is a new data transfer protocol that is optionally employed for accessing data from an IBM DS8000 storage subsystem
– Data accessed by DB2, PDSE, VSAM, zFS, QSAM, BSAM, BPAM and Extended Format SAM can benefit from the improved transfer technique
zHPF may help reduce the infrastructure costs for System z I/O by efficiently utilizing I/O resources so that fewer CHPIDs, fibers, switch ports and control unit ports may be needed
– zHPF also compliments the System z EAV strategy for growth by increasing the I/O rate capability as the volume sizes expand vertically
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation106
zHPF Evolution
2009
2010
2011
DS8100/DS8300 with R4.1 or above
z10 processor
Format writes, multi-domain I/O
QSAM/BSAM exploitation
z/OS R11 and above, EXCP
Multi-track, but <= 64K
Multi-track any size
Extended Distance I
z196 processor >64K transfers
Single domain, single track I/OReads, update writes
Media manager exploitationz/OS R8 and above
DS8700/DS8800 with R6.2
z196 FICON Express 8S
Extended Distance II
SDM, DFSORT, TPF, etc….
EXCPVR support
ISV Exploitation
100% of DB2 I/O is now converted to zHPF
Typical Client will have 90%+ of all DASD I/O converted to zHPF
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation107
High Performance FICON Highlights
Maximum Application Benefit for Typical OLTP Workloads– Optimize DB2 Performance. Media Manager builds a new type of Channel
Program • Transport Control Word (TCW) instead of Command Control Word (CCW)
– Use simpler protocols to encapsulate channel programs while preserving the enterprise class qualities of service of FICON
– Complex channel programs continue to use CCW Chains with base FICON protocols
– Maximum I/O rate for a channel with a simple 4KB read hit benchmark doubles with zHPF
– Realistic production workloads with a mix of data transfer sizes may see up to 30% savings in channel utilization compared to FICON
– Sequential workloads that transfer up to a single track (for example, 12 x 4KB per I/O) may also benefit
– OLTP Workloads that exploit zHPF could see up to 30% improvement in DS8000 throughput
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation108
High Performance FICON (zHPF)
Improve FICON Scale, Efficiency and RAS– As the data density behind a CU and device increase, scale I/O rates and
bandwidth to grow with the data• Significant improvements in I/O rates for OLTP• Improved I/O bandwidth • New ECKD commands for improved efficiency
– Improved first failure data capture
– Additional channel and CU diagnostics for MIH conditions
Value– Reduce the number of channels, switch ports, control unit ports and optical cables
required to balance CPU MIPS with I/O capacity
– Reduce elapsed times (DB2, VSAM) 2X
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation109
High Performance FICON Highlights (continued)
Compatibility Between Existing CCWs and New TCWs– Bilingual Channel and Control Unit Ports
– CCWs continue to use FICON protocols
– TCWs use new Transport Mode Protocols
DS8000 Code Structure Optimized for Simple I/O Chains– No CKD operation (ECKD only)
– Streamlined Internal Communication Protocols (Equivalent to FCP Exchanges)
Improved RAS and Workload Management – Additional channel and control unit diagnostics for MIH conditions
– I/Os are queued in control unit when a device is reserved by another host
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation110
Link Protocol Comparison for a 4 KB Read
CHANNEL
CONTROL
UN I
T
OPEN EXCHANGE,
PREFIX CMD & DATA
READ COMMAND
CMR
4K of DATA
STATUS
CLOSE EXCHANGE
FICON
C
H
A
N
N
E
L
C
O
N T
R O L
U
N
I
T
OPEN EXCHANGE, send a Transport Command IU
4K OF DATA
Send Transport Response IU
CLOSE EXCHANGE
zHPF
zHPF provides a much simpler link protocol than FICON
zHPF requires System z10 processor or higher
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation111
Backup System/Restore for System z (zCDP for DB2)
Application based continuous data protection for DB2 on System z– Joint solution between DFSMS, DB2 and DS8000
Solution based on Point-in-Time (PIT) backups combined with DB2 logging– Eliminates the need for DB2 Log Suspend
• Only object level creates, extends, renames and deletes are suspended• Hundreds of volumes backed up in a matter of minutes
– Managed tape copies created from PIO copies
– Recovery at the System or Tablespace level
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation112
SMS Enhancements to Support zCDP
New “copy pool” SMS construct
– Defines which storage groups should be processed collectively for fast replication functions
New “copy pool backup” SMS storage group type
– Defines which volumes DFSMShsm may select as target volumes for fast replication backup versions
ISMF modified to support these SMS enhancements
Three new DFSMShsm Commands
– FRBACKUP – creates a fast replication backup version for each volume in a specified copy pool
– FRRECOV – Use fast replication to recover • Entire copy pool from a disk copy• Individual volume from a disk or tape copy• One or more data sets from a disk or tape copy
– FRDELETE – delete one or more unneeded fast replication backup versions
– DB2 utilities uses these HSM functions
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation113
z/OS FICON Discovery and Auto-Configuration (zDAC)
DS8700 and DS8800 supports new z/OS FICON Discovery and Auto-Configuration function of z/Enterprise z196
Reduces the complexity in a complex FICON environment
Discovery– Capability to discover attached disk connected to FICON fabrics
– Detects new storage subsystems and new control units
– Proposes paths for all systems to newly discovered logical control units
Auto-configuration– Compares newly discovered system with target IODF and proposes new
configuration to user
Requires z196 and z/OS 1.12+
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation114
zDAC Flow
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation115
Power Systems Synergy Functions
AIX– End-to-end I/O Priority
– Cooperative caching
– Long busy wait host tolerance
– PowerHA extended distance extensions
IBM i and DS8000 Synergy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation116
End-to-end I/O Priority
Introduced to the SCSI T10 standards body by IBM
Allows a trusted application to override the priority given to each I/O by the operating system
– Normally all I/O requests inherit the priority of the AIX process
DB2 on AIX exploits this function
DS8000 host adapter will give preferential treatment to higher priority I/O
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation117
Cooperative Caching
Cooperative caching allows a trusted host application to provide cache hints to the DS8000– For example, DB2 can tell DS8000 that recently accessed data will unlikely be
accessed again soon so DS8000 can destage and use cache slots for data more likely to be reaccessed again soon
Currently supported by AIX and DB2 on DS8000
System p servers with AIX, MPIO and the Path Control Module exploit this function
Raw file systems and AIX 64-bit kernel
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation118
Long Busy Wait Host Tolerance
SCSI long busy wait is another SCSI T10 standard
Hosts are notified that port is in long busy and DS8000 provides status response on how long the initiator should wait before retry
AIX on System p uses this status response to reduce retries and to prevent exceeding retry threshold set.
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation119
PowerHA Extended Distance Extensions
PowerHA System Mirror Enterprise Edition provides LPAR and server failover capability over extended distances
PowerHA supports DS8000 Metro and Global Mirror– DS8000 with Metro Mirror supports this technology to distances up to 300 km
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation120
IBM i and DS8000 Synergy 2008 - 2011
Synergy items 2008-2011 – IBM i support for DS8800
– PowerHA - ACS integration with TPC-R
– Increased integration of IBM i performance tools with DS8000
– IBM i OS based ‘hot data’ ASP Balancer and DB2 media preference support for DS8000 Solid State Drives (SSD) and EasyTier
– PowerHA NPIV Virtual I/O Server with DS8000
– Full support of Virtual I/O Server (VIOS) and PowerHA
– PowerHA + DS8000 copy services end-to-end integration
– Common Smart-IOA fiber for disk and tape
– New 4Gb / 8Gb Smart fiber I/O Adapter (IOPless) 6.1 (++ performance)
– Tagged Command Queuing and Header Strip Merge (+++ performance)
– Common one-stop POWER/DS8000 support from Supportline experts
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation121
DS8000 and POWER IBM i Performance Synergy
Scalable, DS8000 Performance and management for IBM i
– IBM i Smart-IOA fiber channel increase performance while significantly reducing hardware
– DS8000 Management Toolkit – IBM Lab Services
DS8800
– Up to 40% better performance, with a reduction in both power consumption and floor space
Solid State Drives (SSD)
– SSD tools and automation for both internal and DS8000
Performance Planning – IBM Disk Magic
– IBM/BP delivered custom performance and capacity sizing for your workloads
Performance Monitoring - Integration of IBM i Tools and DS8000
– Focus on end-to-end performance monitoring and investigation for an IBM i and DS8000 environment
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation122
IBM i and DS8000 Performance Monitoring Integration
POWER IBM i DS8000Cache
Ranks
Links
LUN
Mem
CPU
Other
PerformanceStatistics
IBM i Collection Services SAN Infrastructure
Performance monitoring integration and ease of use
Focus on end-to-end performance monitoring and investigation for an IBM i and DS8000 environment
IBM i 7.1 adds a new category to IBM i Collection Services
– *EXTSTG – new collection performance metrics from DS8000 • Requires DS8000 R4 or later firmware
– Data can be presented in graphs using iDoctor today• Performance Data Investigator (PDI), in a future release (next update semi-annual function
update)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation123
Why PowerHA SystemMirror for IBM i and DS8000?
Hardware level resiliency and replication
– Does not rely on software ‘application transaction replay’ techniques
– Eliminates the causes of out-of-sync situations
– Fussy applications, complex SQL, reorgs, deletes, heavy batch, etc. are not a replication headache
– Scalable, robust and automated solution
– Always ready to switch
Network
PROD (source) HA (target)
LPAR-1 LPAR-2
PowerHA-DS8000 storage replication of IASP
DS8000 DS8000
PROD (source) HA (target)
LPAR-1 LPAR-2
PowerHA-DS8000 LUN-IASP switching
DS8000
Network
PROD (source) DR (remote)
LPAR-2
PowerHA-DS8000 combinations
HA (local)
LPAR-1 LPAR-1
DS8000DS8000
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation124
Encryption
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation125
The Need for Encryption
Increasing number and sophistication of threats. You have to be able to defend against all threats rather than just respond to intrusions
Preventing data breaches and inappropriate data disclosure, while ensuring no impact on business and productivity
Intrusions that affect customer confidence and business productivity. Security breaches can destroy your brand image and affect your critical business processes
Growing demand for regulatory compliance and reporting. You must be able to meet a growing number of compliance initiatives without diverting resources from core activities
Protecting your data and maintaining appropriate levels of access
Security issues are both internal and external. How do you protect against the well-intentioned employee who mishandles information, and the malicious outsider?
Having your business comply with a growing number of corporate standards and government regulations; you must have tools that can document the status of your application security
Growing number of regulatory mandates. You have to prove that your physical assets are secure
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation126
Why Encrypt Disks in the Storage Unit
Breach of Security is defined as the loss of confidentiality (secret data exposed), integrity (unauthorized users modifying data), or availability (system unusable)
The DS8000 Full Drive Encryption (FDE) features address data confidentiality– Active authentication mechanisms in place while storage in use
– This disappears when equipment is removed from environment (and no one should be authorized access to this data)
– DDM replacement
– Lease expiration
Logical volume does not equal physical volume– Not all customer data resident on disk is host accessible
Secure erasure is an option
Encryption is an option
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation127
What’s on a data disk in the DS8000?
What if we removed a disk from the DS8000? (Assuming the disk isn’t a spare) What makes this different than a disk from a PC?– Disk and Logical Volume Meta-data, available extents
– Data written on 256KB strips• One part in 3 or 4 if RAID10 array format• One part in 6 or 7 if RAID 5 array format• Rotating Parity (one strip in 7 or 8 disks is parity)• Possible RAID rank types (5, 6, 10)• Minimum Logical Volume size, one extent=1GB(binary)=4096 strips
RAID5 6+PRAID5 7+PRAID10 3+3
D DD D
D PD S
D DD D
D DD P
D DD S
D DD S
D DD D
D DD D
RAID10 4+4
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation128
Encryption for Data at Rest
The DS8000 uses special drives, known as Full Drive Encryption (FDE) to encrypt data at rest
Data is always encrypted on write to the drive and decrypted on read. Data stored on the drive is encrypted
Drives do the encryption at full data rate so no impact to disk response times using AES 128 bit encryption– Protection for disk removal (repair, replace or stolen)
– Protection for disk subsystem removal (retired, replaced or stolen)
Requires authentication with key server before access to data is granted– Key management is via IBM Tivoli Key Lifecycle Manager (TKLM)
– Key exchange with TKLM is via 256 bit encryption
– z/OS can also use IBM Security Key Lifecycle Manager (ISKLM)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation129
IBM Tivoli Key Lifecycle Manager (TKLM) The TKLM works with IBM encryption-enabled storage devices in generating, protecting,
storing and maintaining encryption keys that are used to encrypt information being written to and decrypt information being read from storage media.
TKLM executes in the IBM Java run time environment and it uses IBM Java security components for the cryptographic capabilities used.
Supported Operating Systems – TKLM AIX 5.3 and 6.1 Red Hat AS 4.0 Red Hat AS 5.0 Suse Linux 9, 10, 11 Solaris 9, 10 Sparc Windows Server 2003 or 2008 z/OS V1 R9, R10, or R11+ (TKLM Version 1, hosted in the System Services
Runtime Environment for z/OS)
Designed to be Easy to use Provide a Graphical User Interface
Initial configuration wizards
Easy backup and restore of TKLM files One button, single jar file
Lifecycle functions Notification of certificate expiry Automated rotation of groups of keys
Same TKLM can be used with IBM DS8000, DS5000, and IBM tape
Products from Emulex, Brocade and LSI also work with TKLM
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation130
TKLM continued
The keys used by TKLM are a public/private asymmetric key pair referred to as the public Key Encrypting Key (KEK) and the private Key Encrypting Key (KEK’), respectively.
The key generation and propagation processes on the TKLM, associate a Key Label to each wrap/unwrap pair
This Key Label is a user specified text string and retained with each wrap/unwrap pair
Key negotiation and authentication between TLKM and the DS8000 take place at DS8000 power on.
One TKLM key server can easily handle multiple DS8000s and DS5000s, the network traffic requirement is small
Two TKLM servers are required to prevent a deadlock condition
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation131
Security Key Lifecycle Manager for z/OS V1.1
Attributes of encryption and key management:
– Encryption in storage hardware does not hurt performance
– Encryption and key management doesn’t require changing applications, middleware, JCL, operating systems
– Key management completely separate from the data path
– Storage arrays and libraries contact the key manager on behalf of the application and hosts doing I/O• With disk arrays done at power up• With tape libraries at each cartridge mount
Encryption and key management fits into your operations management
– Separation of duties
– Leverage investments in high availability and security
ISKLM V1.1 benefits:
– Easy upgrade from EKM, easy SMPE install
– Still supports ICSF, RACF, crypto express hardware
– Writes SMF records type 83 subtype 6 audit records
– Supports all of the latest system z/OS centric storage – tape and disk
– No longer requires DB2 or SSRE • Goal was simplest key serving with no co-reqs Disk Storage
ArrayEnterprise Tape
Library
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation132
Encryption Drives
These drives are specialized drives that include encryption capabilities and encryption needs to be enabled before the drive is used
Current FDE drive technology available:
DS8700 DS8800
300 GB/15,000 RPM 146 GB/15,000 RPM
450 GB/15,000 RPM 300 GB/15,000 RPM
450 GB/10,000 RPM
600 GB/10,000 RPM
900 GB/10,000 RPM
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation133
DS8000 and TKLM Working Together
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation134
Copy Services
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation135
IBM Copy Services Timeline
1990 1992 1994 1996 1998 2000 2002 2004 2006
3990/3390 RVA ESS/Shark DS6000/8000
XRC
Async Copy
PPRC
Sync CopyMetro Mirror
Sync Copy
zOS/GM
Async Copy
Global Copy
Async Copy
FlashCopy
Global Mirror
ESS & DS6000/8000
PPRC
Sync Copy
FlashCopy
V1
PPRC/XD
Async Copy
XRC
Async Copy
FlashCopy
V2
Base technologies for
Global Mirror
PPRC
Sync Copy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation136
DS8000 Copy ServicesPe
er T
o Pe
er R
emot
e C
opy
(con
tinuo
us c
opy)
Metro Mirror (synchronous)
For local distance HA
Global Copy (asynchronous copy utility)
For data migration, never for HA or DR
Global Mirror (asynchronous)
Long distance for HA or DR
Consistency Group
Snapshots for backups, OS and application upgrade role-backs, BI and queries
FlashCopy
Poin
t In
Tim
e C
opy
Reduced space snapshots for backups
Space Efficient FlashCopy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation137
FlashCopy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation138
FlashCopy (Point-in-Time) Replication
FlashCopy
– Instant, time zero (T0) copy of what a set of volumes looked like at the one particular instant in time• Any changes after the point in time are not
replicated unless another point in time copy is requested
• Not continuous – can be repeated as often as necessary
– Logical copy – physical copy (done in background) can be accomplished later
– The target volumes are immediately available for read and write activity
– Target volumes are on the same storage subsystem as the source volumes
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation139
Why Use FlashCopy?
Benefits of using FlashCopy for Backup
–Reduced Backup window•No application impact•No pre-sync needed•No prescheduling
–Reduced restore/recovery time
–Protection from corruption (logical errors)•Operational errors•Application, middleware, operating system, hardware errors
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation140
Why Use FlashCopy for Backup?
Tape Backup–“No copy” option allows target to be immediately brought online to backup
server, dumped to tape, and released after dump is complete
Disk Backup–Incremental option makes future backups efficient–Multi-target allows check pointing and versioning
• Nightly mail DB restore point• Nightly pre-batch restore point• Nightly market results
Test backup–Pre-testing restore point
D/R backup–Maintain consistent copy during resynchronization–Create consistent copy before replication
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation141
Why Use FlashCopy to Create Extra Copies?
Immediate use of production data without impact to production servers Extra copies for business processes:
– Parallel processing • Seismic
– Analysis• Data warehousing, data mining, business intelligence
– Reporting– Clones/instances
• For internal use• For business partners and vendors
– Information Lifecycle Management (ILM)• Different RAID level• Different drive size or type
Extra copy for IT processes:– Test– Development– Support
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation142
Accessing the FlashCopy Target
Target can be brought online and made available for read and write as soon as FlashCopy relationship is established
– Before physical copying is complete
– Optionally can prohibit write access to target• If target is changed, this alters the Point-in-Time image
– Immediately begin tape backup, testing remote mirroring etc. from Point-in-Time image
Target can be brought online to a different server– Reduce production server impact
– If brought on to same server, need to change duplicate volume ID or volume table of contents information
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation143
FlashCopy Copy Options
Background Copy– Complete copy of volume– Relationship is terminated when copy is
complete• Persistent relationship may be specified
– Optional– Relationship remains after copy is complete
No Copy– ‘Copy on write’ as needed to preserve PiT
image– Read from source as necessary– ‘No Copy’ may be changed to ‘Background
Copy’
Track ID Checklist
TargetSource
Target
Track ID Checklist
Source
FlashCopy NoCopy
‘Copy on Write’
Write 2
34
1
FlashCopy Background Copy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation144
Why Use FlashCopy with Background Copy?
If there is a heavy change rate or a long time between FlashCopies
– Background copy will be more efficient than Copy-on-Write for full volume copy
If there is concern about the impact of FlashCopy target access on source volume, and there is a window of low activity
– FlashCopy with background copy may be completed during the window
– After copy is complete, there is no impact to source volume when target is accessed
If there is a requirement to make a FlashCopy of the FlashCopy target
– A FlashCopy target cannot be the source of another FlashCopy at the same time (FlashCopy ‘cascading’ is not supported)
– e.g. Backup copy for test
Track ID Checklist
TargetSource
FlashCopy Background Copy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation145
Why Use FlashCopy with NoCopy? When the FlashCopy will be used for a tape backup
– Relatively short-lived– Physical disk copy is not needed
Reduce FlashCopy target disk space requirements– Allows use of Track Space Efficient FlashCopy target volumes
Reduce FlashCopy workload– When there are frequent checkpoints and/or a low change rate– When the FlashCopy will have a short life
Delay copy until a period of low activity – Or until a copy of the target is needed– The FlashCopy PiT can be established at any time, and the background copy transition
can be scheduled for a period of low activity or when a physical copy is needed.
TargetTrack ID Checklist
Source
FlashCopy NoCopy ‘Copy on Write’
Write 2
35
1
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation146
FlashCopy Options
Incremental FlashCopy (refreshes target volume)
Persistent FlashCopy
Data Set FlashCopy
Multiple relationships
Consistency group FlashCopy
Inband commands over remote mirror link
FlashCopy Space Efficient
Copy and NoCopy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation147
IBM Tivoli Storage FlashCopy Manager
Application System
ApplicationData
Local FlashCopy Versions
FlashCopy Backup
FlashCopy Restore
FlashCopy Manager
Storage Manager 6
With Optional
TSM Backup
Integration
SVCXIVDS8000DS 3/4/5*
For IBM Storage
Online, near instant FlashCopy backups with minimal performance impact
High performance, near instant restore capability
Integrated with IBM Storage Hardware
Simplified deployment
Online, near instant FlashCopy backups with minimal performance impact
High performance, near instant restore capability
Integrated with IBM Storage Hardware
Simplified deployment
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation148
FlashCopy Manager - Key Functionality
Exchange FlashCopy Restore* of Exchange
storage groups File copy restore of a storage group or
database from a mounted FlashCopy image
– Restore into a Recovery Storage Group, alternate storage group, or relocated storage group
Individual mailbox or mail item restore from a FlashCopy backup
SQL FlashCopy Restore* of a full database
backup File copy restore of a full database from
a mounted FlashCopy image – To an alternate database name– To an alternate location
* As supported by Volume Shadow Copy Services (VSS) provider
DB2 FlashCopy Restore of a Full database FlashCopy restore of one or more
database partitions in the case of a multi-partition database
Oracle
FlashCopy restore of a Full database
SAP
FlashCopy Restore of a Full database
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation149
Tivoli Storage FlashCopy Manager Benefits
Simplifies deployment and management of advanced, application-aware data protection for IBM storage systems
Improves backup and recovery times from hours to a few minutes
Improves backup and recovery times from hours to a few minutes
Leverages existing investments in IBM storage systems and as an option can provide tight integration with Tivoli Storage Manager
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation150
Remote Replication
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation151
Metro Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation
Synchronous replication
–Continuous
–Target access requires suspension of replication
Minimal RPO
–Designed for 0 data loss
2-site, volume-based hardware replication
Metro Distances
–300 km standard support
–Additional distance via RPQ
Application response time impacted by copy latency
–1 ms per 100km roundtrip
–Utilizes pre-deposit writes to reduce protocol exchange to a single round-trip
System z, open systems and System i volume replication in one or multiple consistency groups
152
Metro Mirror Overview
Metro Mirror
Metro Distances
Local Site
Remote Site
Metro Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation153
Metro Mirror Configurations Metro Mirror between 2 DS8000s in the same
physical location Provides high or continuous availability
– Clustering– Protection from hardware failure
Also may be used for planned outages– Maintenance
Metro Mirror between 2 DS8000s in a metro region Protects against local datacenter disaster 300km standard support Additional distance via RPQ
Metro Mirror within a single DS8000 Fibrechannel ‘loopback’ Typically used only for testing
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation154
IBM Metro Mirror Normal Operation
21. Write to local
2. Write copied to remote (placed in cache + persistent memory)
3. Write complete from remote to local
4. Write complete to application
1
4
Remote DS8000Local DS8000
Application Server Metro Mirror•Synchronous•Minimizes data loss•Application response time affected by remote mirroring •Metro Distance (300 km standard)
3
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation155
Global Copy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation156
IBM Global Copy Overview 2-site, 2-volume hardware replication
Asynchronous replication without consistency
– Continuous replication
RPO depends on procedures and consistency creation interval
Unlimited Global Distances
Minimal application impact
System z, open systems and System i volume replication in same or different consistency groups
Target access requires suspension of replication
Global Copy
Global Distances
Local Site
Remote Site
Global Copy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation157
IBM Global Copy Normal Operation
41. 1. Write to local2. 2. Track ID added to checklist of
tracks to be copied to secondary3. 3. Write complete to application4. 4. At a later time, write copied to the
remote5. 5. Write complete sent from remote
to local6. 6. Track ID removed from checklist
1
3
Remote DS8000Local DS8000
Application Server
5
Global Copy• Asynchronous• Minimal application impact• Efficient use of bandwidth• ‘Fuzzy’ data -- requires
additional procedures to create consistency
• Global Distance
2
6
Global Copy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation158
Global Copy – Creating Consistency
There are 3 approaches to creating consistency with Global Copy:
– Stop updates at source (quiesce application)• Wait for all data to be replicated to the target• Suspend, or use FlashCopy to create an additional consistent copy
– Transition to synchronous mode and suspend as soon as data is consistent (aka ‘duplex’) at remote• Optionally use FlashCopy to create an additional consistent copy
and then resume mirroring
– Create a consistent FlashCopy at the local site and then replicate it with Global Copy
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation159
Global Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation160
What is Global Mirror
Global Copy
+ FlashCopy
+ Automated consistency creation
---------------------------------------------
= Global Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation161
IBM Global Mirror Overview2-site, volume-based hardware replication
– 3-volume design– Space-Efficient FlashCopy targets may be used
Asynchronous with consistency– Global Copy + FlashCopy + built-in automation to
create consistency
Near-continuous replication
Lowest Recovery Point Objective (RPO)– Designed to be as low as 3-5 seconds– Depends on bandwidth, distance, user specification
Unlimited Global Distances
Minimal application response time impact
Single consistency group can include– System z + open systems + System i
Multiple consistency groups
Global Mirror
Global Distances
Local Site
Remote Site
Flash Copy
Global Copy
Global Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation162
IBM Global Mirror Normal Operation
6
1.1. Write to local2.2. Write complete to application3.3. Autonomically or on a user-specified interval,
consistency group formed on local4.4. CG sent to remote via Global Copy (drain) 5.5. After all consistent data for CG is received at
remote, FlashCopy with 2-phase commit (this can be a FlashCopy SE target volume)
6.6. Consistency complete to local7.7. Tracks with changes (after CG) are copied to
remote via Global Copy, and FlashCopy Copy-on-Write preserves consistent image
Application Server
4 (CG only)
Global Mirror• Asynchronous with consistency• Minimizes application impact• Uses bandwidth efficiently• Consistent data -- RPO/currency
depends on workload, bandwidth & user• Global Distance
Global CopyFlash
Copy
5
3
7 (changes after CG)
Site 1 Site 2
1
2
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation163
Metro/Global Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation164
What is Metro/Global Mirror
Metro Mirror
+ Global Mirror
---------------------------------------------
= Metro/Global Mirror
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation165
IBM Metro/Global Mirror Overview
3-site, volume-based hardware replication
– 4-volume design (Global Mirror FlashCopy target may be Space Efficient)
Synchronous (Metro Mirror) + Asynchronous (Global Mirror)– Continuous + near-continuous replication
– Cascaded
Metro Distance + Global Distance RPO as low as 0 at intermediate or remote for local failure RPO as low as 3-5 seconds at remote for failure of both local and intermediate Application response time impacted only by distance between local & intermediate Fast resynchronization of sites after failures and recoveries Single consistency group may include open systems, System z & System i volumes
Global Mirror
Global DistanceIntermediate Site Remote Site
Metro Mirror
Metro DistanceLocal Site
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation166
Metro Mirror
Metro/Global Mirror Introduction
Application Server
Local DS8000 Intermediate DS8000 Remote DS8000
Metro Mirror
Synchronous
Minimizes data loss – RPO as low as 0
Application response time affected by remote mirroring
Metro Distance (300 km standard)
Global Mirror
Asynchronous
Minimizes application impact
Consistent data – RPO as low as 3-5 seconds depending on workload & bandwidth
Global Distance
Global MirrorFlashCopy
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation167
Metro/Global Mirror Normal Operation
Application Server
Local DS8000 Intermediate DS8000 Remote DS8000
1. 1. Write to local DS80002. 2. Copy to intermediate DS8000 (Metro Mirror)3. 3. Copy complete to local from intermediate4. 4. Write complete from local to applicationOn user-specified interval or autonomically (asynchronously) 5. 5. Global Mirror consistency group formed on intermediate,
sent to remote, and committed on FlashCopies6. 6. GM consistency complete from remote to intermediate7. 7. GM consistency complete from intermediate to local
(allows for incremental resynch from local to remote)1
2
3
4
5
67
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation168
z/OS Global Mirror (XRC)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation169
z/OS Global Mirror (XRC) Overview
Remote Local Application Server
z/OS Global Mirror• Asynchronous with consistency• Low application impact
• Potential host impact due to data storage in cache can be managed
• Consistent data – using timestamping• Global Distance• System z volumes only
Pri
JrnlSec
System Data Mover Server (SDM)
z/OS G
lobal Mirr
or
FICON channels (e
xtended)
FICON channels (local)
FICON Channels
(Extended)
FICON Channels
(Extended)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation170
z/OS Global Mirror (XRC) Normal Operation
6
12
Remote
Local Application Server
z/O
S Glo
bal M
irror
Jrnl2nd
5
System Data Mover Server (SDM)
1. Write timestamped by z/OS server
2. Write to local (timestamp retained)
3. Write stored in cache on local (with timestamp)
4. Write complete to application
5. SDM reads updates to server (with timestamp)
6. SDM forms Consistency Groups (CG) using timestamps
7. SDM writes to Journal data set on remote
8. SDM writes to secondary volumes
9. SDM updates control dataset that copy is complete.• Optionally an additional FlashCopy may be created from the
secondary
3
4
7 8
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation171
DS8000 Performance
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation172
SPC-2 Benchmark
DS8800 host adapter performance measured using 8 Gb/second host adapterDS8300 and DS8700 host adapter performance measured using 4 Gb/second host adapter
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation173
Maximum Throughput Benchmarks
DS8800 host adapter performance measured using 8 Gb/second host adapterDS8300 and DS8700 host adapter performance measured using 4 Gb/second host adapter
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation174
Host Adapter Performance
DS8800 host adapter performance measured using 8 Gb/second host adapterDS8300 and DS8700 host adapter performance measured using 4 Gb/second host adapter
DS8800/DS8700/DS8300 Open HA 4KB IOPS Performance
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation175
Host Adapter Performance
DS8800 host adapter performance measured using 8 Gb/second host adapterDS8300 and DS8700 host adapter performance measured using 4 Gb/second host adapter
DS8800/DS8700/DS8300 Open HA 64KB Bandwidth Performance
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation176
Open Host Adapter IOPS Performance8 Gbps versus 4 Gbps
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation177
Open Host Adapter Bandwidth Performance8 Gbps versus 4 Gbps
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation178
Device Adapter PerformanceDS8800/DS8700/DS8300 Device Adapter with SSDs, 4KB Random I/O
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation179
Device Adapter PerformanceDS8800/DS8700/DS8300 Device Adapter with SSDs, 64KB Sequential I/O
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation180
Logical Configuration
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation181
Tools to Manage DS8000
System Storage Productivity Center (SSPC)
IBM Tivoli Storage Productivity Center (TPC)– TPC Basic Edition
– TPC Disk
– TPC Standard Edition
– TPC Replication
DS Storage Manager graphical user interface
Command Line Interface (DSCLI)
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation182
System Storage Productivity Center (SSPC)
The SSPC is a hardware appliance coupled with pre-installed management software
SSPC is 1U rack-mounted System x 64-bit server, Windows Server 2008
SSPC is optional and provides a convenient platform for centralized management– SSPC pre-installed with TPC-Basic Edition
– SSPC easily upgraded to TPC-Standard Edition and TPC-Replication (activated by license key) as both come pre-installed
– TPC-Basic Edition, at a minimum, is recommended. SSPC is a convenient solution that provides server platform and pre-installed software
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation183
SSPC Architecture
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation184
Remote Support and Management Interfaces
Corporate Network
Corporate Network
Primary HMC
Secondary HMC
(optional)
2 to 4 Encryption
Key Servers DS8000 Private Network
Firewall
InternetInternet
IBM Support Center
Phone Line
Optional secure VPN
Storage Admin
SSPC Or
TPC
Analo
g te
leph
one
line
Remote support- Phone home- Remote technical support- Using analog modem or
secure Internet VPN connection
Storage Admin- Maintain storage configuration- Establish replication - Receive alert messages via email- Review system performance
data - Uses web browser, DSCLI, TPC
Encryption Key Servers- Provides key management for encryption of data at rest
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation185
IBM Tivoli Storage Productivity Center
TPC is a family of products– TPC Basic Edition
• Discovery and configuration• Event and error logging• Launch element managers• Provisioning• Asset and capacity reporting
– TPC Disk• Performance monitoring and management• Alerting• Advanced configuration and allocation
– TPC Data• Enterprise reporting and management of storage utilization and file systems• Provides capacity management and automated storage provisioning
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation186
IBM Tivoli Storage Productivity Center
– TPC Standard Edition• Bundles functions of TPC-Basic Edition, TPC-Disk and TPC-Data
– TPC Replication• Supports 2 or 3 site replication• Automates administration, configuration and recovery of
– Global Mirror– Metro Mirror– Metro/Global Mirror– FlashCopy– Open HyperSwap and HyperSwap on z/OS
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation187
DS Storage Manager
The DS Storage Manager GUI is a web based application to perform storage administration functions on the DS8000
Access the DS Storage Manager GUI from– System Storage Productivity Center (SSPC)
– Remote desktop to the SSPC
– TPC on a workstation connected to the HMC
– From a web browser connected to SSPC or TPC
Key functions include– Monitoring and status
– Creating arrays, extent pools, volumes and hosts
– Defining users (LDAP supported)
– Administrating FlashCopy and replication services
– Configuring encryption
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation188
DSGUI Welcome Screen
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation189
DSGUI Internal Storage Screen
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation190
DSCLI
The DS Command-Line Interface (DSCLI) is a full function CLI and supports scripting and basic automation
DSCLI can be used to– Create and maintain authorized users
– Configure hosts, arrays, extent pools and ports
– Install activation license keys
– Define logical volumes
– Manage copy services such as FlashCopy and replication
– Establish and maintain encryption configuration
Three ways to use the DSCLI– Single-shot
– Interactive
– Scripted
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation191
CLI Examples
setioport -topology scsi-fcp I0103
setioport -topology ficon I0230
mkextpool -rankgrp 1 -stgtype ckd p07
mkextpool -rankgrp 0 -stgtype fb p08
mkarray -raidtype 5 -arsite S1
mkrank -array A0 -stgtype ckd -extpool P0
mkrank -array A1 -stgtype fb -extpool P1
mklcu -qty 1 -id 00 -ss 5000
mklcu -qty 1 -id 01 -ss 5100
mkckdvol -extpool p00 -cap 10017 5012-5018
mkckdvol -extpool p01 -cap 10017 5100-51FF
Define I/O port as FICON or FCP
Create an extent pool
Assign a rank to an extent pool
Make S/390 logical control unit
Define groups of 3390-9 volumes
Return
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation192
Notices, Disclaimer and Trademarks
IBM, the IBM logo and ibm.com are registered trademarks, and other company, product or service names may be trademarks or service marks of International
Business Machines Corporation in the United States, other countries, or both. A current list of IBM trademarks is available on the Web at “Copyright and trademark
information” at www.ibm.com/legal/copytrade.shtml
Adobe, the Adobe logo, PostScript, the PostScript logo, Cell Broadband Engine, Intel, the Intel logo, Intel Inside, the Intel Inside logo, Intel Centrino, the Intel
Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, IT Infrastructure Library, ITIL, Java and all Java-based trademarks, Linux, Microsoft, Windows,
Windows NT, the Windows logo, and UNIX are trademarks or service marks of others as described under “Special attributions” at:
http://www.ibm.com/legal/copytrade.shtml#section-special
Other company, product and service names may be trademarks or service marks of others.
References in this publication to IBM products or services do not imply that IBM intends to make them available in all countries in which IBM operates.
IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and conditions of the agreements (e.g., IBM Customer
Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. IBM is not responsible for the
performance or interoperability of any non-IBM products discussed herein. The performance data contained herein was obtained in a controlled, isolated
environment. Actual results that may be obtained in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a specific
situation, there is no guarantee that the same or similar results will be obtained elsewhere. Statements regarding IBM’s future direction and intent are subject to
change or withdraw without notice, and represent goals and objectives only. The provision of the information contained herein is not intended to, and does not grant
any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.
No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation. Product data has been reviewed for
accuracy as of the date of initial publication. Product data is subject to change without notice. This information may include technical inaccuracies or typographical
errors. IBM may make improvements and/or changes in the product(s) and/or programs(s) at any time without notice. References in this document to IBM products,
programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does
business. THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS“ WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IBM
EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT.
Copyright © 2011 by International Business Machines Corporation.
Advanced Technical Skills (ATS) North America
© 2011 IBM Corporation193