IBM Power Systems
© 2012 IBM Corporation
Getting Started with IBM® i PowerVM® Live Partition Mobility
Willy GüntherIBM Power Systems FTSS GermanyIngo DimmerIBM ATS System Storage Support Europe
June, 27nd 2012
http://w3.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD105970http://www-03.ibm.com/partnerworld/partnerinfo/src/atsmastr.nsf/WebIndex/TD105970
© 2012 IBM Corporation2
IBM Power Systems
Agenda – Getting Started with IBM i Live Partition Mobility
PowerVM® Live Partition Mobility OverviewPrerequisitesValidation and Migration Process
© 2012 IBM Corporation3
IBM Power Systems
PowerVM Edition Features
IVM, HMC, SDMC, VM
Control
IVM, HMC, SDMC, VM
ControlIVM, VM ControlManagement
Thin Provisioning
Integrated Virtualization Manager
Dynamic Logical Partitioning
PowerVM Hypervisor
N_Port ID Virtualization
Shared Storage Pools
Suspend/Resume
Live Partition Mobility
2 per server+ VIOS
Express(5765-PVX)
PowerVM Lx86
Active Memory Sharing
Shared Processor Pools
(redundant)(redundant)Virtual I/O Server
10 per core(up to 1000)
10 per core(up to 1000)Maximum VMs
Enterprise(5765-PVE)
Standard(5765-PVS)PowerVM Edition Features
Note: PowerVM upgrades are availableto higher edition offerings.
© 2012 IBM Corporation4
IBM Power Systems
PowerVM Live Partition Mobility for IBM i
PowerVM support for IBM i Live Partition Mobility allows the user to migrate an active / inactive or suspended partition to another system
– Partition mobility allows to free up server resources used by a partition e.g. for rebalancing system resource usage, server migration / consolidation or for scheduled maintenance
– Active partition mobility allows migration with no application downtime– No solution to protect against unplanned outages
© 2012 IBM Corporation5
IBM Power Systems
M
Validate environment for appropriate
resources
POWER7® System #2POWER7® System #1
StorageSubsystem
A
HMC
Hypervisor
VIOS
A
DC01
vtscsi0
vhost0
fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
LIN1
CMN01
Hypervisor
VIOS fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
MoverService
VASI
MoverService
VASI
Suspended PartitionIBM i Client 1
M M M M M MMM
PowerVM Live Partition Mobility Migration Process (1/6)
© 2012 IBM Corporation6
IBM Power Systems
MCreate shell partition on target system
POWER7® System #2POWER7® System #1
StorageSubsystem
A
HMC
Hypervisor
VIOS
A
DC01
vtscsi0
vhost0
fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
LIN1
CMN01
Hypervisor
VIOS fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
DC01
LIN1
CMN01
MoverService
VASI
MoverService
VASI
Shell PartitionSuspended Partition Shell PartitionIBM i Client 1
M M M M M MMM
PowerVM Live Partition Mobility Migration Process (2/6)
© 2012 IBM Corporation7
IBM Power Systems
M
POWER7® System #2POWER7® System #1
StorageSubsystem
A
HMC
Hypervisor
VIOS
A
DC01
vtscsi0
vhost0
fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
LIN1
CMN01
Hypervisor
VIOS fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
A
DC01
LIN1
CMN01
vtscsi0
vhost0
MoverService
VASI
MoverService
VASI
Shell PartitionSuspended Partition Shell PartitionIBM i Client 1
Create virtual SCSI devices
M M M M M MMM
PowerVM Live Partition Mobility Migration Process (3/6)
© 2012 IBM Corporation8
IBM Power Systems
M
POWER7® System #2POWER7® System #1
StorageSubsystem
A
HMC
Hypervisor
VIOS
A
DC01
vtscsi0
vhost0
fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
LIN1
CMN01
Hypervisor
VIOS fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
A
DC01
LIN1
CMN01
vtscsi0
vhost0
MoverService
VASI
MoverService
VASI
Shell PartitionSuspended Partition Shell PartitionIBM i Client 1Start migrating memory pages
M M M M M
PowerVM Live Partition Mobility Migration Process (4/6)
MM
© 2012 IBM Corporation9
IBM Power Systems
MOnce enough memory pages have been
moved, suspend the source system
POWER7® System #2POWER7® System #1
StorageSubsystem
A
HMC
Hypervisor
VIOS
A
DC01
vtscsi0
vhost0
fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
LIN1
CMN01
Hypervisor
VIOS fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
A
DC01
LIN1
CMN01
vtscsi0
vhost0
MoverService
VASI
MoverService
VASI
Shell PartitionSuspended Partition IBM i Client 1Suspended Partition
M M M M M M M
PowerVM Live Partition Mobility Migration Process (5/6)
© 2012 IBM Corporation10
IBM Power Systems
M
POWER7® System #2POWER7® System #1
StorageSubsystem
A
HMC
Hypervisor
VIOSfcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
Hypervisor
VIOS fcs0
en2(if)
VLAN
ent2SEA
ent0
ent1
A
DC01
LIN1
CMN01
vtscsi0
vhost0
MoverService
VASI
MoverService
VASI
Shell PartitionIBM i Client 1Finish the migration and remove the original LPAR
definitions
M M M M M M M
PowerVM Live Partition Mobility Migration Process (6/6)
© 2012 IBM Corporation11
IBM Power Systems
IBM i LPM Prerequisites – Summary
POWER7 tower/rack server with firmware level Ax730_087_035 or later(currently not supported for Blades and PureFlex Power compute nodes)HMC V7R7.5 or laterPowerVM Enterprise EditionVIOS 2.2.1.4 (FP25-SP2) or laterIBM i 7.1 TR4 or laterAll VIOS virtual adapters assigned as “desired”All IBM i client I/O needs to be virtualIBM i partition properties need to be set to “Restricted I/O Resources”All partition storage must be externally attached either via VSCSI or NPIV and shared between source and target VIOS(s)If using NPIV both logical WWPNs of a virtual FC server adapter need to be zoned (2nd WWPN used on destination server)VLAN ID for IBM i client to be migrated must be identical on source and target systemVIOS(s) flagged as “Mover Service Partition” (for active partition migration only)Virtual optical devices need to be deallocatedWorking RMC connection for VIOS partitions
© 2012 IBM Corporation12
IBM Power Systems
IBM i LPM Prerequisites – PowerVM Enterprise Edition
General partition mobility capability provided by PowerVM EE“IBM i Partition Mobility Capable” enabled by firmware level Ax730_087_035 or higher
© 2012 IBM Corporation13
IBM Power Systems
IBM i LPM Prerequisites – Restricted I/O Resources
IBM i partition properties need to be set to “Restricted I/O Resources”– requires IBM i partition to be deactivated
© 2012 IBM Corporation14
IBM Power Systems
Complexity– Specialized setup confined to VIOS
Resilience– Protection against single VIOS, switch port,
switch, and Ethernet adapter failure
Throughput / Scalability– Load-sharing between primary and backup SEA
• requires using more than one VLAN andtwo or more trunk adapters per VIOS
– SEA failure initiated by:• Backup SEA detects the active SEA has failed• Active SEA detects a loss of the physical link• Manual failover by putting SEA in standby mode• Active SEA cannot ping a given IP address
Notes– Can be used by AIX, IBM i, and Linux – Outside traffic may be tagged
VIO Client
VIOS
NIC
VIOS
NIC
SEA SEA
NIC
Switch
Shared Ethernet Adapter Failover
IBM i Prerequisites – Dual VIOS SEA Failover Configuration (1/4)
© 2012 IBM Corporation15
IBM Power Systems
IBM i Prerequisites – Dual VIOS SEA Failover Configuration (2/4)IBM i Client 1
Hypervisor
IBM i Client 2
PVID=99
VIOS 2VIOS 1
PVID1
CMN01(vir)
ETH01(LIND)
PVID1
CMN01(vir)
ETH01(LIND)
PVID1
Ethernet Switch
VLAN 1
Ethernet Switch
ent4(SEA)
ent2(vir)
ent3(vir)
en4(if)
ent0(phy)
ent4(SEA)
ent2(vir)
ent3(vir)
en4(if)Primary Backup
Untagged Untagged
PVID1
Untagged
ent0(phy)
ActivePassive
Control Channel
Notes:Primary and backup SEA roles are defined by the trunk priority settings for the virtual Ethernet adapters used for SEA external network access in the VIOS partition profile.Optionally link aggregation can be used for ent0.
VLAN ID needs to be identical on LPM source and destination system
© 2012 IBM Corporation16
IBM Power Systems
IBM i Prerequisites – Dual VIOS SEA Failover Configuration (3/4)mkvdev – use for creating a Shared Ethernet Adapter failover configuration on both Virtual I/O Servers
Example:$ mkvdev -sea ent0 -vadapter ent2 -default ent2 -defaultid 1-attr ha_mode=auto ctl_chan=ent3 netaddr=9.155.50.1 largesend=1ent4 Availableen4et4
$ lsdev -dev ent4 -attrattribute value description user_settablectl_chan ent3 Control Channel adapter for SEA failover TrueGvrp no Enable GARP VLAN Registration Protocol (GVRP) Trueha_mode auto High Availability Mode Truejumbo_frames no Enable Gigabit Ethernet Jumbo Frames Truelarge_receive no Enable receive TCP segment aggregation Truelargesend 1 Enable Hardware Transmit TCP Resegmentation Truenetaddr 9.155.50.1 Address to ping Truepvid 1 PVID to use for the SEA device Truepvid_adapter ent2 Default virtual adapter to use for non-VLAN-tagged packets Truereal_adapter ent0 Physical adapter associated with the SEA Truethread 1 Thread mode enabled (1) or disabled (0) Truevirt_adapters ent2 List of virtual adapters associated with the SEA (comma separated) True
Notes:The IP interface needs to be detached from the physical port before creating a SEA (chdev -dev en0 -attr state=detach)If using a HEA/IVE port for SEA, VIOS needs to be set as the “promiscuous LPAR” for exclusive HEA port usage.
SEA failover configuration parameters
Default adapter and VLANused for untagged packages
© 2012 IBM Corporation17
IBM Power Systems
IBM i Prerequisites – Dual VIOS SEA Failover Configuration (4/4)mktcpip – use for configuring the VIOS TCP/IP setup on the SEA interface
Example:$ mktcpip -hostname i7VIOS -inetaddr 9.155.50.108 -interface en4 -netmask 255.255.255.0
-gateway 9.155.50.1 -nsrvaddr 9.xxx.yyy.zzz -nsrvdomain mainz.de.ibm.com -startor:
$ cfgassistwith selecting menu item “VIOS TCP/IP Configuration”
VIOS TCP/IP Configuration
Type or select values in entry fields.Press Enter AFTER making all desired changes.
[Entry Fields]* Hostname [i7vios]* Internet ADDRESS (dotted decimal) [9.155.50.108]
Network MASK (dotted decimal) [255.255.255.0]* Network INTERFACE en4
Default Gateway (dotted decimal) [9.155.50.1]NAMESERVER
Internet ADDRESS (dotted decimal) [9.xxx.yyy.zzz]DOMAIN Name [mainz.de.ibm.com]
CableType bnc +
F1=Help F2=Refresh F3=Cancel F4=ListF5=Reset F6=Command F7=Edit F8=ImageF9=Shell F10=Exit Enter=Do
© 2012 IBM Corporation18
IBM Power Systems
IBM i LPM Prerequisites – Deallocation of Optical Devices
Virtual optical devices need to be deallocated– virtual IOP #290A reset from IBM i SST– unconfigure of virtual optical target device in VIOS (e.g. “rmdev -dev vtopt0 -ucfg”)
Note:Partition migration will not retain optical device mappings.
© 2012 IBM Corporation19
IBM Power Systems
IBM i LPM Prerequisites – VIOS Mover Service Partitions
VIOS(s) on source and destination server flagged as “Mover Service Partition”– will automatically configure a vasiX device in VIOS)
© 2012 IBM Corporation20
IBM Power Systems
IBM i LPM Prerequisites – Known HMC Code Issues
© 2012 IBM Corporation21
IBM Power Systems
IBM i LPM Prerequisites – RMC Connection
Working RMC connection for VIOS partitions required– be careful with cloned VIOS generated e.g. by FlashCopy or alt_disk_copy as the
ct_node_id needs to be uniqueProcedure to create unique ct_node_id:1. #/usr/sbin/rsct/bin/rmcctrl –k
(stop rmc)2. #/usr/sbin/rsct/bin/mknodeid –f
(force node id creation)3. #/usr/sbin/rsct/bin/rmcctrl –s
(start rmc)4. # /usr/sbin/rsct/bin/rmcctrl –p
(this will enable remote client connections to the RMC subsystem)
There are some cases when the nodeidis not regenerated, if so replace step 2 with:#/usr/sbin/rsct/install/bin/recfgct -i <new_ct_node_id>(for the new node id you can just can just modify the last 3-4 digits from /etc/ct_node_id)
© 2012 IBM Corporation22
IBM Power Systems
IBM i LPM Prerequisites – VSCSI Adapter IDs
VSCSI server adapter IDs don’t need to match between source and target VIOS (mismatch creates a warning message only)
© 2012 IBM Corporation23
IBM Power Systems
IBM i LPM Validation Task (1/2)
Selecting the IBM i partition to be migrated and choosing Operations →Mobility → Validate to check if all requirements for LPM are met before actually starting the migration
© 2012 IBM Corporation24
IBM Power Systems
IBM i LPM Validation Task (2/2)
After successful “Validate” operation the virtual storage assignments can optionally be changed and a partition migration task can be started
All possible MSP network pairs are shown and user has to select a valid combination
© 2012 IBM Corporation25
IBM Power Systems
IBM i LPM Migration Task
Starting the partition migration task …
Note:VSCSI mappings and VSCSI server adapters are automatically removed on source VIOS(s)
© 2012 IBM Corporation26
IBM Power Systems
IBM i LPM Migrated Partition on Destination Server
IBM i client partition successfully running on destination server
Note:The partition profile of the migrated LPAR is automatically removed from source server.While LPM tries to maintain the source server LPAR ID, the system serial number is different on the destination server with corresponding implications for ISV applications which base their licensing model on serial numbers.
© 2012 IBM Corporation27
IBM Power Systems
IBM i Client Partition QSYSOPR Messages for LPM Migration Process
Additional Message Information
Message ID . . . . . . : CPI09A5 Severity . . . . . . . : 00 Message type . . . . . : Information Date sent . . . . . . : 05/29/12 Time sent . . . . . . : 20:28:11
Message . . . . : Partition suspend request in progress.Cause . . . . . : A request was made to suspend the partition.
Additional Message Information
Message ID . . . . . . : CPI09A8 Severity . . . . . . . : 00 Message type . . . . . : Information Date sent . . . . . . : 05/29/12 Time sent . . . . . . : 20:30:04
Message . . . . : Partition resumed after migration.Cause . . . . . : The partition has been migrated to another machine.
Note:
Suspend (QIBM_QWC_SUSPEND) and resume (QIBM_QWC_RESUME) exit points:http://publib.boulder.ibm.com/infocenter/iseries/v7r1m0/index.jsp?topic=%2Fapis%2Fxsuspend.htm
http://publib.boulder.ibm.com/infocenter/iseries/v7r1m0/index.jsp?topic=%2Fapis%2Fxresume.htm
© 2012 IBM Corporation28
IBM Power Systems
References
IBM i developerWorks – Live Partition Mobilityhttps://www.ibm.com/developerworks/mydeveloperworks/wikis/home?lang=en#/wiki/IBM%20i%20Technology%20Updates/page/Live%20Partition%20Mobility
PowerVM Virtualization Introduction and Configuration, SG24-7940http://www.redbooks.ibm.com/abstracts/sg247940.html?Open
PowerVM Virtualization Managing and Monitoring, SG24-7590http://www.redbooks.ibm.com/abstracts/sg247590.html?Open
How to Setup SEA Failover on DUAL VIO servershttp://www-304.ibm.com/support/docview.wss?uid=isg3T1011040
© 2012 IBM Corporation29
IBM Power Systems
Trademarks / DisclaimersTrademarks
The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: AS/400, IBM, IBM Logo, iSeries, MVS, OS/390, pSeries, PowerVM, Power Systems, VM/ESA, VSE/ESA, Websphere, xSeries, z/OS, zSeries, z/VM
The following are trademarks or registered trademarks of other companies
Lotus, Notes, and Domino are trademarks or registered trademarks of Lotus Development CorporationJava and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countriesLINUX is a registered trademark of Linux TorvaldsUNIX is a registered trademark of The Open Group in the United States and other countries.Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation.SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC.Intel is a registered trademark of Intel Corporation•PATROL for iSeries – Predict is registered trademark of BMC Software Inc.•Performance Navigator and “What if” are registered trademarks of Midrange Performance Group•All other products may be trademarks or registered trademarks of their respective companies.
Disclaimers
Copyright © 2010 by International Business Machines Corporation.No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or programs(s) at any time without notice.Unless otherwise note, the performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be obtained in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Other data may have been obtained from publicly available documents or sources. In such cases, IBM has endeavored to provide comparable measurements for systems being compared.References in this document to IBM products, programs, or services does not imply that IBM intends to make such such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM’s intellectually property rights, may be used instead. It is the user’s responsibility to evaluate and verify the operation of anynon-IBM product, program or service.The information provided in this document is distributed "AS IS" without any warranty, either express or implied. IBM EXPRESSLY DISCLAIMS any warranties of merchantability, fitness for a particular purpose OR NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-IBM products discussed herein.Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785U.S.A.