When running Linux on IBM System z When running Linux on IBM System z makes sense...makes sense...Xmakes sense...makes sense...Xcents
Velocity Software Inc. / Hill Gang Velocity Software Inc. / Hill Gang –– July 14thJuly 14th
Len DiegelLen Diegel
L @ l it ftL @ l it [email protected]@velocitysoftware.com
Agenda
• Today’s “legacy system”! Times have changed.h i ff i i ll i• How changes in IT are affecting your installation.
– IT Trends & Directions– TCO Considerations
• Measure, Track, and Report - Why bother?• Results – What should you expect?• Tools of the trade: The good, the bad, & the sad
Background • This session is based on actual customer experience & p
history as it relates to migrations from x86 (commodity) server “farms” to Linux on z.Th i d bl th t d l th• The issues and problems they encountered along the way.
• Their efforts to resolve those issues and problems.• How they adapted overcame and survived ! !• How they adapted, overcame, and . . .survived ! !
Definition: “Legacy System”
“A legacy system is an old computer system or application program that continues to be used because the user (typically an organization) does not want to replace or redesign it.”
en.wikipedia.org/wiki/en.wikipedia.org/wiki/
IT in 2010 has a new set of issues.
Unfortunately, the “legacy systems” of the 90s are creating more issues to deal with than thecreating more issues to deal with than the mainframe of the 70s. A nightmare to manage & administer. Limited to horizontal capacity increases. (see below) Increasingly expensive to maintain.
Today’s legacy systems are the byproduct of the Today’s legacy systems are the byproduct of the “add another rack” IT generation of the 90’s“add another rack” IT generation of the 90’sadd another rack IT generation of the 90 s . . .add another rack IT generation of the 90 s . . .
Challenges to the IT Professional . .
Environmentals –Space, Heat, Power, “Green” Efforts
AdministrativeMaintenance, Upgrades, Charge Back, Capacity
Planning Performance DR Data Security and morePlanning, Performance, DR, Data Security, and more Internal IT PressuresMigrations, Consolidations, Compliance Issues, g , , p ,
Maximizing Resources, Flexible Capacity Industry Strategies & DirectionsCloud / Storm / SUN-down / Sky-BLUE/ Cloudburst
What everyone wants from IT . .
Secure, high speed access to data any time from anydata any time, from any
device, and from any where.
Even from a dive boat in fTurks Caicos working with Neale on breakfast for this
eventevent . .
What IT Execs are thinking:
Response from 100 IT executives at companies using Linux operatingsystem on IBM mainframes.
93% of respondents projected that their use of IBM's IFL (Integrated93% of respondents projected that their use of IBM s IFL (IntegratedFacility for Linux) would increase or remain steady.
The two main reasons:
1. Advantage of computing capacity available on their mainframe'sand IFLs.
22 LiLi thth i fi f ldld bb tt ff tiff ti thth thth2.2. LinuxLinux onon thethe mainframemainframe wouldwould bebe moremore costcost--effectiveeffective thanthan otherotherplatformsplatforms..
* http://finance.yahoo.com/news/Survey-Predicts-Continued-prnews-15547427.html?.v=1
What is the greatest problem with your primary data center ?
“P“Power and Cooling will be a top 3 issue with all CIOs in the next 6 “Power and Cooling will be a top 3 issue with all CIOs in the next 6 to 12 months.” Michael Bell to 12 months.” Michael Bell –– Gartner GroupGartner Group
$250
$300
Installed Base(M Units)
Spending(US$B)
40
45
50Power and cooling costs x8Server mgmt and admin costs x4New server spending
$100
$150
$200
15
20
25
30
35
$0
$50
0
5
10
15
Source: IDC, Virtualization 2.0: The Next Phase in Customer Adoption, Doc #204904, Dec 2006
IBM’s Global CEO Study• Over 1500 CEOs were interviewed across all geosOver 1500 CEOs were interviewed across all geos.• Most intensive study conducted in this space.• More than 80% see an increase in complexity complexity as their p yp y
number one IT issue. Fewer than 50% say they are ready.• Challenges differ from region to region:
US / G R l i– US / Government Regulations– Japan / Power shift to emerging markets– China / “Thinking Global” as opposed to closed society.
• Conclusion: “No matter what the industry, those that can manage and react to complexitycomplexity will have the advantage over those that can’t ”over those that can t.
IBM’s Mainframe Cloud Agenda
• Focus on Mainframe Strengths– Extend System z’s massive virtualization capabilities
• Leverage System z efficiencies• Offer System z (mainframe) customers value beyond that
i th l tf / l ti d d iseen via other platforms / solutions and doing so transparently to the broader community. (90 Day Free Trials on IFLs)
“Keeping your head above the clouds” STG Tech Conference 09Keeping your head above the clouds STG Tech Conference 09
IBM’s Cloud Website: http://www.ibm.com/ibm/cloud/
System z for Cloud
“As the IT world prepares for cost“As the IT world prepares for cost--cutting in 2009 servercutting in 2009 server“As the IT world prepares for cost“As the IT world prepares for cost--cutting in 2009 servercutting in 2009 serverAs the IT world prepares for costAs the IT world prepares for cost--cutting in 2009, server cutting in 2009, server virtualization seems like one of the few technologies bound for virtualization seems like one of the few technologies bound for continued growth and success. . . . continued growth and success. . . . The IBM System z may be the The IBM System z may be the b ib i l i li i l f il bl d ”l i li i l f il bl d ”
As the IT world prepares for costAs the IT world prepares for cost--cutting in 2009, server cutting in 2009, server virtualization seems like one of the few technologies bound for virtualization seems like one of the few technologies bound for continued growth and success. . . . continued growth and success. . . . The IBM System z may be the The IBM System z may be the b ib i l i li i l f il bl d ”l i li i l f il bl d ”best enterprisebest enterprise--class virtualization platform available today . . .”class virtualization platform available today . . .”best enterprisebest enterprise--class virtualization platform available today . . .”class virtualization platform available today . . .”
Enterprise Strategy Group – January 2009 Report “IBM System z: The Enterprise Server Virtualization Platform.”
IBM’s Project “BIG Green”• Consolidation of several thousand Linux servers to 30 IBM• Consolidation of several thousand Linux servers to 30 IBM
z10s. (may end up being fewer than 20..)• 80% Reduction in Energy gy• 85% Reduction in Floor Space
– 1,600 for z10s vs 11,600 square feet for Distributed• Categories as a % of Gross Savings:
FacitilitesFacitilitesConnectLaborSoftwareHW MaintHW Purch
Internal IT Study for TCO
• Determine consolidation ratio for given workloads (Linux, Windows, etc.) 20
Server Migration : x86 vs. zLinux
• Estimate annual cost to operate virtualized servers for 3-5 years.C hi d l 10
15
0
• Compare this to stand-alone provisioning or public Cloud offerings like Amazon
• N< PPR*PUR*CPF supports 0
5
10 x86/otherVirtual/IFL
• N< PPR*PUR*CPF supports that the more servers you can consolidate to virtual, the lower your cost.
01stQtr
2ndQtr
3rdQtr
4thQtr
Consolidation MathConsolidation Math
N= # of virtual servers / Proc Perf Ratio / Proc Util Ratio / Cores per FrameN= # of virtual servers / Proc Perf Ratio / Proc Util Ratio / Cores per Frame
Response Time Comparison100
nds
708090
100
me
in S
econ
40506070
zVMVMware
spon
se T
im
10203040 VMware
N b f Vi t l M hi
Res
010
10 20 30 40 50 60 70 80 90
Number of Virtual Machines
Throughput Comparison160
120
140
160
60
80
100zVMVMware
tal T
PS
20
40
60 VMware
N b f Vi t l M hi
Tot
010 20 30 40 50 60 70 80 90
Number of Virtual Machines
But keep in mind, not everything was meant to run on z . .
Cost Per Image for Linux Serversin
$k
350
400
oad
Imag
e i
200
250
300S/A ServersPublic Virt.P i t Vi t
Per W
orkl
o
100
150
200 Private Virt.zLinux AddzLinux New
Cos
t P
0
50
IT Solutions
Detailed Cost Breakdown - 5 yr TCO
Platform 100 x3250 100 Amazon Large EC2
VMware & Five x3950 – 8ways
7 IFLs on z10
Hardware $5 000 000 $2 800 000 $1 080 000 $2 030 000Hardware $5,000,000 $2,800,000 $1,080,000 $2,030,000
Software $21,490,000 $20,840,000 $2,380,000 $1,520,000
Facilities and Administration
$11,020,000 $4,020,000 (admin only)
$2,760,000 $1,210,000
Total Cost for 100 Images
$37,510,000 $27,740,000 $6,220,000 $4,780,000
Cost per image $375,100 $277,400 $62,200 $47,600
IBM SWG White Paper / April 2009 & IBM Internal Study of 4000 xSeries Distributed Servers
System z Rising
• More than 1800 customers are using System z for Li
18002000
Linux.• In 09, IBM added 54 new
“mainframe” customers 120014001600
mainframe customers for the purpose of Linux
• Customer experience and l h 400
600800
1000
results support the current strategy to migrate Linux applications to System z. 0
200400
2002 2004 2006 2008pp y 2002 2004 2006 2008The number of Linux IFLs sold/year
doubled between 2006 and 2008.
Unique Value of the IFLs
• Run at full MIPS even on sub-capacity models 1.72 391%
4.4 1157% IFL Value Increase• If/when you upgrade, the
IFLs move up in power at no additional charge.
4.4 1157%
81012
IFL Value Increase
1100% over G555% over z9BC
no additional charge.• IFL’s have increased in
power by roughly 55% h i
0246
G6 z80z90z89z99z9 z10
IFL Capacity%IFL Increase
over each previous generation or 1100% since G5.
IFL Value Increase
6 8009008909909 10
Machine Type
Note: IBM delivered more IFL MIPS in 1Q10 than in 2009.
What Horizontal vs. Vertical Scale means to you (x86 vs. z)
• How about adding a second engine to a car at the flip of a switch . . . without stopping.
• Increasing horse power from 180 HP to 450 HP at the flip of a switch . . . without stopping.
• Or being able to do both on the fly without• Or, being able to do both . . . on the fly . . . without stopping.
• System z allows you to increase your capacity either horizontally or vertically and it’s the only platform to achieve this flexibility.
But don’t get yourself in a “hole” . .
Some applications run better on z than others.
’ d dIt’s a good idea to understand the difference when ff
migrating to Linux on z.
What applications make a best fit?W bS h MQ S iWebSphere MQ Series DB2 Connect CICS Transaction Gateway , IMS Connect for Java SAP High Availability, High Availability,
h / d d hh / d d hWebSphere and JAVA applications developmentWebSphere Application Server (WAS), PortalDominoN t k I f t t FTP NFS DNS t
High I/O Bandwidth, High I/O Bandwidth, Security and DR Requirements Security and DR Requirements Simplifying the Support Model Simplifying the Support Model Shortening endShortening end toto end Path Reqsend Path ReqsNetwork Infrastructure, FTP, NFS, DNS etc..,
Oracle DatabaseApplications requiring top end disaster recovery model
Shortening endShortening end--toto--end Path Reqs end Path Reqs Reduction in Network Traffic Reduction in Network Traffic Collocation of Applications Collocation of Applications Consolidation of diverse loadsConsolidation of diverse loadsComServer and Communications Controller for
LinuxVirtualization and Security Services InfoSphere/Cognos
Consolidation of diverse loadsConsolidation of diverse loads
Communicate Pro (VoIP) “Selecting Projects for System z” “Selecting Projects for System z” Bill Reeder, IBM Corp.Bill Reeder, IBM Corp.
What applications make a good fit?
UDB (DB2) Informix, (IDS)
Evaluate Server ChoicesEvaluate Server ChoicesCorrect Application Availability Correct Application Availability S ti A li tiS ti A li tiApache web serving
SAMBA TIM/TAM (LDAP Services)
Supporting Applications Supporting Applications Political Issues within the Org. Political Issues within the Org. Potential Porting IssuesPotential Porting Issues
( )TSMExisting Linux Workloads
What applications are difficult to move?
ISV and IBM applications that have not yet ported their application to run on Linux on System z
Applications that by design run at VERY High sustained utilization which I will define here as >95%.
Stand-Alone single applications as the only Linux for System z applications
Applications that are to internally sensitive to try and move
This URL is a link to the IBM software running and supported to run under Linux on System z.http // 1 ibm com/ser ers/eser er/ series/os/lin /soft are htmlhttp://www-1.ibm.com/servers/eserver/zseries/os/linux/software.html
The question becomes . . . . . .
When you hit the next “bump” on the road to better performance,
do you keep buying new tires, or do you get the front end aligned ? ? ?
Responsible IT professionals should always consider
• Performance Management to ensure service levels are met.
• Capacity Planning to ensure future needs are met.• Operational Alerts that detect issues such as p
looping, exceeding disk capacity, etc., and doing so for hundreds/thousands of servers concurrently.
• Charge back and accounting information to allow your business to charge for resources consumed.
Real Life Situations
• Financial in NY Area - Crit Sit / 2 months / SRM parms wrong – SLES8 server wouldn’t initialize
• State of Montana - Crit Sit / 1 month / Linux Server• State of Montana - Crit Sit / 1 month / Linux Server Looping, 1 IFL offline. POC Failed.
• University Crit Sit - Response Time issues.• Financial in DFW Area - System spikes every 5 minutes .
. Wrong choice of monitors . .• Industrial – Decided to buy 20G of Storage rather than use y g
best practices and set the correct parms.. • Industrial #2 – “We keep adding IFLs.” (now leaving the
platform )platform . .)
Large Insurance Company• Pain Points (history)• Pain Points (history) –
– Too many servers– Space Power CoolingSpace, Power, Cooling– Inability to Allocate Resources Where Needed– New Apps - Time/Expense of Basic Trials and Testing
• Today – (strategy deployed in less than 4 months)– +700 Servers on 2 Boxes / Dynamic Capacity Options– 100,000 Active Users– Zero Production Outages Since 2005
$3M/Y i S i– $3M/Year in Savings
Some installations are smaller:
• One CEC (system)• Two IFLs (one production, one test)• 20 Linux Servers replaced, more planned• Proof of Concept – 90 Days (free trial – including zVPS)• Savings – space, power, admin, etc.
Typical QuotesTypical Quotes• "My TSM debugging could not have been done without Velocity.
IBM’s DB2 Level 2 support was wondering where I got some of theIBM s DB2 Level 2 support was wondering where I got some of the info ;-)” …. and that level of detail…
• “Using any other monitor would cost me at least 5% of my IFL That• Using any other monitor would cost me at least 5% of my IFL. That means one (1) IFL for every 20 that I install just to monitor my system.”
• “They gave me their monitor for free.”– What about the drag on your system?
Wh t b t d di ti th Tb t f t f th ft ?– What about dedicating more than a Tbyte of storage for the software?– What did it cost you in time to install? (Did it install?)– What does it give you for data, health checks, support, etc…?
B li IT WASN’T “F ”– Believe me, IT WASN’T “Free”.
Or, one or more of the following:
• “Time to add another IFL.”• “It’s just the way Linux runs on System z.”• “You might consider using System p instead.”• “You might consider leaving that appl. on the x86.”• “This was your idea. You might consider a new job.”• “I told you this would never work on a mainframe.”• “DRT Dead Right There ”• “DRT…. Dead Right There.”
However, accomplishing the basics does create certain challenges . . As you move g yforward, what should you expect?
Prerequisites
• A single suite of products that can deliver whatever is necessary to manage performance on z/VM and zLinux.– Keeps cost to a minimum.– Simplifies service, upgrades, education, etc.
Si l d i t f– Single vendor interface• A vendor with a history for delivering best-of-breed
products and support. p pp• A cross industry install base with references in all major
geographies.
Velocity Software Inc.• Founded in 1988 by former IBMers• Founded in 1988 by former IBMers.• Velocity products are used by hundreds of installations
worldwide; many are IBM showcase customers.• IBM uses Velocity’s products internally and makes frequent
references to our products in their Redbooks.• We have a working relationship with IBM System z labs inWe have a working relationship with IBM System z labs in
Endicott(z/VM), Poughkeepsie(zOS), and Boeblingen(zVSE).• Velocity has participated in every VM Early Support Program
since 1988 and has been a PartnerWorld member since 89since 1988 and has been a PartnerWorld member since 89.• Velocity presents research at many conferences including:
– SHARE, GSE, CMG, WAVV, and Local Linux User Groups
Velocity Software Delivers :
l• Total Data Accuracy– When necessary, identifies, prorates and corrects inaccurate data.
• Complete Data CollectionComplete Data Collection– Variety of Platforms (NT, Linux, SUN, HP, etc)– Standard Interfaces (SNMP and NETSNMP)
C l 100 1000 f– Concurrently across 100s or 1000s of servers.
• A 100% Data Capture Ratio– Best in the industry . . . Why measure incomplete data . . ?y y p
• Minimum impact on processors by monitoring agents.– Velocity’s target is .1% or less of ONE processor
z/VM – Linux Requirement Velocity Software Product All Other Vendors
Not All Performance and Systems Management Tools are Created Equal . . . .
z/VM Linux Requirement Velocity Software Product All Other Vendors
Support All Linux YES NO
Charge Back Accounting YES NO
Capacity Planning Info. YES NO
Low Cost Operation YES NO
L O h d ALow Overhead Agents YES NO
Performance Serv. (zTUNE) YES NO
Performance Education YES NOYES NO
Included in IBM Redbooks YES NO
Standard Open Linux Agent YES NO
Network Perf. Monitoring YES NO
Accounting – VM Level YES NO
Accounting Linux Process YES NOAccounting – Linux Process YES NO
Accounting – Linux Applic. YES NO
MXG and MICS Interfaces YES NO
“Trying to determine performance issues without Velocity’s suite, is like asking a doctor to give you an y , g g yexamination while wearing your clothes.”
And, the results are pretty much the same.
D O AD.O.A.
Quote from an IBM D.E. at WAVV 2010
zVPS Overview
• zVPS is Velocity’s Performance Suite. It is a collection of several components including:p g– zMON– zVWS– zVIEW– zPRO (optional)zPRO (optional)– zTUNE (optional)
zVPS Overview (cont)
• Provides real time metrics on all facets of server performance including z/VM, Linux, and other di ib d (SUN Wi d )distributed servers (SUN, Windows, etc.)
• Full function capacity planning and interfaces to planning tools such as MXG and MICSplanning tools such as MXG and MICS
• Complete and accurate charge back accounting information for Linux applications and z/VM.pp
• Operational alerts for up to 1000s of virtual servers and immediate detection of performance iissues.
zVPS Example:IBM System z
zVPS can monitor and measure the
activity of Linux on
z/VM
activity of Linux on System z in addition to distributed servers
& Distributed Servers (x86)
SUNLinux (IFL)Linux (IFL)
like SUN Solaris, Linux, HP, Windows,
and more.,
zVPS Linux
,Windows
otherNet: You can determine the level of use forNet: You can determine the level of use for otherNet: You can determine the level of use for Net: You can determine the level of use for any server in your network.any server in your network.
Data can be used for Trend Analyses
l h
90100
IFL CPU Utilization – 12 Months
607080
Prod 1
20304050 Prod 2
Test 1
01020
Jan Mar May Jul Sep NovJan Mar May Jul Sep Nov
zPROzPRO
Cl i f Li S S l i VSE CMS
Cloud Enablement Tool for Systems Administrators
• Cloning for Linux Servers, Solaris, VSE, CMS users and more.
• Provides administrators the ability to define• Provides administrators the ability to define servers based on “golden images”.
• Manages z/VM security (RACF)• Manages z/VM security (RACF)• Provides an easy-to-use, intuitive front-end to
Dirmaint via a Tab approach and syntax checkingDirmaint via a Tab approach and syntax checking for defining directory entries.
• Documentation / Command search capabilitiesDocumentation / Command search capabilities
zTUNE
• Configuration analysis to ensure best possible configuration for performance.g p
• Dozens of Automated “health checks” for both Linux and z/VM.
• Analyses of your Capacity and Performance data.• Recommendations to improve performance and to p p
optimize resources whenever it’s requested.
Velocity works as an extension of the local team.
Summary
• We are committed to providing the best z/VM performance products available.
• Our zVPS Product Suite is a complete set of Performance and• Our zVPS Product Suite is a complete set of Performance and Systems Management tools designed exclusively for z/VM and Linux on IBM System z
• We provide excellent product support services, and education.• Our install base is both global and diverse.
l i S f i h i d l d i hi b• Velocity Software is the recognized leader in this space by most of IBM’s z/VM – Linux “showcase” accounts.
Proven Performance from Velocity Software