System z Performance & Capacity Management using TDSz and DB2 Analytics Accelerator: UNIPOL customer experiences
Bruna Murotti,manager of mainframe IT system environment, UNIPOL
Fabio Riva,zStack Advocate, zClient Architect, IBM Italy
Francesco Borrello, Technical Sales, IBM Italy
2
Please note
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general product direction and it should not be reliedon in making a purchasing decision.
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.
The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment.
The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
3
“After introduction of DB2 Analytics Accelerator it became possible to improve BA and BI solution on System z. One of the many possible exploitations is related to performance and capacity data analysis, a very rich context of structured Big Data to deal with.
In the past, space and database response times limitations restricted the exploitation of this solution. Both of them are now solved, having up to 192 TB of space and 96 parallel cores available with IDAA.
We'll present how it's possible to start from SMF detailed data, collect them with TDSz in a structured way and calculate on the fly performance/SLA enhanced COGNOS reports.
We'll also use SPSS BA tool to develop capacity forecasts based on historical data. Hours of computation will now became minutes, minutes will become seconds. Isn't it an innovative solution?”
System z Performance & Capacity Management using TDSzand DB2 Analytics Accelerator: UNIPOL customer experiences
Abstract
4
Agenda
Introduction – IT capacity management1
Results obtained6
Next steps7
The designed solution for UNIPOL5
Customer needs – Pain points4
Q&A - Closure8
Customer environment3
IBM Capacity Management Analytics for zEnterprise2
5
part 1:
IT Capacity Management
6
Analytics in IT = Capacity Management
Definition from ITIL V3:
– ITIL Capacity Management aims to ensure that the capacity of IT services and
the IT infrastructure is able to deliver the agreed service level targets in a cost
effective and timely manner.
– Capacity Management considers all resources required to deliver the IT service,
and plans for short, medium and long term business requirements.
Sub Processes:
– Component Capacity Management
– Service Capacity Management
– Business Capacity Management
– Capacity Management Reporting
7
Why Capacity Management is important
Helps consolidate and reduce costs
– Reduces HW and labor costs
– Reduces number of physical servers required to run workloads
– Reduce number of required licenses
Helps ensure application availability
– Are any resources overloaded? When will physical resources reach their limits?
– Have there been any significant changes in my environment between two weeks?
– Ensure supply can meet demand
– Ensure business policies are met
Helps optimize resource utilization
– Right size virtual machines
– Identify trends for workload balancing
8
Questions Capacity Management can Answer…..
System/Workload Characteristics, Performance and Trending
• How is my environment performing overall?
– Which are my most used servers/LPARs for a given resource type?
– Are there any bottlenecks in my current environment and where?
– Am I reaching capacity on resources and which resource? When will I exhaust
capacity?
– Which is my top resource consumers for a given resource type?
– Which are my least used servers/LPARs for a given resource type?
– Which are my bottom resource consumers for a given resource type?
– Do I have any outstanding abnormal behavior this week compared to last week
(other periods can be used)?
– Are my systems/workloads balanced or unbalanced?
9
Questions Capacity Management can Answer…..
System/Workload Estimation and Optimization
(optimize and keep optimized – what if)
– How many more VMs can I add to a cluster/server based on usage history?
– How much more resources do I need to add additional VMs to environment?
– How, where do I add capacity if existing systems are not enough for future growth
for optimized capacity usage?
– Where do I place new workloads? Do I really need to add more resources?
– How can I optimize the VM/LPARs placement to maximize usage and minimize
costs?
– How can I optimize the app placement to maximize usage and minimize costs?
10
part 2:
IBM Capacity Management Analytics for zEnterprise
11
Manage the complete time
horizons
System Management
Problem Identification and Resolution
Capacity Forecasting & Real-time Analysis
Historical reporting of past performance
Forecasting future requirements
Real-time transaction monitoring
Jumpstart your time to
value & eases the path to implementation.
Built on IBM’s easy of use analytics
Includes prepackaged, interactive reports
Optional services and education
A single, integrated cost
effective solution
What is IBM Capacity Management Analytics?
It’s everything necessary for the cost effective analysis of zEnterprise usage, service objectives, resource utilization, system tuning, accounting, cost recovery, and more…..
IBM Capacity Management Analytics
12
Capacity Management Analytics: understand how current system is running
System Management
Complete reporting and dashboards
capabilities so all system managers &
executives can view, interact with and
personalize it in ways that support the
unique way they analyze performance
and make decisions
13
Problem Identification and
Resolution
Delivers a top down view of zEnterprise
workloads with the ability to drill into
further detail, perform simple adhoc
analysis to get to the "why", create
system alerts or monitor performance in
near real-time to predict potential issues
before they impact the business.
Capacity Management Analytics : Understand why and how to fix it
14
Capacity Forecasting & Real-time
Analysis
Forecast future capacity to ensure the
capacity is available that the business
needs, when they need it.
Real-time scoring of transactions as they
flow through the system enabling you to
compare with forecast.
Capacity Management Analytics: Build a plan and track against it
15
Pixel perfect reportingA workspace with greater
power, intuitive navigation & cleaner look
Seamlessly shift to more advanced
analysis interaction
Communicate your analysis
using Microsoft Office
Analytics on the go with Mobile devices and disconnected interaction
Advanced Filtering
Built on IBM’s ease of use analytics solution
16
Prebuilt interactive reports and models
MIPS Used -zServer I LPAR Level
Analyze CPU usage by processor type (CP, IFL, zl lP, etc) at the mainframe/CEC
level and identify the LPARs driving the usage.
........ ....
MIPS Used - System Level (Captured vs Uncaptured)
Analyze a system's capture ratio to determine if CPU time consumed by system related
processes (uncaptured CPU time) is too high .
MIPS Used - Service Class Period Level
Analyze the workloads (service classes) driving CPU usage on a system.
Latent Demand
DcPu MIPS Used - zServer I LPAR Level w/Forecast
Analyze future CPU usage based on the results of
the SPSS predictive analytics CPU forecast model.
Determine if latent demand (hidden capacity demand) exists on a system due to the number of tasks wanting to be dispatched exceeds the number
of processors/engines online to a system.
IBM Capacity
Management Analytics
WLM (Workload Manager)
Delays by Importance Level
Analyze the types of delays impacting each WLM
importance level (highest importance to lowest importance). Is your most impor tant work
being negatively impacted by delays?
Delays by Service Class Period
Analyze the types of delays impacting each WLM service
class period. Which service class periods assigned to an importance level are being negatively impacted by delays?
Model
Solution Kit (Prebuilt interactive
reports and models)
LPAR CPU Forecast
SPSS predictive analytics model that forecasts LPAR CPU usuage
at the hour, day and month levels
CSA/ECSA/SQA/
ESQA Utilization
Analyze peak/max utilization for the common virtual storage
areas: CSA, ECSA, SQA & ESQA.
Unplanned system outages can occur when available CSA
or ECSA storage is exhausted.
........ ...
Performance Indexes
Analyze how well Workload Manager is doing with goal achievement. How often are WLM
goals being met (Pl <= 1) or missed (Pl > 1)?
Pulse2014 The Premier Cloud Conference
17
Optional: SCCM Optional: Distributed data feed
IBM CMA core architecture diagram
18
Cognos Business Intelligence
provides the range of analysis capabilities necessary for optimizing zEnterprise use by
confidently and simply compiling the information necessary to understand and manage
system activity while significantly improving the ability to identify potential issues and
pinpoint their cause.
A look under the covers: IBM Cognos BI on zEnterprise
19
A look under the covers: IBM SPSS Modeler for Linux on zEnterprise
SPSS Modeler with Scoring Adapter
can help you use predictive analytics to
forecast future requirements for zEnterprise
and ensure the capacity required is available
when the business needs it. The Scoring
Adapter provides real-time scoring of transactions as they flow through the
system enabling you to compare actual
usage with expected and identify anomalies
before they can adversely affect the system.
20
A look under the covers: Tivoli Decision Support for z/OS
Tivoli Decision Support for z/OS
enables the data collection for the solution and builds the capacity warehouse in DB2 for
z/OS that Cognos Business Intelligence and SPSS Modeler access for reporting, analysis
and predictive modeling. Tivoli Decision Support for z/OS is also able to collect capacity and
performance data for virtually all platforms that are used in business today.
Performance
Chargeback & Accounting-
Cost Recovery
Service Level Reporting
21
IBM DB2 Analytics Accelerator : Providing faster analysis of your capacity requirements!
• What does it do?
– Base forecasts off larger samples of historical SMF data to improve accuracy of predictive models
– Dramatically accelerate the analysis of your zEnterprise usage & performance data
– Significantly speed up complex queries of the large volumes of data that are being created by zEnterprise.
– Lower the cost of long-term storage of large volumes of historical SMF data with a high-performance storage saver feature
IBM DB2 Analytics Accelerator
� What is it?
• A high performance appliance that speeds analysis, enabling you to base your projections on a larger sample of historical data
22
IBM DB2 Analytics Accelerator : Query Execution Flow
DB2 for z/OS
Optimizer
IDA
A D
RD
A R
equesto
r
IBM DB2 Analytics Accelerator
Application
Application
Interface
Queries executed with DB2 Analytics Accelerator
Queries executed without DB2 Analytics Accelerator
Heartbeat (DB2 Analytics Accelerator availability and performance indicators)
Query execution run-time for
queries that cannot be or should
not be off-loaded to IDAA
SPU
CPU FPGA
Memory
SPU
CPU FPGA
Memory
SPU
CPU FPGA
Memory
SPU
CPU FPGA
Memory
SM
P H
ost
Heartbeat
Faster Answers, Faster Reports
23
IBM DB2 Analytics Accelerator : High Performance Storage Saver
Reducing the cost of high speed storage
• Time-partitioned tables where:
– only the recent partitions are used in a transactional context (frequent data changes, short running queries)
– the entire table is used for analytics (data intensive, complex
queries).
• DB2 partitions are deleted after the High Performance Storage Saver are created on the accelerator
DB2
#1
Accelerator
#1
Query from
Application
Or
Accelerator
#2
Accelerator
#3
Accelerator
#4
Accelerator
#5
Accelerator
#6
Accelerator
#7
No longer present on DB2 Storage
24
part 3:
The Customer environment
25
UNIPOL Hardware Technical Environment
2 IBM 2827-H20 (707 + 704)
Production :
o 7 GP + 3 zIIP + 6 zAAP + 1 ICF
o 384 GB Memory
o 1092 MSU – 8954 MIPS
Development :
o 4 GP + 1 zIIP + 3 zAAP + 1 ICF
o 384 GB Memory
o 664 MSU – 5409 MIPS
Appliance:
o DB2 Analytics Accelerator for z/OS
Storage MGM Configuration
o DS8870 + DS8800 MM (Sync)
o DS8800 + DS8700 GM (Async)
o 2 X (TS7720 + TS7680) Virtual + TS3500 Real
14363MIPS
14363MIPS
1756MSU
1756MSU
26
UNIPOL Hardware Technical Environment
Disaster Recovery site
IBM 2817- M15
o 7 GP processors
o 1 zIIP SE
o 5 zAAP SEs
o 1 ICF
o 165 GB Memory
Three sites configuration
27
UNIPOL Mainframe Technical Environment
z/OS version 1.12
DB2 version 10 NFM
6 Subsystems
CICS TS 4.2
50 Subsystems
9 million transactions/daily
WAS 7.0 on z/OS
11 Application Servers (2 clustered)
6.5 million threads/daily
WebSphere MQ 7.0.1
28
UNIPOL Application Environment
• Cobol cics/batch – static and dynamic
• Assembler
• JAVA (SQLJ/JDBC)
• DELPHI (ODBC) on Workstation
• Visual Basic - .NET on Workstation
29
part 4:
The Customer needs – pain points
30
• Customer needed a solution to control all resources required to deliver the IT service, and plans for short, medium and long term business requirements
• Solution should follow cost reduction directive, so the consumption of MIPS and use of storage should be reduced and kept as small as possible
• Improvement in the existing user interface should be provided, allowing intuitive navigation and easy-to-use tools
31
part 5:
The designed solution for UNIPOL
32
An overview of UNIPOL solution architecture
DB2
TDSz
SMF logs
Cognos
SPSS
Most recent data (2-8 days) <= 1TB
NNN weeksstored in HPSS
Reports
Most recent data(2-8 days)
+
Physical 16 TB(64 TB uncompressed)
33
The plan to realize the designed solution
• Phase 1: upgrade & reports•environment upgrade (TDSz new features) and setup of Cognos report environment
• Phase 2: speed up & archives:•setup and test of IDAA environment•test of IDAA queries•tables partitioning and archives with data compression
•Phase 3: ideas for new functions – to be verified and discussed•use only TDSz detailed data with IDAA aggregations•use of LOAD instead of INSERT in order to use Turboloader•SPSS for forecasting
34
part 6:
Results obtained
35
Phase 1: upgrade & reports
• TDSz: upgrade maintenance and installation of CICS feature
– maintenance update of PTS
– Installation of CICS feature
• COGNOS: setup and connection to TDSz DB2 database
– Migration of existing reports and definition of new ones
• installation of TDSz provided reports
• migration of existing reports to COGNOS
• development of new local reports to satisfy new requirements
• definition of users group in order to control access to database
– Reports scheduling and automatic distribution inside UNIPOL
• schedule of predefined reports with output in PDF format
• distribution of PDF reports to defined users
• output in PDF with graphs format or columnar data
36
Phase 2: speed up & archives 1/2
• Identification of environment to test solution
– Test environment with TDSz database or directly in production?
• Identification of TDSz queries
– Analysis on existing elapsed time/consumption data to define the list of queries
to be used for benchmarking
• TDSz queries measurements
– measurement forcing DB2 execution
• SET CURRENT QUERY ACCELERATION NONE;
– measurement forcing IDAA execution
• SET CURRENT QUERY ACCELERATION ALL;
37
Phase 2: speed up & archives 2/2
• Identification of TDSz tables for partitioning
– analysis based on TDSz queries
– list of tables to be modified
• Partitioning tables (partitioning based on time criteria)
– Alter + Reorg DB2 commands
– archiving tables
• Force the use of IDAA
– reduction of MIPS usage
– access to all data in tables
– we force the use of IDAA from the queries instead of forcing it from DB2, so DB2
administrator can globally select the databases to be under IDAA optimization
38
Queries on TDSz data: values we detected (Montpellier lab environment)
sed time Seconds CPU time from zOS
DB2+1DAA difference 082 082+10AA difference
08209 1,1 -99,9% 1.671 0,02 -99,999%
08210 1,1 -99,9% 1.660 0,02 -99,999%
08215 1 1 -99,5% 192 0,02 -99,990%
08216 3, 1 -99,8% 1.427 0,02 -99,999%
08217 1,1 -95,0% 19 0,02 -99,896%
08218 21 -99,9% 1.609 0,02 -99,999%
08216 3, 1 -99,9% 2.062 0,02 -99,999%
avera e 1.286 1,8 -99,9% 1.234 0,02 -99,998%
10000 i-----------------------::=====:::1
1000
100
10
DB209 DB210 DB215 DB216 DB217 DB218 DB216
• DB2
oDB2+1DAA
average
Pulse 2014 The Premier Cloud Conference 0
39
Space usage: compression rate we detected (Montpellier lab environment)
Using the average compression rate
5,69
A full rack model 2001
will store 273 TB
40
part 7:
Next steps
41
Next steps: ideas for the future we’re working on
• Use of IDAA v4
– performance improvements
– improvements in archive operation (automation of manual activities)
• Keep only detailed data in TDSz tables.
– All the depending data will be calculated on the fly by IDAA and only detailed data will be stored in IDAA storage
• Use of SPSS to forecast resources requests
– SPSS can forecast future capacity to ensure the capacity is available that the business needs, when they need it.
– Real-time scoring of transactions as they flow through the system will enable us to compare with forecast.
• Evaluate with TDSz labs the possibility to use LOAD function instead of INSERT
– This in order to have performance data only on IDAA
42
part 8:
Q & A
43
© Copyright IBM Corporation 2014 All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Cognos, the Cognos logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
Thanks for your attention!
44
© Copyright IBM Corporation 2012. All rights reserved.
– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
– Please update paragraph below for the particular product or family brand trademarks you mention such as WebSphere, DB2,
Maximo, Clearcase, Lotus, etc
IBM, the IBM logo, ibm.com, [IBM Brand, if trademarked], and [IBM Product, if trademarked] are trademarks or registered trademarks of
International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are
marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common
law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law
trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
f you have mentioned trademarks that are not from IBM, please update and add the following lines:
[Insert any special 3rd party trademark names/attributions here]
Other company, product, or service names may be trademarks or service marks of others.
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all
countries in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are
provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice
to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is
provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of,
or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the
effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the
applicable license agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may
have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these
materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific
sales, revenue growth or other results.
Acknowledgements and Disclaimers: