© 2009 IBM CorporationJune 9, 2010
IBM Optim Query Tuning OfferingsOptimize Performance and Cut Costs
Saghi AmirsoleymaniSolution Architect Integrated Data [email protected]
Information Management
© 2009 IBM Corporation2
© Copyright IBM Corporation [current year]. All rights reserved.U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.
IBM, the IBM logo, ibm.com, DB2 and IBM Optim Query Tuner are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml\
Disclaimer
Information Management
© 2009 IBM Corporation3
Query Performance Causes Application Outage
Customer example – Application service drops from 3 seconds to 5 minutes – Limited performance management expertise – Took 3 days to identify the problem query– 5 day application outage– Loss of confidence in IT team– Time and money in analysts
Expert analysis revealed – Relevant statistics were not being collected – Wasted CPU resources due to default statistics collection– Index design was very poor – Rapid development practices drove untuned queries into production
Poor data
Wasted resourcesPoor design
Poor process
Information Management
© 2009 IBM Corporation4
Why is Query Tuning Such a Pain?
Development– Lack of query tuning skills– Use of frameworks that generate SQL– Inadequate testing environments to drive the workload and data scale– Disconnect between application and data groups
Production– Performance problems appear without warning– Aggregate information from multiple places– Complexity of analysis requires specialized skills– Need to consider entire workloads, not just single queries
Information Management
© 2009 IBM Corporation5
IBM Optim Query Tuner Offerings
Optimize performance– Improve customer satisfaction– Improve organizational productivity– Satisfy service level objectives– Fit workloads into maintenance
windows
Reduce costs – Increase overall system capacity– Reduce chargebacks – Simplify analysis for DBAs,
developers, designers, and others – Improve statistics collection efficiency– Reduce time to respond
Optimize performance and reduce costs
Information Management
© 2009 IBM Corporation6
Single Query and Workload Offerings
Optim Query Tuner
Tune SQL pre-production while costs and impact are low
Optim Query Workload Tuner
Optimize workload for peak performance
Development Environment Production Environment
Information Management
© 2009 IBM Corporation7
Expert-Enabled Query Tuning Out of the Box
Identify query candidates– DB2 catalog– Dynamic statement cache– Development Studio hot spots– Query or performance monitors
Facilitate analysis– Query formatting– Query annotation– Access path visualization and
annotation
Get expert tuning advice– Improve query design– Improve statistics quality– Improve database design
Information Management
© 2009 IBM Corporation8
Define or select workload Execute advisors
Streamlined Analysis
Drill into advice Validate improvement
Information Management
© 2009 IBM Corporation9
Gather High Cost Queries and Workloads
DB2
Plan table
Statement table
Catalog plan or package
Statement cache
Optim Development Studio
QMF and QMF HPO
DB2 Query Monitor
OMEGAMON XE for DB2
File, text, or exported workload
User defined category
Define or select workload
Information Management
© 2009 IBM Corporation10
Visualize Queries and Costs to Speed Analysis
Accelerate analysis, reduce downtime– Spot human errors – Identify where filtering should occur
Easily see tables, sections, join
predicates, etc.Examine table statistics and
additional information
Information Management
© 2009 IBM Corporation11
View Optimizer Transformations
Accelerate analysis, reduce downtime– Spot transformations which will occur
Information Management
© 2009 IBM Corporation12
Execute Advisors
Statistics – Get recommendations on the
best statistics to capture to influence access path selection
Query– Get recommendations
regarding how to rewrite the query for better efficiency
Index – Get recommendations on
additional indexes that can reduce database scans
Information Management
© 2009 IBM Corporation13
Visualize access path – See flow of query processing– See indexes and operations– See optimizer rationale
Assess access path stability to reduce risk of performance regression– Is the optimizer able to apply the filtering early?– Are there indexes that support an efficient path?– Do statistics allow distinction between the choices?
Analyze Access Plans
Information Management
© 2009 IBM Corporation14
Analyze Access Plans (cont.)
Interrogate the plan table to discover access plan flags
Information Management
© 2009 IBM Corporation15
Improve Statistics Quality and Collection
Provides advice on – Missing statistics– Conflicting statistics– Out-of-date statistics
Results – Accurate estimated costs – Better query performance– Less CPU consumption – Improved maintenance
window throughput
Generates RUNSTATS
control statements
Indicates conflicting and
missing statistics
Conflicting statistics
explanation“Half of access path PMRs could be resolved by statistics advisor before calling IBM support.”
– IBM Support
Information Management
© 2009 IBM Corporation16
Improve Query Design
View analysis summary Highlights relevant
components of the query
Recommendation and
rationaleFilter recommendations
by severity
Guard against errors and oversights:Further constrain query, increase index utilization, and reduce data reads
Query Advisor checks for– Missing join predicate for referential
constraint– Predicates that can be rewritten as
indexable– Stage 2 predicates that can be
rewritten as stage 1 predicates
Information Management
© 2009 IBM Corporation17
Indexing Advice to Improve Query Efficiency
Improve query efficiency– Indexing foreign keys in queries
that do not have indexes defined– Identifying index filtering and
screening– Support for index only access – Indexing to avoid sorts
Simplify use– Consolidate indexes and provide
a single recommendation – Enables what-if analysis– Provides DDL to create indexes– Run immediately or save
Test before deployment– Utilize virtual index capabilities
built into the DB2 engine
Information Management
© 2009 IBM Corporation18
Validate Improvement Capture workloads repeatedly to see tuning
effects
Visualize changes in access plan
“I was able to test a small workload in QWT. 213 dynamic statements were run and captured from cache. Before tuning the workload was 2.49 cpu seconds and 50.97 elapsed seconds. After the advisors the workload was 0.42 cpu seconds and 6.33 elapsed seconds.”
Renee Walton, UPS
Information Management
© 2009 IBM Corporation19
Version to Version Migration Saved
Challenge– Performance regression discovered across range of QMF workloads– Client had been using a “trick” to affect optimizer choices that was not
available in the new version– Expected 6-8 months delay plus additional staff costs – Delay in promised value of new database version
Solution – Optim Query Workload Tuner– Handled complexity of queries and workloads
• 30 table joins nested behind several layers of views• Column names changed in views• 100s of queries in multiple workloads
– Saved hours of time formatting, annotating, and analyzing queries – Recommended additional statistics and predicates for better performance – Reduced migration from months to weeks
Order of magnitude reduction in query workload analysis. All workloads as good as or better than prior version.
Information Management
© 2009 IBM Corporation20
Workload Tuning vs. Query Tuning
Optimization decisions are based on tradeoffs– Statistics – CPU costs vs. query savings– Indexing – query speed vs resource and transaction
Workload tuning speeds up analysis– Analyzes multiple queries at once
Workload tuning consolidates and optimizes recommendation for overall workload– Statistics recommendations– Index recommendations
Speed up analysis, optimize design, and balance resource usage
Information Management
© 2009 IBM Corporation21
Environment Capture Facilitates Collaboration
Enable environment reproduction Speed up service process
Information Management
© 2009 IBM Corporation22
Query Tuner Architecture
Windows XP or Vista
Optim Query Tuner Client
Res
trict
ed u
se D
B2
Con
nect
DB2 for z/OS, V8 or laterwith DDF enabled
DB2 stored procedures for use with Optim Query Tuner products
Catalog tablesQT tablesProfile tables
Query Tools• Formatting• Annotating• Access plan graph• Visual plan hints• Reporting• Environment captureQuery Advice• Statistics• Query• Index• Access PathWorkload Advice• Statistics• Index• Workload control
center
Information Management
© 2009 IBM Corporation23
Streamline Monitor and Respond Scenarios
Optim Query Tuner
DB2 Query Monitor
OMEGAMON XE
Information Management
© 2009 IBM Corporation24
Foster cross-role collaboration
Challenge – Disconnect between application and
data group
Solution- verbose HTML file – Overview– Recommended actions– Input sql statement– Access Plan Summary
• Transformed/optimized query• Predicates• Table access• Join operation
– Table and index catalog information: table, column and index
Released in December!
Information Management
© 2009 IBM Corporation25
Reduce Risk of Performance Regression After Migration
Run Access Plan Cost comparison report
Comparing query costs via an HTML report• Version to Version migrations of DB2
– Requires the packages are bound with EXPLAIN in the old version, and then the binds with EXPLAIN are done on the new version into new collections
• Cases where a mass-rebind is needed
Obtain Access Plan information
Filter Criteria
Information Management
© 2009 IBM Corporation26
View analysis summary based on TOTAL_COST
Identify Problematic Package Tables
SQL statement with the highest
% cost
Prevent Problems Before They Impact the Business
Tune a single query or at a
workload level
Identify Problematic SQL statement
Information Management
© 2009 IBM Corporation27
Validation Against a Production System
Accelerate development– Develop, test, and run queries,
procedures, and functions. Enhance productivity
– Analyze impact of changes– Enhance problem isolation
Be proactive– Leverage best practices for new
development– Test before deployment– Adapt ahead of impact – Minimize production slowdowns
Change database connection
Clone project
Perform analysis
Information Management
© 2009 IBM Corporation28
IBM Optim Query Tuner Offerings
Optimize performance– Improve customer satisfaction– Improve organizational productivity– Satisfy service level objectives– Fit workloads into maintenance
windows
Reduce costs – Increase overall system capacity– Reduce chargebacks – Simplify analysis for DBAs,
developers, designers, and others – Improve statistics collection efficiency– Reduce time to respond
Optimize performance and reduce costs
Information Management
© 2009 IBM Corporation29
Tuning SQL with Optim Query Tuner, Part 1: Understanding access paths
– Learn how to monitor and tune queries and workloads to improve application performance
If you are a developer, DBA, or query tuning specialist, it is critical that you understand the basics of access paths so that you can precisely tune queries and query workloads before they cause problems in your production environment. This basic understanding, coupled with the visualization and tuning advice provided by IBM® Optim® query tuning solutions, can help make you more efficient at this task. This article provides conceptual background on access paths, shows you how to read an access path graph, and walks through the access path graph to demonstrate critical information regarding access path selection. The article concludes with a sample scenario that demonstrates how to use query annotation, a feature in Optim Query Tuner that helps you with query analysis by providing vital statistical information directly in the SQL statement.
http://www.ibm.com/developerworks/data/library/techarticle/dm1006optimquerytuner1/index.html
New! Article
Information Management
© 2009 IBM Corporation30
Resources
Integrated Data Management Community– http://www.ibm.com/developerworks/spaces/optim
IBM Optim Query Workload Tuner web page– https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-
expert-zos/
SQL Tuning, Not just for hardcore DBAs anymore– http://www.ibmdmmagazinedigital.com/dmmagazine/
Redbooks at www.redbooks.ibm.com– DB2 9 for z/OS Performance Topics SG24-7473– IBM DB2 9 for z/OS: New Tools for Query Optimization SG24-7421
Information Management
© 2009 IBM Corporation32
UGF Usage of Query Workload Tuner
Their new Process to Tune Queries1.Every request of a new object (table or index) from the application
developer
2. From daily/weekly/monthly reports on costs
3. From stress test after existing applications upgrade
4. From alert coming from the Scheduling office What they liked
– Statistics Advisor – Easiness in tracing dynamic statements – What-if analysis– Query Environment Capture
Information Management
© 2009 IBM Corporation33
Daily performance reports
After query tuner activities:
AVG CPU: from 0.08 to 0.05
50000*0.03=1500 sec
Information Management
© 2009 IBM Corporation34
Achievements:SYSSH200
0
0,02
0,04
0,06
0,08
0,1
0,12
0,14
10-fe
b
11-fe
b
12-fe
b
15-fe
b
16-fe
b
17-fe
b
18-fe
b
19-fe
b
22-fe
b
23-fe
b
24-fe
b
25-fe
b
26-fe
b
1-mar
2-mar
3-mar
4-mar
5-mar
8-mar
9-mar
10-m
ar
11-m
ar
12-m
ar
15-m
ar
16-m
ar
17-m
ar
18-m
ar
19-m
ar
22-m
ar
23-m
ar
24-m
ar
25-m
ar
26-m
ar
29-m
ar
30-m
ar
31-m
ar
1-ap
r
6-ap
r
7-ap
r
8-ap
r
9-ap
r
EL
A
0
0,0002
0,0004
0,0006
0,0008
0,001
0,0012
0,0014
0,0016
0,0018
0,002
CP
U
ELAmedCPUmed
SQLLF000
0
0,02
0,04
0,06
0,08
0,1
0,12
11-fe
b
12-fe
b
15-fe
b
16-fe
b
17-fe
b
18-fe
b
19-fe
b
22-fe
b
23-fe
b
24-fe
b
25-fe
b
26-fe
b
1-mar
2-mar
3-mar
4-mar
5-mar
8-mar
9-mar
10-m
ar
11-m
ar
12-m
ar
15-m
ar
16-m
ar
17-m
ar
18-m
ar
19-m
ar
22-m
ar
23-m
ar
24-m
ar
25-m
ar
26-m
ar
29-m
ar
30-m
ar
31-m
ar
1-ap
r
6-ap
r
7-ap
r
8-ap
r
9-ap
r
EL
A
0
0,0002
0,0004
0,0006
0,0008
0,001
0,0012
0,0014
0,0016
0,0018
CP
U ELAmedCPUmed
Information Management
© 2009 IBM Corporation35
Examples at XXX of SQL Query Performance Tuning using Query Workload Tuner PeopleSoft SQL Query Example #1 - continued
– This is the report after applying the changes recommended by Query Tuner’s Tuning Advisors. STMT_ID before (5481805) and after tuning (368), Shows the average CPU per execution improvement and considering this statement is executed by PeopleSoft every 30 seconds, the CPU savings will add up