Understanding/Managing the Access Plan Changes …laadb2ug.org/Downloads/Data_Studio_Phoenix-Sept...

© 2010 IBM Corporation

Information Management

Understanding/Managing the Access Plan

Changes and Tuning Data Studio

Saghi Amirsoleymani WW Solution Architect Technical Enablement Specialist [email protected]



2 2

Performance tuning challenges across the organization

“We can’t adequately test for

peak workload since we don’t have

enough human or IT resources.”

“I need to get my business results

fast and accurate. What’s going on ?”

”I don’t have time to hone my SQL skills. I need to

focus on developing core

application functionality.”

Application Developer

“I don’t understand why our developers aren’t

focused on creating better performing SQL.”

“Performance problems seem to appear without

warning and deep technical skills are hard

to find.”

QA Manager

DBA

“It is very challenging to

aggregate performance data

across our complex data environment.”

IT Manager LOB Manager



3 3

3

Advanced Workgroup Server Edition •  Fully functional offering for small OLTP and

analytic deployments •  Includes Optim Tools •  Primarily used in department environments

within large enterprises. Also available for SMB/Midmarket deployments.

•  Limited by TB, memory, sockets and cores. •  Supports BLU, pS and DPF deployment models

Advanced Enterprise Server Edition •  For Enterprise Class OLTP and/or analytic

deployments. •  Includes Optim Tools •  Targeting full enterprise / full data center

requirements •  Fully functional •  No TB, memory, socket or core limit •  Supports BLU, pS and DPF deployment models

Workgroup Server Edition •  Capabilities for an entry level offering •  Targeting single server requirements with less

intense workloads in both the OLTP and analytic space.

•  Limited by TB, memory, sockets and cores •  No support for BLU, pureSclae or DPF

deployment models

Developer Edition

Departmental Market Enterprise Market

Enterprise Server Edition •  Capabilities for an entry level offering. •  Targeting single server, enterprise

requirements with more intense workloads in both the OLTP and analytic space.

•  No TB, memory, socket or core limit •  No support for BLU, pureScale or DPF

deployment models

Express and Express-C

DB2 10.5 Packages – Simplicity!



4

What is IBM Data Studio? §  Comprehensive tool

–  An integrated environment for managing databases and developing database applications

§  Built on the popular Eclipse framework

§  Runs on Red Hat Linux, SUSE Linux, Windows

§  Web and Eclipse components

§  Data Studio Web console: health and availability monitoring

§  NO-charge to download!

§  http://www.ibm.com/developerworks/downloads/im/data/



5 © 2013 IBM Corporation

What’s New 4.1.1 GA Aug 22

§  General Enhancements •  Navigation Improvements •  DB2 client command line processor added to SQL editor •  Simplification of running SQL scripts •  Generating scripts for object changes •  Additional debugger diagnostic support

§  DB2 for LUW Specific Enhancements •  Support for FP4

•  Support latest BLU capabilities, including alter BLU table by adding column Columnar MQT, Shadow MQT Support

§  Support Big SQL for Infosphere Big Insights (Big SQL) §  Browse and navigate Hadoop tables and show table contents §  Create Big SQL physical data models (PDM) §  Create and alter Hadoop tables, views, and constraints §  Generate and run DDL from Big SQL PDM or Hadoop table §  Execute DDL and DML via SQL editor



6

Web Console

Eclipse Client

Common Tool across Roles Improve Collaboration and Productivity

Database Administration

q  Catalog navigation q  Object management q  Privilege management

Develop

q  SQL development q  Routine development q  Deployment management q  Query tuning q  Java data access

Monitor

q  Alerts q  Dashboards

Schedule

q  JDBC SQL Scripts



7

Workspace - folder in the file system that contains the metadata about your Data Studio projects and artifacts



8

Perspectives

§ What are they? – Eclipse term

• Visual container for a set of views and editors – Data Studio has many

• Data • Database Administration • IBM Query Tuner • And more

– The first time you start Data Studio it will open the Database Administration perspective

8



9

Opening a Perspective

9

1 2

3

• In the upper corner click on the highlighted icon and select Other…

• From the menu bar select Window -> Open Perspective - > Other

Both methods will then bring up the displayed dialog box

• Select <IBM Query Tuning>à Click OK



10

Tip: Reset Perspective gets you back to normal state

Perspective Management- Resetting Perspective

Task Navigator changes

perspective



11

Web browser-like navigation bar

Object Properties / Results view

Administration Explorer • Connect to databases • Browse database objects • Organized in folders by their type

Editors area • Listing objects • Editing scripts • Running tasks

Basic Navigation

Advance • Filter • Sort



12

SQL Result View Improvement §  Show history and details of SQL result in different tabs instead of in the same pane. §  Add Filter to SQL execu=on history


12

New!



13

Basic Navigation

§  Right-mouse-click on an object brings a context-based menu will all supported tasks for that object.

Context menu for Databases

Context menu for Tables



14

Simplify Database Administration §  Manage database objects and save time

–  Simple to Complex changes –  Preserve data during destructive alters –  Manage dependencies automatically –  Generate maintenance commands/scripts

§  Compare/Migrate and reduce errors –  Compare with model or database, and synch –  Migrate data and privileges

§  Commands and Utilities –  Wizards to build commands/utilities –  Run immediately or schedule

§  Document changes for audit

"We think that Data Studio is a comprehensive, reliable, stable and complete solution for designing, administering and monitoring databases as well as tuning SQL queries for performance.”

- Metin Deniz, Onur Basturk Computer Research and Application Center Experts, Anadolu University, Turkey



15

Database Object Management : Working with Objects

§  Context menu displays different options based on the object type. Some of these options are common to all objects.

§  Most database objects support the following actions: –  Create New, Drop, Alter – Copy and Paste – Generate DDL – Analyze Impact (similar to CC’s “Show Related” function) – Manage Privileges



16

§ More naviga=onal capabili=es available

“Show” func:ons enhancement New!



17

Basic navigation : Create/Alter/Drop

Right-click Alter/Drop Right-click Creates

Create a new object



18

Database Object Management :: Editing Data § Browse and edit data in a table

–  Almost identical UI for browse and edit, main difference is read-only for browsing data

– Edited table cells are highlighted with yellow

§  Click or press Ctrl + S to commit data changes •  Results are displayed in the SQL Results view

Edit data

Filter



19

More informa:on added to Object List Editor Objects Added

Table space Managed By (LUW only)

Percentage (LUW only)

Data Pages (LUW only)

Table Sta:s:cs Time

MQT Regular Tablespace(LUW only)

Index Tablespace (LUW only)

Large Tablespace(LUW only)

Tablespace (zOS only)

Maintained by (zOS only)

Cardinality

Sta:s:cs Time

Stored Procedure Specific Name

Fenced (LUW only)

Language

Package Binder (LUW only)

Cursor (LUW only)

Number of sec:ons(LUW only)

Op:miza:on class(LUW only)

Explain snapshot (LUW only)

Collec:on ID( zOS only)

New!



20

Change Management Enhancement §  When user click DDL, all impact objects with errors and warnings will be shown §  User can select the object to view and mark “Reviewed” or alter it in property view. §  User resolve the risks and set reviewed . §  If all warning resolved, users click DDL to generate DDL.


20

New!



21

SQL Development

§ SQL Scripts Editor – Launch with New SQL Script – Integrated toolbar for easy access to common tasks – Options to specify database connection, run method, validate syntax against a

particular database, etc. – Advanced editor capabilities: Syntax highlighting and validations, Format SQL,

Toggle comments, Content tip, Content Assist, Compare/Replace with local history, etc.



22

�

DDL Genera:on Enhancement for LUW §  Create or Replace support

−  CREATE OR REPLACE ALIAS −  CREATE OR REPLACE VIEW −  CREATE OR REPLACE NICKNAME −  CREATE OR REPLACE SEQUENCE −  CREATE OR REPLACE VARIABLE −  CREATE OR REPLACE TRIGGER −  CREATE OR REPLACE MASK −  CREATE OR REPLACE PERMISSION −  CREATE OR REPLACE TYPE

§  More Alter Support −  ALTER TABLE ADD PARTITION −  ALTER BUFFERPOOL −  ALTER TABLE ALTER COLUMN DROP GENERATED −  ALTER TABLE ALTER COLUMN SET NOT HIDDEN/IMPLICITLY HIDDEN

…

§  Keep Run stats Profile if the table is dropped and recreate §  Disable Data Capture before alter, and re-‐enable it aXer for rela:ve alter op:on.

New!



23

Template-based Routine Development § Create (native, Java, SQL, PL/SQL) procedures and UDFs using predefined

templates

http://www.ibm.com/developerworks/data/library/techarticle/dm-1010devstudioroutines/index.html



24 IBM Confidential

Deploy and Run Routines

Deploy User Defined Function

Run User Defined Function

Specify run settings and generate test cases



25 IBM Confidential

Debug Routines

Launch debugger



26

Debug Routines

§ Fully integrated debugger for stored procedures, functions …

Set breakpoints

Step over, step into, … Display and edit variables

right click to change value



27

Debugger Diagnose Enhancement §  Cannot ping session manager is the most frequent problem user encountered while debugging. This problem might be caused by firewall, improper IP/network configura=on, mul= network adaptors, etc..

§  This new feature help user diagnose the root cause and give user some common solu=ons to fix simple problems by himself.


27

New!



28

CLP Client Runner Support in SQL Editor §  A[er user catalog a remote database using IBM Data Server Run=me Client §  User can run command by DB2 Client CLP in SQL Editor


28

New!



29

Query Tuning checklist Data Studio Optim Query Workload

Tuner for LUW Optim Query Workload Tuner for

z/OS

Queries from all sources • • •

Reports • • •

Query Formatter • • •

Access Plan Graph • • •

Query Statistics Advisor • • •

Query Annotation • •

Visual Plan Hint • •

Query Index Advisor • •

Query Advisor • •

Access Path Explorer

Access Path Advisor • •

Workload Access Path Advisor

Workload Statistical View Advisor

Workload Statistics Advisor •

Workload Access Path Comparison

Workload Index Advisor •

Workload Query Advisor •



30

§  Identify query candidates from numerous sources –  DB2 catalog –  Dynamic statement cache –  Data Studio hot spots –  Query or performance monitors

§  Facilitate analysis –  Query formatting –  Access path visualization and

annotation

§  Get expert tuning advice –  Improve statistics quality

Expert-Enabled Query Tuning Out of the Box - Avoid Common Mistakes



31

Streamlined Analysis with workflow assistant

Validate improvement

Define or select a workload Execute Advisors

Drill Down into advice



32

Getting Started Query Tuning: Option 1 In the Data Source Explorer, right-click a connection and select Connect, expand the connection, right-click the data server, and select Analyze and Tune > Start Tuning.



33 33

Creating a Query Tuner Project



34

Gather High Cost Queries and Workloads DB2

§ Plan table

§ Statement table

§ Catalog plan or package

§ Statement cache

Optim Performance Manager

QMF and QMF HPO

DB2 Query Monitor

OMEGAMON XE for DB2

File, text, or exported workload

User defined SQL Repository

Define or select workload

new



35

Capture from DB2 Catalog

TIP: Filter by package collection

and name, and some potential “bad” access plans

Source

Filter Rows



36

User Defined SQL Repository Define, edit, delete filter

Define Filter

Column Mapping

Limit number of returned statements

Supply table qualifier and table/view name



37

Review & Invoke Advisors

Capture

Define Filter Criteria

Tune a single query

or at a workload level



38

Mining through the Dynamic Statement Cache with IBM Data Studio



39

Sample Process to Explain Statements from the Global Cache Using Data Studio

1.  Start the collection of dynamic statement cache performance statistics -START TRACE(P) CLASS(30) IFCID(318)

2.  Use Data Studio to easily create tables DSN_STATEMENT_CACHE_TABLE to hold the execution statistics data, and explain tables DSN_FUNCTION_TABLE, DSN_STATEMENT_TABLE and PLAN_TABLE to hold access path details

or use JCL provided within program files Data Studio

3.  Generate individual EXPLAIN statements for each SQL statement in the cache

4.  Analyze and Tune

5.  Optional import the contents of capture into a spreadsheet



40

Query Tuner Workflow Assistant §  Intended to provide a logical workflow for tuning activities

– 1. Status •  Provides configuration status information

– 2. Capture •  Input or capture of SQL statements

– 3. Manage •  Not available in the “No-Charge” offering

– 4. Invoke •  Verify or modify Explain Options and Select Tuning Activities

– 5. Review •  Review results of the tuning process

– 6. Compare •  Not available in the “No-Charge offering

40



41

Gather High Cost Queries from the Statement Cache

§  Enable cache trace –  Turns on a performance trace with IFCID

0318 to collect statistics for the dynamic statement cache. You must enable this trace if you want to capture query runtime information together with SQL statements.

§  Disable cache trace –  Turns off the performance trace. Disable

this trace only if you no longer need query runtime information and cannot afford the overhead to run this trace.

2. Define or edit filter 1. Define or select workload



42

Customize Capture

42

•  Accelerate analysis, reduce downtime

collect top 100 that match the filter criteria

TIP: Query Tuner issues dynamic SQL if you

check the box we will not captured that information

3. Filtering Options



43

What to do next … Schedule capture

§ Specify the Capture profile as immediately – You can also specify the capture to be at the client or at the server.

§ You can capture information once, several times, or at intervals – Options are:

•  Immediately: Once • One-time: Once, at a specified time (in the future) •  Time period: Start now or at a future time, repeat every n minutes,

stop at a specified time •  Periodic sampling: Start now or at a future time, repeat every n

minutes, open-ended

•  Customize: More advanced options



44



45 © 2013 IBM Corporation 45

Export Captured Statements and meta data New!

Choose different formats



46

Visualize Queries and Costs to Speed Analysis

§  Accelerate analysis, reduce downtime –  Spot human errors –  Identify where filtering should occur

Easily see tables, sections, join

predicates, etc. Examine table statistics

and additional information



47

View Optimizer Transformations

§  Accelerate analysis, reduce downtime –  Spot transformations which will occur



48

§  Review and Visualize access path –  See flow of query processing –  See indexes and operations –  See optimizer rationale

§  Assess access path stability to reduce risk of performance regression –  Is the optimizer able to apply the filtering early? –  Are there indexes that support an efficient path? –  Do statistics allow distinction between the choices?

Analyze Access Plans :Avoid Common Mistakes



49

How to read an access path graph

§  An access path graph not only describes the "operational details" of query execution, it also describes how the data flows.

§  A leaf node of an access path graph is either a table node, a workfile node, or an index node that represents a data source in the query execution plan of nodes).

§  Data flows from the bottom up (as displayed in the graph) and is processed by operation nodes in the access path graph. http://www.ibm.com/developerworks/data/library/techarticle/dm1006optimquerytuner1/index.html



50 50

§  Statistics –  Get recommendations on

the best statistics to capture to influence access path selection

Execute Advisor



51

Improve Statistics Quality and Collection

§  Results –  Accurate estimated costs –  Better query performance –  Less CPU consumption –  Improved maintenance

window throughput Conflicting statistics

explanation

Indicates conflicting and

missing statistics

Generates RUNSTATS

control statements

§  Provides advice on –  Missing statistics –  Conflicting statistics –  Out-of-date statistics

“70 % of access path PMRs could be resolved by statistics advisor before calling IBM support.” – IBM Support



52

What to do next §  How are Statistics currently collected? §  Why are Histogram statistics not collected?

§  Distribution Statistics – Are you over collecting statistics?

§  simple answer – Some collect no or insufficient statistics

•  Prime reason for poor performing access paths – Do you want to collect statistics on every column and permutations of combination of

columns? •  No way!

§  Requires similar analysis of SQL as for index design – Have to include columns which you may not benefit from adding to an index – Analysis of queries labor intensive without QWT –  Iterative process analyzing explain data (as always)



53 53

Tune : Working with SQL Statements reports §  Summary report of the recommendations

–  Foster cross-role collaboration

§  Challenge –  Disconnect between application and

data group –  Disconnect between remote

employees and locations

§  Solution- –  verbose HTML file review, email, or

print – Overview –  Recommended actions –  Statistics Advisor Summary –  Access Plan Summary

•  Predicates •  Table access •  Join operation

–  Table and index catalog information: –  Access Path Advisor Summary –  Indexes – Query Tuner Report Log



54

Console Server

Data Studio web console

Repository

Health Monitor

Data Studio web console

Data Studio client

Monitored Servers

HTTP/Rest

TCP/IP and DRDA

Job Manager Web browser

Can optionally use DB2 for repository



55

Monitor Health from Data Studio client Monitor tab assists with configuration and common tasks

Health summary

Launch from a context menu



56 IBM Confidential

Data Studio Health Monitor for DB2 LUW and z/OS Monitor database health

–  Health summary –  Alert list –  Current application connections –  Data Sharing Members –  Current utilities –  z/OS system log –  Current Table spaces

Diagnose health problems –  Diagnostic dashboards –  DB2 information center

Edit monitoring settings –  Health alerts configurations –  Alerts notification –  Purge alerts interval

Administer and troubleshoot –  Configuration repository –  Console security and privileges –  Services –  Logs

Integrated with

–  Data Studio –  Optim Performance Manager



57 57

Special Registers for Temporal Tables

Advisors can take advantage of these special registers:

•  Current Temporal SYSTEM_TIME

•  Current Temporal BUSINESS_TIME

•  Current GET_ARCHIVE

New DB2 11!



58

§  Detailed Status Report

§  Display status of required and optional objects in html format.

§  It tells how to fix a configuration problem.

§  It allows user to save the report to a html file or save only the recommended DDLs to a text file.

§  Consolidated installation

§  Preferences simplification and deployment

* see appendix for configuration details

New DB2 z/OS Database Configuration Deployment Made Easy



59

Change Default SQLID and Schema

59



60

Environment Capture Facilitates Collaboration

§  Enable environment reproduction

–  DDL –  Catalog statistics –  Zparms (if DSNWZP

stored procedure available)

–  Explain tables information (PLAN_TABLE, DSN_DETCOST_TABLE, DSN_PREDICAT_TABLE) (if SQL used as input)

§  Speed up service process

DB2 z/OS



61



62

Resources

§  Integrated Data Management Community –  http://www.ibm.com/developerworks/spaces/optim

§  IBM Optim Query Workload Tuner web page –  https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-expert-zos/

§  Tuning SQL with IBM Optim Query Workload Tuner –  Part 1 Understanding Access Paths

•  http://www.ibm.com/developerworks/data/library/techarticle/dm1006optimquerytuner1/index.html –  Part 2 Tuning Individual Queries

•  http://www.ibm.com/developerworks/data/library/techarticle/dm1006optimquerytuner1/index.html –  Part 3 Workload Capturing and Tuning

•  http://www.ibm.com/developerworks/data/library/techarticle/dm-1207optimquerytuner3/index.html

§  SQL Tuning, Not just for hardcore DBAs anymore –  http://www.ibmdmmagazinedigital.com/dmmagazine/



63

Query Tuning checklist Data Studio Optim Query Workload

Tuner for LUW Optim Query Workload Tuner for

z/OS

Queries from all sources • • •

Reports • • •

Query Formatter • • •

Access Plan Graph • • •

Query Statistics Advisor • • •

Query Annotation • •

Visual Plan Hint • •

Query Index Advisor • •

Query Advisor • •

Access Path Explorer

Access Path Advisor • •

Workload Access Path Advisor

Workload Statistical View Advisor

Workload Statistics Advisor •

Workload Access Path Comparison

Workload Index Advisor •

Workload Query Advisor •



64

DB2 10.5 Features Exploited by Optim Tools

BLU

Enhanced PureScale

Enhanced HADR

Rolling Upgrade

Explicit Hierarchical

Lock

Data Studio ü

ü

ü

Optim Performance Manager

ü

ü

ü

Optim Configuration

Manager

ü

ü

ü

ü

Optim Query Workload Tuner

ü

InfoSphere Data Architect

ü

Product Feature

Date post:	06-Apr-2018
Category:	Documents
Upload:	trinhhuong
View:	227 times
Download:	4 times

Understanding/Managing the Access Plan Changes …laadb2ug.org/Downloads/Data_Studio_Phoenix-Sept...

Documents