1 12/6/2015 SQL Tuning Kyle Hailey VST Diagrams.

transcript

104/21/23

SQL Tuning

Kyle Haileyhttp://oraclemonitor.com

VST Diagrams

Systematic Methodology

• How do you tune a SQL statement?• Reading the SQL Text, scribbling notes on it –

• table sizes, join sizes, index info, histograms existence, table stats?

• Trace/tkprof, autotrace, display cursor()• If query doesn’t return , EXPLAIN PLAN

• Compare different explain plans ?

• Move filtered tables into inline views or CTE (sub-query

refactoring)

• Hints

• elapsed time, CPU usage, IO

•OK, but how do you analyze the SQL systematically?

204/21/23

Problems with SQL

• Externalized Info – tons !• Indexes, histograms, parameters, statistics, explain plans, erss

• Internal info , event 10053

• Unavailable information

• internal Optimizer routines

(10g SQL Tuning Advisor doesn’t even run the query)304/21/23

Suggested Profile

Alberto D’era : xplan

404/21/23

Parameters

Exec Stats

Execution Plan

Last Execute Stats

Partition Info

Index & constraints

Table Partition Info

Index Partition Info

http://www.adellera.it/scripts_etcetera/xplan/

Explain Plan

-----------------------------------------------------------------------| Id | Operation | Name | Cost |------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 278K|| 1 | TABLE ACCESS BY GLOBAL INDEX ROWID | ORDER_ITEMS | 3 || 2 | NESTED LOOPS | | 278K|| 3 | NESTED LOOPS | | 69829 || 4 | PARTITION HASH ALL | | 131 ||* 5 | TABLE ACCESS FULL | ORDERS | 131 || 6 | TABLE ACCESS BY GLOBAL INDEX ROWID| CUSTOMERS | 1 ||* 7 | INDEX UNIQUE SCAN | CUSTOMERS_PK | 0 ||* 8 | INDEX RANGE SCAN | ORDER_ITEMS_PK | 2 |------------------------------------------------------------------------

SELECT …FROM orders o ,

order_items oi, customers c WHERE o.order_id = oi.order_id and o.customer_id = c.customer_id and o.order_status <= 4;

---------------------------------------------------| Id | Operation | Name | Cost |---------------------------------------------------| 0 | SELECT STATEMENT | | 1058||* 1 | HASH JOIN | | 1058|| 2 | PARTITION HASH ALL | | 140|| 3 | TABLE ACCESS FULL | CUSTOMERS | 140|| 4 | PARTITION HASH ALL | | 439| |* 5 | HASH JOIN | | 439| |* 6 | TABLE ACCESS FULL| ORDERS | 130| | 7 | TABLE ACCESS FULL| ORDER_ITEMS | 293| ---------------------------------------------------

One Query, two plans, how are they different?

Comparing Plans

•Difficult Comparison

Graphical Explain Plan

Graphical Compare Still Difficult

O CNested Loops

OINested Loops

O CHash Join

OIHash Join

Visual SQL Tuning (VST)

804/21/23

22HJHJNLNL

HJHJ 11

VST Steps

Obejcts:

1. Tables drawn as nodes

2. Joins drawn as connector lines

3. Filters in where clause mark on each table

Statistics:

1.Table sizes for each table

• display in green above table node

2. Join Sizes Calculate two table join sizes

3. Calculate filter ratios

• filter ratio = number of rows returned with filter / number of rows

904/21/23

Basic Joins

Many to single value

CartesianCartesian

Tables and JoinsSELECT C.Phone_Number, C.Honorific, C.First_Name, C.Last_Name,

C.Suffix, C.Address_ID, A.Address_ID, A.Street_Address_Line1,

A.Street_Address_Line2, A.City_Name, A.State_Abbreviation,

A.ZIP_Code, OD.Deferred_Shipment_Date, OD.Item_Count,

ODT.Text, OT.Text, P.Product_Description, S.Shipment_Date

FROM Orders O, Order_Details OD, Products P, Customers C, Shipments S,

Addresses A, Code_Translations ODT, Code_Translations OT

WHERE UPPER(C.Last_Name) LIKE :Last_Name||'%'

AND UPPER(C.First_Name) LIKE :First_Name||'%'

AND OD.Order_ID = O.Order_ID

AND O.Customer_ID = C.Customer_ID

AND OD.Product_ID = P.Product_ID(+)

AND OD.Shipment_ID = S.Shipment_ID(+)

AND S.Address_ID = A.Address_ID(+)

AND O.Status_Code = OT.Code

AND OT.Code_Type = 'ORDER_STATUS'

AND OD.Status_Code = ODT.Code

AND ODT.Code_Type = 'ORDER_DETAIL_STATUS'

AND O.Order_Date > :Now - 366

ORDER BY C.Customer_ID, O.Order_ID DESC, S.Shipment_ID, OD.Order_Detail_ID;

Tables

Orders O,

Order_Details OD,

Products P,

Customers C,

Shipments S,

Addresses A,

Code_Translations ODT,

Code_Translations OT

OD.Order_ID = O.Order_ID

O.Customer_ID = C.Customer_ID

OD.Product_ID = P.Product_ID(+)

OD.Shipment_ID = S.Shipment_ID(+)

S.Address_ID = A.Address_ID(+)

O.Status_Code = OT.Code

OD.Status_Code = ODT.Code

Dan Tow – SQL TUNING

FiltersWHERE UPPER(C.Last_Name) LIKE :Last_Name||'%' AND UPPER(C.First_Name) LIKE :First_Name||'%' AND OT.Code_Type = 'ORDER_STATUS' AND O.Order_Date > :Now – 366 AND ODT.Code_Type = 'ORDER_DETAIL_STATUS'

Layout tables and connections

Tables

Orders O,

Order_Details OD,

Products P,

Customers C,

Shipments S,

Addresses A,

OTJoins

Layout tables and connections

Tables

Orders O,

Order_Details OD,

Products P,

Customers C,

Shipments S,

Addresses A,

OTJoins

Unstructured

OTJoins

Neater, but can you do anything with it?What’s the optimal execution path?

Parents and Children

Primary Key (unique index)

No index or non-unique

Master

Detail

Structurethe tree

VST – implied cartesian

1604/21/23

A client only has one broker at a time but might have many over time so we can take out the connection client to broker

Filter Ratios

1704/21/23

select v.green, f.yellow from vegetables v, fruit f where f.blue=v.blue and f.red='1000';

(select count(*) from fruit f where f.red='1000')---------------------------(select count(*) from fruit )

Filter Ratio =

OK, Diagrams indicate schema issuesBut what about execution Path?

Adding filter ratios help determine best path

VST – best path

Hailey

Parent

ChildConcept:1. Start at most selective filter2. Join down first, before joining upwards

FiltersWHERE UPPER(C.Last_Name) LIKE :Last_Name||'%' AND UPPER(C.First_Name) LIKE :First_Name||'%' AND OT.Code_Type = 'ORDER_STATUS‘ AND ODT.Code_Type = 'ORDER_DETAIL_STATUS' AND O.Order_Date > :Now – 366

Filter Ratios help determine best path

0.0002

VST – best path

0.0002

Note: Oracle only joins in one table to the previous result set

SELECT O.ORDER_ID, LINE_ITEM_ID, PRODUCT_ID, UNIT_PRICE, QUANTITY, ORDER_MODE, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID, C.CUSTOMER_ID, CUST_FIRST_NAME, CUST_LAST_NAME, CREDIT_LIMIT, CUST_EMAIL, ORDER_DATEFROM ORDERS O, ORDER_ITEMS OI, CUSTOMERS CWHERE O.ORDER_ID = OI.ORDER_ID AND O.CUSTOMER_ID = C.CUSTOMER_ID AND O.ORDER_STATUS <= 4

VST vs Explain Plan

2004/21/23

22MJMJ HJHJ

HJHJ i

Bad Plans vs Good Plan

2104/21/23

22MJMJ HJHJ

CartesianCartesian 11

VST – levels of information

2204/21/23

22NLNL

Basic VST VST with join stats

VST with Execution Path

DVD Now Planned

Join Set Sizes – upper bounds

Join type max result set size possible

one-to-scalar A this is equivalent to a filter on A table B returns one value like a max(), min(), count() etc

one-to-one min(rA,rB)

one-to-many rA Joining from A to B will not increase the result set size

many-to-many rA*rB-------------------------min(ndv(A),ndv(B))

The more duplicates in both tables the greater chance the result set size will explode

2304/21/23

We can say a lot about the join set size between two tables just by the join type and the number of rows in the table and NDV in the join column, but the easiest way most often is just to extract the two table joins and run a count(*) on that subset of the query

Execution Priority

1. Order of Joins

• Keep the running rows set as small as possible

2. Indexes used Index access – RS, FFS, SS (index_join , index_combine, and_equal)

3. Type of Joins (NL, MJ, HJ)

(other issues PQO, table caching etc)

2404/21/23

Once the order of execution is known, adding indexes and testing joins should be easy and meaningful

best execution order

Given 3 tables, what’s the best join Order?

Predicate Filter

VST – best execution order

Keep intermediate row set (running rowset) smallest fraction as long as possible

Result set stays same size or smaller

Result generally get’s bigger

1. Start at most selective filter

2. Join towards masters

3. Join towards the most selective filters

Detail

master

Predicate Filter1

B -> C -> A

Start here

go to A or C?

Now what?

VST – What does Oracle do?

defaultordered

Other joins and subqueries

•Subqueries

•Correlated subqueries

•Outer Joins

•In/Exists

•Not In/Not Exists

3004/21/23

Putting it together

3104/21/23

VST – cartesian

VST – implied cartesian

Something is wrongClient has *a* broker A client_transaction has *a* broker

Putting it all together, Q1

3404/21/23

SELECT /*+ ORDERED qb_name(outer) */ A0.zuchinis, A0.brocoli, C0.OrangesFROM ( SELECT /*+ NO_UNNEST */ A1.planted_date, A1.pears, A1.zuchinis, A1.brocoli FROM foo.A A1, ( SELECT /*+ NO_UNNEST */ zuchinis, brocoli FROM foo.A A2 WHERE pears = 'M' AND planted_date + 0 >= ADD_MONTHS ((SELECT /*+ NO_UNNEST */ MAX (planted_date) FROM foo.B B1 WHERE pears = 'M'), - 11) GROUP BY zuchinis, brocoli HAVING COUNT (*) = 12) i2 WHERE A1.planted_date = (SELECT /*+ NO_UNNEST */ MAX (planted_date) FROM foo.B B2 WHERE pears = 'M') AND A1.pears = 'M' AND A1.zuchinis = i2.zuchinis (+) AND A1.brocoli = i2.brocoli (+) UNION SELECT /*+ NO_UNNEST */ A4.planted_date, A4.pears, A4.zuchinis, A4.brocoli FROM foo.A A4 WHERE A4.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND A4.pears = TRIM ('D') AND A4.green_beans = '1' AND NOT EXISTS (SELECT /*+ NO_UNNEST */ * FROM foo.A A5 WHERE pears = 'M' AND planted_date = (SELECT /*+ NO_UNNEST */ MAX (planted_date) FROM foo.B B3 WHERE pears = 'M' ) AND A4.zuchinis = A5.zuchinis AND A4.brocoli = A5.brocoli)) B0, foo.A A0, foo.C C0, foo.D D0, foo.E E0WHERE A0.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND A0.pears = TRIM ('D') AND A0.green_beans = '1' AND A0.zuchinis = B0.zuchinis AND A0.brocoli = B0.brocoli AND A0.planted_date = C0.planted_date AND A0.pears = C0.pears AND A0.zuchinis = C0.zuchinis AND A0.brocoli = C0.brocoli AND A0.planted_date = D0.planted_date AND A0.pears = D0.pears AND A0.harvest_size = D0.harvest_size AND C0.Oranges = D0.Oranges AND C0.apples = D0.apples AND (D0.lemons = 0 OR D0.lemons IS NULL) AND A0.planted_date = E0.planted_date AND A0.pears = E0.pears AND A0.harvest_size = E0.harvest_size AND C0.Oranges = E0.Oranges AND C0.apples = E0.apples AND (E0.lemons = 0 OR E0.lemons IS NULL)ORDER BY A0.zuchinis, A0.brocoli

Q1 – get rid of scalar filter joins

3504/21/23

Now we can go farther simplifying the query because UNION has to be executed before joining into the rest of the query. Oracle, as of now, AFIK, has no way of merging a UNION inot the main query (I ran this by Benoit at Oracle who confirmed)But the UNION doesn't have that much interesting going on. We have an outer join between A1 and A2 and a NOT EXISTS between A4 and A5. We can vary how we do these joins, but the big question of join Order is mute because there are only two tables. Thus we can farther simplify the query as:

/*+ LEADING(A) */

Q1 – TTJ sizes

3604/21/23

Q1 – optimized path

3704/21/23

Q1 – before and after

3804/21/23

3904/21/23

SELECT CASE WHEN M.NYC IS NULL THEN (SELECT /*+ qb_name(qb1) */ MAX (Kona) FROM foo.F WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Norway = F_OUTER.Norway ELSE M.NYC END AS NYC, CASE WHEN F_OUTER.Perth IS NULL THEN NULL ELSE (SELECT /*+ qb_name(qb2) */ Georgia FROM foo.P WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Paris = F_OUTER.Perth) END AS richard, CASE WHEN F_OUTER.Aruba IS NULL THEN NULL ELSE (SELECT /*+ qb_name(qb3) */ Georgia FROM foo.P WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Paris = F_OUTER.Aruba) END AS Jody, CASE WHEN F_OUTER.Portland IS NULL THEN NULL ELSE (SELECT /*+ qb_name(qb4) */ Georgia FROM foo.P WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Paris = F_OUTER.Portland) END AS Tom FROM foo.F F_OUTER, foo.M , foo.J , foo.N , (SELECT /*+ qb_name(qb5) */ H.SF, Oregon, H.Haiti, K.Bermuda, L.Denmark FROM (foo.H LEFT OUTER JOIN foo.K ON H.harvest_date = K.harvest_date AND H.Argentina = K.Argentina AND H.SF = K.SF AND K.Dallas = '001') LEFT OUTER JOIN FOo.L ON H.harvest_date = L.harvest_date AND H.Argentina = L.Argentina AND H.SF = L.SF WHERE H.harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND H.Argentina = TRIM ('D')) extra WHERE F_OUTER.harvest_date = M.harvest_date(+) AND F_OUTER.Argentina = M.Argentina(+) AND F_OUTER.Norway = M.Norway(+) AND M.Norway(+) = M.Texas(+) AND F_OUTER.harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND F_OUTER.Argentina = TRIM ('D') AND M.harvest_date(+) = to_date('08/10/2008','dd/mm/yyyy') AND M.Argentina(+) = TRIM ('D') AND F_OUTER.Norway = F_OUTER.Hawaii AND F_OUTER.harvest_date = J.harvest_date(+) AND F_OUTER.Argentina = J.Argentina(+) AND F_OUTER.Norway = J.Texas(+) AND J.harvest_date(+) = to_date('08/10/2008','dd/mm/yyyy') AND J.Argentina(+) = TRIM ('D') AND F_OUTER.Iraq = extra.SF(+) AND F_OUTER.harvest_date = N.harvest_date(+) AND F_OUTER.Argentina = N.Argentina(+) AND F_OUTER.Norway = N.Hawaii(+) AND N.Jordon(+) = '0'/

Two important qualities:•All outer joins to F_OUTER•4 subqueries in select

4004/21/23

682348

select

4104/21/23

The subqueries in the select clause look like

select CASE WHEN F.f1 IS NULL THEN NULL ELSE (SELECT X.f2 FROM X WHERE code_vl = F.f1) END AS f0from F;

and should be merged into the query like: select CASE WHEN F.f1 IS NULL THEN NULL ELSE ( X.f2) END AS f0 from F , X where code_vl(+) = F.f1;

select CASE WHEN F.f1 IS NULL THEN NULL ELSE (SELECT X.f2 FROM X WHERE code_vl = F.f2) END AS f0from F;

select CASE WHEN F.f1 IS NULL THEN NULL ELSE ( X.f2) END AS f0 from F , X where code_vl(+) = F.f1;

4204/21/23

select distinct * from foo.a, foo.c, foo.d, foo.g WHERE a.planted_date = to_date('02/10/2008','dd/mm/yyyy') AND a.pears = 'D' AND a.green_beans = '1' AND a.planted_date = c.planted_date AND a.pears = c.pears AND a.zuchinis = c.zuchinis AND a.brocoli = c.brocoli AND a.planted_date = d.planted_date AND a.pears = d.pears AND a.harvest_size = d.harvest_size AND c.oranges = d.oranges AND c.apples = d.apples AND (d.lemons = 0 OR d.lemons IS NULL) AND a.planted_date = g.planted_date AND a.pears = g.pears AND a.harvest_size = g.harvest_size AND c.oranges = g.oranges AND c.apples = g.apples AND (g.lemons = 0 OR g.lemons IS NULL) and a.zuchinis='0236'ORDER BY a.zuchinis, a.brocoli;

4304/21/23

Default Optimized

First Path 4.5 secs, 1M logical readsSecond path 1.8 secs 0.2M Logical reads

Q3: Transitivity

4404/21/23

Q3: Transitivity

4504/21/23

Add a line for TransitivityCalculate two table join size

44,309

1,126,402

7,136,362

4604/21/23

SELECT * FROM ( SELECT /*+ NO_MERGE */ c.apples, c.oranges, a.harvest_size FROM a, c WHERE a.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND a.pears = 'D' AND a.green_beans = '1' AND a.planted_date = c.planted_date AND a.pears = c.pears AND a.zuchinis = c.zuchinis AND a.brocoli = c.brocoli AND a.zuchinis = '0236') X,( SELECT /*+ NO_MERGE */ d.apples, d.oranges, d.harvest_size FROM d, g WHERE d.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND g.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND g.apples = d.apples AND d.oranges = g.oranges AND d.pears = 'D' AND g.pears = 'D' AND g.pears = d.pears AND g.harvest_size = d.harvest_size AND (d.lemons = 0 OR d.lemons IS NULL) AND (g.lemons = 0 OR g.lemons IS NULL)) YWHEREX.oranges = Y.oranges ANDX.apples = Y.apples ANDX.harvest_size = Y.harvest_size;

It doesn't really matter where we start. We join (G,D) and separately we join (C,A) then we join these two result sets.

This final version runs inelapsed 0.33 secs and 12K logical readsdown from an originalelapsed 4.5 secs and 1M logical reads

VST Steps Summary

1.Diagram tables and joins

2.Draw connectors for each join

3.Calculate filter ratios

4.Find the table sizes

5.Calculate two table join sizes• Alternative Master Join Filter and Detail Join Filter

Execution Path

• Start at the most selective join filter

• Join to keep the running result set size the lowest

4704/21/23

4804/21/23

Time Component

4904/21/23

Tanel’s Explain Tool http://tech.e2sn.com/apps/planviz

• ORDERED - good on 9i

• Leading(tab_alias , table_alias … ) – 10g format

• USE_NL(table_alias) – Inner Table (not driving)

• USE_HASH(table_alias) – 2cd table, probe into

• INDEX(tab_alias index_name)

• NO_MERGE

Oracle first decides join order then join type

(example http://www.adp-gmbh.ch/blog/2008/01/17.php)

5004/21/23

Simple queries and sub-queries

select * from a, b where b.f=a.f

Non-correlated sub-queryselect * from a,(select b.f1 from b where b.f2=10) c where c.f1=a.f1

Simple join

select * from a where a.f1 = (select max(b.f1) from b )

Special case, where non correlated sub-query only returns one row

Correlated sub-queries

5204/21/23

Correlated aggregate subquery:

select ename from emp a where a.sal > (select avg(sal) from emp b where a.deptno=b.deptno)

Scalar Subqueries

select ename, (select avg(sal) from emp b where a.deptno=b.deptno) from emp a;

Exists and Not In

5304/21/23

SELECT d.*FROM dept d WHERE exists ( SELECT null FROM emp e WHERE e.deptno=d.deptno);

SELECT d.*FROM dept d WHERE d.deptno in ( SELECT deptno FROM emp e );

select distinct d.* from dept1 d ,emp e where e.deptno = d.deptno;

SELECT d.*FROM dept d WHERE not exists ( SELECT null FROM emp e WHERE e.deptno=d.deptno);

SELECT d.*FROM dept d WHERE d.deptno not in ( SELECT deptno FROM emp e where e.deptno is not null )or d.deptno is null;

select d.* from dept1 d left outer join emp e on e.deptno = d.deptno where e.deptno is null;

Outer Joins

5404/21/23

Outer Joins

1 12/6/2015 SQL Tuning Kyle Hailey VST Diagrams.

Documents