Date post: | 20-Jan-2016 |
Category: |
Documents |
Upload: | jodie-watson |
View: | 237 times |
Download: | 2 times |
104/21/23
SQL Tuning
Kyle Haileyhttp://oraclemonitor.com
VST Diagrams
Systematic Methodology
• How do you tune a SQL statement?• Reading the SQL Text, scribbling notes on it –
• table sizes, join sizes, index info, histograms existence, table stats?
• Trace/tkprof, autotrace, display cursor()• If query doesn’t return , EXPLAIN PLAN
• Compare different explain plans ?
• Move filtered tables into inline views or CTE (sub-query
refactoring)
• Hints
• elapsed time, CPU usage, IO
•OK, but how do you analyze the SQL systematically?
204/21/23
Problems with SQL
• Externalized Info – tons !• Indexes, histograms, parameters, statistics, explain plans, erss
• Internal info , event 10053
• Unavailable information
• internal Optimizer routines
• \
(10g SQL Tuning Advisor doesn’t even run the query)304/21/23
Suggested Profile
Alberto D’era : xplan
404/21/23
Parameters
Exec Stats
Execution Plan
Last Execute Stats
Partition Info
Index & constraints
Table Partition Info
Index Partition Info
http://www.adellera.it/scripts_etcetera/xplan/
Explain Plan
Copyright 2006 Kyle Hailey
-----------------------------------------------------------------------| Id | Operation | Name | Cost |------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 278K|| 1 | TABLE ACCESS BY GLOBAL INDEX ROWID | ORDER_ITEMS | 3 || 2 | NESTED LOOPS | | 278K|| 3 | NESTED LOOPS | | 69829 || 4 | PARTITION HASH ALL | | 131 ||* 5 | TABLE ACCESS FULL | ORDERS | 131 || 6 | TABLE ACCESS BY GLOBAL INDEX ROWID| CUSTOMERS | 1 ||* 7 | INDEX UNIQUE SCAN | CUSTOMERS_PK | 0 ||* 8 | INDEX RANGE SCAN | ORDER_ITEMS_PK | 2 |------------------------------------------------------------------------
SELECT …FROM orders o ,
order_items oi, customers c WHERE o.order_id = oi.order_id and o.customer_id = c.customer_id and o.order_status <= 4;
---------------------------------------------------| Id | Operation | Name | Cost |---------------------------------------------------| 0 | SELECT STATEMENT | | 1058||* 1 | HASH JOIN | | 1058|| 2 | PARTITION HASH ALL | | 140|| 3 | TABLE ACCESS FULL | CUSTOMERS | 140|| 4 | PARTITION HASH ALL | | 439| |* 5 | HASH JOIN | | 439| |* 6 | TABLE ACCESS FULL| ORDERS | 130| | 7 | TABLE ACCESS FULL| ORDER_ITEMS | 293| ---------------------------------------------------
One Query, two plans, how are they different?
Comparing Plans
•Difficult Comparison
Copyright 2006 Kyle Hailey
Graphical Explain Plan
Graphical Compare Still Difficult
O CNested Loops
OINested Loops
O CHash Join
OIHash Join
Visual SQL Tuning (VST)
804/21/23
11
33
22HJHJNLNL
HJHJ 11
33
22
NLNL
i
io
VST Steps
Obejcts:
1. Tables drawn as nodes
2. Joins drawn as connector lines
3. Filters in where clause mark on each table
Statistics:
1.Table sizes for each table
• display in green above table node
2. Join Sizes Calculate two table join sizes
3. Calculate filter ratios
• filter ratio = number of rows returned with filter / number of rows
904/21/23
Basic Joins
Many to single value
CartesianCartesian
Tables and JoinsSELECT C.Phone_Number, C.Honorific, C.First_Name, C.Last_Name,
C.Suffix, C.Address_ID, A.Address_ID, A.Street_Address_Line1,
A.Street_Address_Line2, A.City_Name, A.State_Abbreviation,
A.ZIP_Code, OD.Deferred_Shipment_Date, OD.Item_Count,
ODT.Text, OT.Text, P.Product_Description, S.Shipment_Date
FROM Orders O, Order_Details OD, Products P, Customers C, Shipments S,
Addresses A, Code_Translations ODT, Code_Translations OT
WHERE UPPER(C.Last_Name) LIKE :Last_Name||'%'
AND UPPER(C.First_Name) LIKE :First_Name||'%'
AND OD.Order_ID = O.Order_ID
AND O.Customer_ID = C.Customer_ID
AND OD.Product_ID = P.Product_ID(+)
AND OD.Shipment_ID = S.Shipment_ID(+)
AND S.Address_ID = A.Address_ID(+)
AND O.Status_Code = OT.Code
AND OT.Code_Type = 'ORDER_STATUS'
AND OD.Status_Code = ODT.Code
AND ODT.Code_Type = 'ORDER_DETAIL_STATUS'
AND O.Order_Date > :Now - 366
ORDER BY C.Customer_ID, O.Order_ID DESC, S.Shipment_ID, OD.Order_Detail_ID;
Copyright 2006 Kyle Hailey
Tables
Orders O,
Order_Details OD,
Products P,
Customers C,
Shipments S,
Addresses A,
Code_Translations ODT,
Code_Translations OT
Joins
OD.Order_ID = O.Order_ID
O.Customer_ID = C.Customer_ID
OD.Product_ID = P.Product_ID(+)
OD.Shipment_ID = S.Shipment_ID(+)
S.Address_ID = A.Address_ID(+)
O.Status_Code = OT.Code
OD.Status_Code = ODT.Code
Dan Tow – SQL TUNING
FiltersWHERE UPPER(C.Last_Name) LIKE :Last_Name||'%' AND UPPER(C.First_Name) LIKE :First_Name||'%' AND OT.Code_Type = 'ORDER_STATUS' AND O.Order_Date > :Now – 366 AND ODT.Code_Type = 'ORDER_DETAIL_STATUS'
Layout tables and connections
Copyright 2006 Kyle Hailey
Tables
Orders O,
Order_Details OD,
Products P,
Customers C,
Shipments S,
Addresses A,
Code_Translations ODT,
Code_Translations OT
ODT
OD
P
C
A
O S
OTJoins
OD.Order_ID = O.Order_ID
O.Customer_ID = C.Customer_ID
OD.Product_ID = P.Product_ID(+)
OD.Shipment_ID = S.Shipment_ID(+)
S.Address_ID = A.Address_ID(+)
O.Status_Code = OT.Code
OD.Status_Code = ODT.Code
Layout tables and connections
Copyright 2006 Kyle Hailey
Tables
Orders O,
Order_Details OD,
Products P,
Customers C,
Shipments S,
Addresses A,
Code_Translations ODT,
Code_Translations OT
ODT
OD
P
C
A
O S
OTJoins
OD.Order_ID = O.Order_ID
O.Customer_ID = C.Customer_ID
OD.Product_ID = P.Product_ID(+)
OD.Shipment_ID = S.Shipment_ID(+)
S.Address_ID = A.Address_ID(+)
O.Status_Code = OT.Code
OD.Status_Code = ODT.Code
Messy
Unstructured
Copyright 2006 Kyle Hailey
ODT
OD
P
C
A
O S
OTJoins
OD.Order_ID = O.Order_ID
O.Customer_ID = C.Customer_ID
OD.Product_ID = P.Product_ID(+)
OD.Shipment_ID = S.Shipment_ID(+)
S.Address_ID = A.Address_ID(+)
O.Status_Code = OT.Code
OD.Status_Code = ODT.Code
Neater, but can you do anything with it?What’s the optimal execution path?
Parents and Children
Copyright 2006 Kyle Hailey
Joins
OD.Order_ID = O.Order_ID
O.Customer_ID = C.Customer_ID
OD.Product_ID = P.Product_ID(+)
OD.Shipment_ID = S.Shipment_ID(+)
S.Address_ID = A.Address_ID(+)
O.Status_Code = OT.Code
OD.Status_Code = ODT.Code
ODT
OD
P
CA
OS
OT
Primary Key (unique index)
No index or non-unique
Master
Detail
Structurethe tree
VST – implied cartesian
1604/21/23
A client only has one broker at a time but might have many over time so we can take out the connection client to broker
Filter Ratios
1704/21/23
select v.green, f.yellow from vegetables v, fruit f where f.blue=v.blue and f.red='1000';
(select count(*) from fruit f where f.red='1000')---------------------------(select count(*) from fruit )
Filter Ratio =
OK, Diagrams indicate schema issuesBut what about execution Path?
Adding filter ratios help determine best path
VST – best path
Copyright 2006 Kyle
Hailey
ODT
OD
P
CA
OS
OT
Parent
Child
Parent
ChildConcept:1. Start at most selective filter2. Join down first, before joining upwards
F
FiltersWHERE UPPER(C.Last_Name) LIKE :Last_Name||'%' AND UPPER(C.First_Name) LIKE :First_Name||'%' AND OT.Code_Type = 'ORDER_STATUS‘ AND ODT.Code_Type = 'ORDER_DETAIL_STATUS' AND O.Order_Date > :Now – 366
Filter Ratios help determine best path
F
0.3
0.0002
VST – best path
ODT
OD
P
CA
OS
OTF
0.3
0.0002
F
Note: Oracle only joins in one table to the previous result set
SELECT O.ORDER_ID, LINE_ITEM_ID, PRODUCT_ID, UNIT_PRICE, QUANTITY, ORDER_MODE, ORDER_STATUS, ORDER_TOTAL, SALES_REP_ID, PROMOTION_ID, C.CUSTOMER_ID, CUST_FIRST_NAME, CUST_LAST_NAME, CREDIT_LIMIT, CUST_EMAIL, ORDER_DATEFROM ORDERS O, ORDER_ITEMS OI, CUSTOMERS CWHERE O.ORDER_ID = OI.ORDER_ID AND O.CUSTOMER_ID = C.CUSTOMER_ID AND O.ORDER_STATUS <= 4
VST vs Explain Plan
2004/21/23
11
33
22
11
33
22MJMJ HJHJ
NLNL
HJHJ i
Bad Plans vs Good Plan
2104/21/23
33
22 33
22MJMJ HJHJ
NLNL
HJHJ
22
33
CartesianCartesian 11
11
11
i
VST – levels of information
2204/21/23
0.72
0.5M
0.9M
1.8M
1.3M
0.4M
33
22NLNL
NLNL
i
io
11
Basic VST VST with join stats
VST with Execution Path
DVD Now Planned
Join Set Sizes – upper bounds
Join type max result set size possible
notes
one-to-scalar A this is equivalent to a filter on A table B returns one value like a max(), min(), count() etc
one-to-one min(rA,rB)
one-to-many rA Joining from A to B will not increase the result set size
many-to-many rA*rB-------------------------min(ndv(A),ndv(B))
The more duplicates in both tables the greater chance the result set size will explode
2304/21/23
We can say a lot about the join set size between two tables just by the join type and the number of rows in the table and NDV in the join column, but the easiest way most often is just to extract the two table joins and run a count(*) on that subset of the query
Execution Priority
1. Order of Joins
• Keep the running rows set as small as possible
2. Indexes used Index access – RS, FFS, SS (index_join , index_combine, and_equal)
3. Type of Joins (NL, MJ, HJ)
(other issues PQO, table caching etc)
2404/21/23
Once the order of execution is known, adding indexes and testing joins should be easy and meaningful
best execution order
Given 3 tables, what’s the best join Order?
Predicate Filter
VST – best execution order
Keep intermediate row set (running rowset) smallest fraction as long as possible
Result set stays same size or smaller
Result generally get’s bigger
1. Start at most selective filter
2. Join towards masters
3. Join towards the most selective filters
Detail
master
VST – best execution order
Predicate Filter1
2
3
B -> C -> A
VST – best execution order
Start here
go to A or C?
Now what?
VST – What does Oracle do?
defaultordered
defaultordered
Other joins and subqueries
•Subqueries
•Correlated subqueries
•Outer Joins
•In/Exists
•Not In/Not Exists
3004/21/23
Putting it together
3104/21/23
VST – cartesian
VST – implied cartesian
z
Something is wrongClient has *a* broker A client_transaction has *a* broker
Putting it all together, Q1
3404/21/23
SELECT /*+ ORDERED qb_name(outer) */ A0.zuchinis, A0.brocoli, C0.OrangesFROM ( SELECT /*+ NO_UNNEST */ A1.planted_date, A1.pears, A1.zuchinis, A1.brocoli FROM foo.A A1, ( SELECT /*+ NO_UNNEST */ zuchinis, brocoli FROM foo.A A2 WHERE pears = 'M' AND planted_date + 0 >= ADD_MONTHS ((SELECT /*+ NO_UNNEST */ MAX (planted_date) FROM foo.B B1 WHERE pears = 'M'), - 11) GROUP BY zuchinis, brocoli HAVING COUNT (*) = 12) i2 WHERE A1.planted_date = (SELECT /*+ NO_UNNEST */ MAX (planted_date) FROM foo.B B2 WHERE pears = 'M') AND A1.pears = 'M' AND A1.zuchinis = i2.zuchinis (+) AND A1.brocoli = i2.brocoli (+) UNION SELECT /*+ NO_UNNEST */ A4.planted_date, A4.pears, A4.zuchinis, A4.brocoli FROM foo.A A4 WHERE A4.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND A4.pears = TRIM ('D') AND A4.green_beans = '1' AND NOT EXISTS (SELECT /*+ NO_UNNEST */ * FROM foo.A A5 WHERE pears = 'M' AND planted_date = (SELECT /*+ NO_UNNEST */ MAX (planted_date) FROM foo.B B3 WHERE pears = 'M' ) AND A4.zuchinis = A5.zuchinis AND A4.brocoli = A5.brocoli)) B0, foo.A A0, foo.C C0, foo.D D0, foo.E E0WHERE A0.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND A0.pears = TRIM ('D') AND A0.green_beans = '1' AND A0.zuchinis = B0.zuchinis AND A0.brocoli = B0.brocoli AND A0.planted_date = C0.planted_date AND A0.pears = C0.pears AND A0.zuchinis = C0.zuchinis AND A0.brocoli = C0.brocoli AND A0.planted_date = D0.planted_date AND A0.pears = D0.pears AND A0.harvest_size = D0.harvest_size AND C0.Oranges = D0.Oranges AND C0.apples = D0.apples AND (D0.lemons = 0 OR D0.lemons IS NULL) AND A0.planted_date = E0.planted_date AND A0.pears = E0.pears AND A0.harvest_size = E0.harvest_size AND C0.Oranges = E0.Oranges AND C0.apples = E0.apples AND (E0.lemons = 0 OR E0.lemons IS NULL)ORDER BY A0.zuchinis, A0.brocoli
Q1 – get rid of scalar filter joins
3504/21/23
Now we can go farther simplifying the query because UNION has to be executed before joining into the rest of the query. Oracle, as of now, AFIK, has no way of merging a UNION inot the main query (I ran this by Benoit at Oracle who confirmed)But the UNION doesn't have that much interesting going on. We have an outer join between A1 and A2 and a NOT EXISTS between A4 and A5. We can vary how we do these joins, but the big question of join Order is mute because there are only two tables. Thus we can farther simplify the query as:
/*+ LEADING(A) */
Q1 – TTJ sizes
3604/21/23
Q1 – optimized path
3704/21/23
Q1 – before and after
3804/21/23
Q2
3904/21/23
SELECT CASE WHEN M.NYC IS NULL THEN (SELECT /*+ qb_name(qb1) */ MAX (Kona) FROM foo.F WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Norway = F_OUTER.Norway ELSE M.NYC END AS NYC, CASE WHEN F_OUTER.Perth IS NULL THEN NULL ELSE (SELECT /*+ qb_name(qb2) */ Georgia FROM foo.P WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Paris = F_OUTER.Perth) END AS richard, CASE WHEN F_OUTER.Aruba IS NULL THEN NULL ELSE (SELECT /*+ qb_name(qb3) */ Georgia FROM foo.P WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Paris = F_OUTER.Aruba) END AS Jody, CASE WHEN F_OUTER.Portland IS NULL THEN NULL ELSE (SELECT /*+ qb_name(qb4) */ Georgia FROM foo.P WHERE harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND Argentina = TRIM ('D') AND Paris = F_OUTER.Portland) END AS Tom FROM foo.F F_OUTER, foo.M , foo.J , foo.N , (SELECT /*+ qb_name(qb5) */ H.SF, Oregon, H.Haiti, K.Bermuda, L.Denmark FROM (foo.H LEFT OUTER JOIN foo.K ON H.harvest_date = K.harvest_date AND H.Argentina = K.Argentina AND H.SF = K.SF AND K.Dallas = '001') LEFT OUTER JOIN FOo.L ON H.harvest_date = L.harvest_date AND H.Argentina = L.Argentina AND H.SF = L.SF WHERE H.harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND H.Argentina = TRIM ('D')) extra WHERE F_OUTER.harvest_date = M.harvest_date(+) AND F_OUTER.Argentina = M.Argentina(+) AND F_OUTER.Norway = M.Norway(+) AND M.Norway(+) = M.Texas(+) AND F_OUTER.harvest_date = to_date('08/10/2008','dd/mm/yyyy') AND F_OUTER.Argentina = TRIM ('D') AND M.harvest_date(+) = to_date('08/10/2008','dd/mm/yyyy') AND M.Argentina(+) = TRIM ('D') AND F_OUTER.Norway = F_OUTER.Hawaii AND F_OUTER.harvest_date = J.harvest_date(+) AND F_OUTER.Argentina = J.Argentina(+) AND F_OUTER.Norway = J.Texas(+) AND J.harvest_date(+) = to_date('08/10/2008','dd/mm/yyyy') AND J.Argentina(+) = TRIM ('D') AND F_OUTER.Iraq = extra.SF(+) AND F_OUTER.harvest_date = N.harvest_date(+) AND F_OUTER.Argentina = N.Argentina(+) AND F_OUTER.Norway = N.Hawaii(+) AND N.Jordon(+) = '0'/
Two important qualities:•All outer joins to F_OUTER•4 subqueries in select
Q2
4004/21/23
12
825
845
682348
select
Q2
4104/21/23
The subqueries in the select clause look like
select CASE WHEN F.f1 IS NULL THEN NULL ELSE (SELECT X.f2 FROM X WHERE code_vl = F.f1) END AS f0from F;
and should be merged into the query like: select CASE WHEN F.f1 IS NULL THEN NULL ELSE ( X.f2) END AS f0 from F , X where code_vl(+) = F.f1;
select CASE WHEN F.f1 IS NULL THEN NULL ELSE (SELECT X.f2 FROM X WHERE code_vl = F.f2) END AS f0from F;
select CASE WHEN F.f1 IS NULL THEN NULL ELSE ( X.f2) END AS f0 from F , X where code_vl(+) = F.f1;
Q2
4204/21/23
select distinct * from foo.a, foo.c, foo.d, foo.g WHERE a.planted_date = to_date('02/10/2008','dd/mm/yyyy') AND a.pears = 'D' AND a.green_beans = '1' AND a.planted_date = c.planted_date AND a.pears = c.pears AND a.zuchinis = c.zuchinis AND a.brocoli = c.brocoli AND a.planted_date = d.planted_date AND a.pears = d.pears AND a.harvest_size = d.harvest_size AND c.oranges = d.oranges AND c.apples = d.apples AND (d.lemons = 0 OR d.lemons IS NULL) AND a.planted_date = g.planted_date AND a.pears = g.pears AND a.harvest_size = g.harvest_size AND c.oranges = g.oranges AND c.apples = g.apples AND (g.lemons = 0 OR g.lemons IS NULL) and a.zuchinis='0236'ORDER BY a.zuchinis, a.brocoli;
Q3
4304/21/23
Default Optimized
First Path 4.5 secs, 1M logical readsSecond path 1.8 secs 0.2M Logical reads
Q3: Transitivity
4404/21/23
Q3: Transitivity
4504/21/23
Add a line for TransitivityCalculate two table join size
188
44,309
1,126,402
7,136,362
Q3
4604/21/23
SELECT * FROM ( SELECT /*+ NO_MERGE */ c.apples, c.oranges, a.harvest_size FROM a, c WHERE a.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND a.pears = 'D' AND a.green_beans = '1' AND a.planted_date = c.planted_date AND a.pears = c.pears AND a.zuchinis = c.zuchinis AND a.brocoli = c.brocoli AND a.zuchinis = '0236') X,( SELECT /*+ NO_MERGE */ d.apples, d.oranges, d.harvest_size FROM d, g WHERE d.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND g.planted_date = TO_DATE ('02/10/2008', 'dd/mm/yyyy') AND g.apples = d.apples AND d.oranges = g.oranges AND d.pears = 'D' AND g.pears = 'D' AND g.pears = d.pears AND g.harvest_size = d.harvest_size AND (d.lemons = 0 OR d.lemons IS NULL) AND (g.lemons = 0 OR g.lemons IS NULL)) YWHEREX.oranges = Y.oranges ANDX.apples = Y.apples ANDX.harvest_size = Y.harvest_size;
It doesn't really matter where we start. We join (G,D) and separately we join (C,A) then we join these two result sets.
This final version runs inelapsed 0.33 secs and 12K logical readsdown from an originalelapsed 4.5 secs and 1M logical reads
VST Steps Summary
1.Diagram tables and joins
2.Draw connectors for each join
3.Calculate filter ratios
4.Find the table sizes
5.Calculate two table join sizes• Alternative Master Join Filter and Detail Join Filter
Execution Path
• Start at the most selective join filter
• Join to keep the running result set size the lowest
4704/21/23
4804/21/23
Time Component
4904/21/23
Tanel’s Explain Tool http://tech.e2sn.com/apps/planviz
HINTS
• ORDERED - good on 9i
• Leading(tab_alias , table_alias … ) – 10g format
• USE_NL(table_alias) – Inner Table (not driving)
• USE_HASH(table_alias) – 2cd table, probe into
• INDEX(tab_alias index_name)
• NO_MERGE
Oracle first decides join order then join type
(example http://www.adp-gmbh.ch/blog/2008/01/17.php)
5004/21/23
Simple queries and sub-queries
51
AB
select * from a, b where b.f=a.f
Non-correlated sub-queryselect * from a,(select b.f1 from b where b.f2=10) c where c.f1=a.f1
Simple join
select * from a where a.f1 = (select max(b.f1) from b )
Special case, where non correlated sub-query only returns one row
A B
A
BA B
Correlated sub-queries
5204/21/23
Correlated aggregate subquery:
select ename from emp a where a.sal > (select avg(sal) from emp b where a.deptno=b.deptno)
A
B
A
B
Scalar Subqueries
select ename, (select avg(sal) from emp b where a.deptno=b.deptno) from emp a;
Exists and Not In
5304/21/23
SELECT d.*FROM dept d WHERE exists ( SELECT null FROM emp e WHERE e.deptno=d.deptno);
SELECT d.*FROM dept d WHERE d.deptno in ( SELECT deptno FROM emp e );
select distinct d.* from dept1 d ,emp e where e.deptno = d.deptno;
SELECT d.*FROM dept d WHERE not exists ( SELECT null FROM emp e WHERE e.deptno=d.deptno);
SELECT d.*FROM dept d WHERE d.deptno not in ( SELECT deptno FROM emp e where e.deptno is not null )or d.deptno is null;
select d.* from dept1 d left outer join emp e on e.deptno = d.deptno where e.deptno is null;
Outer Joins
5404/21/23
Outer Joins