Advanced Tuning Oracle

1

© 2008 Wellesley Information Services. All rights reserved.

Advanced Internal Oracle Tuning Techniques for SAP Systems

Michael KennedySMS Consulting

1

What We’ll Cover …

• Understanding index design• Proactive review of poorly performing SQL• 10053 CBO trace• Oracle Wait Events• 10046 tracing• SAP transactions for tuning • Oracle best practices• Wrap-up

2

B*Tree Design

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

2

3

B*Tree Indexes

• B*Tree indexes are “always” balancedAll leaf blocks, which hold the row ID, are on the same level

• Index entries are “always” ordered• Index scans use a “db file sequential read”

Single block fetch (not a multi-block fetch), except for Index Fast Full Scan Can be slow for large index RANGE scans

4

Optimal Oracle Block Access

• What is the “optimal” number of blocks read for an index?

The optimal number of blocks being accessed in Oracle is between 3 and 5.During a unique scan down the index, the root block, branch block, leaf block, then the table data is read into the databasecache. Depending on the height of the index, a unique scan will only read 3 to 5 blocks.Use the 3 to 5 blocks as the optimal value to compare with other inefficient SQL statements.

5


• Understanding index design• Proactive review of poorly performing SQL• 10053 CBO trace• Oracle Wait Events• 10046 tracing• SAP transactions for tuning• Oracle best practices• Wrap-up

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

3

6

Proactive Review of Poor SQL

• What is poor SQL?Inefficient SAP programs, usually custom programs, can cause excessive CPU and I/O loads on the database serverMillions of SQL statements can be called in an SAP system, requiring frequently executed statements to be tuned to reduce the impact on server resources

• What is the return on investment?Improved I/O and CPU timesFrequently used data (e.g., SD users) stays cached in Oracle’s buffer cache longer, resulting in faster user response timesImproved SAP average dialog response timesImproved overall performance of the system

7

Proactive Review of Poor SQL (cont.)

• ST04 Detailed Analysis SQL Requests

8

Proactive Review of Poor SQL (cont.)

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

4

9

Inefficient SQL

• Example of an inefficient SQL statement

10

Efficient SQL

• Example of an efficient SQL statement

11

SQL Statement General Rules

• In general, RANGE scans may be slower and require additional resources (CPU, memory, I/O)

• Try to access the least amount of data down an index• Oracle joins may not be the fastest option depending on the

driving table of the “inner” joinIf a join is slow, determine the inner-most index of the explain planIf the index is not efficient, a custom index may be createdIf the CBO is not picking the correct “driving” index, a HINT should be used (SAP Note 130480)

• Check SAP Notes for performance enhancements for suspected transaction — some standard SAP ABAP code may pass the LITERALS instead of BIND variables

Example: MB51 transaction is slow due to large number of rows inMSEG/MKPFApplying SAP Note 921164 may change that with LITERALS instead of BIND variables

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

5

12



13

CBO 10053 Trace

• Tracing the CBO (event 10053) can help prove why the CBO is not picking the correct index

• Parameters that affect the CBO:db_file_multiblock_read_count = 8

The higher the value, the lower the cost of a full tablescanoptimizer_index_cost_adj = 10

The lower the value, the lower the cost of an index accessEvent 10183

Used in conjunction with optimizer_index_cost_adj when two indexes have a cost calculation less than one

14

CBO Optimizer Trace

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

6

15

CBO Optimizer Trace Error

• The error below is shown for some versions of SAP• However, the trace file is generated in: /oracle/<SID>/saptrace/usertrace

16

CBO Optimizer Trace Example

17

When Should Oracle Indexes Be Rebuilt?

• After large data deletes• When indexes are very large and you would like to

reclaim space• If an index has too many extents (greater than 1,000)• When there are SQL statements that do large RANGE

SCANS (<,>, like, between) on the index. Keeping the indexes small will improve performance.

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

7

18

Rebuilding Indexes

• SAP Note 771929• DB02

Detailed Analysis Enter Index Detailed Analysis Menu: Analyze Index Storage QualityUses Oracle’s DBMS_SPACE package to calculate space usage

19

Rebuilding Indexes (cont.)

• SAP Note 771929• SE38

Program RSORATAD Ran for one hour on 20 GB index MSEG~0 with 304 million rows

20

Rebuilding Indexes (cont.)

• SAP Note 771929• Analyze Index VALIDATE Structure

Analyze index “&index” validate structureSelect * from index_statsAnalyze will lock all DML on the table so run on a copy of production

SQL> select name, height, lf_rows, del_lf_rows, pct_used from index_stats;

NAME HEIGHT LF_ROWS DEL_LF_ROWS PCT_USED

-------------------- ---------- ---------- ----------- ----------

TST03~0 3 159608 115497 54

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

8

21

How Do You Monitor Index Creation or Rebuild Status?

create index "EKBE~Z1" on EKBE(MANDT,VGABE, BUDAT, WERKS)tablespace psaprtlonlineparallel (degree 5)

select a.username, a.tablespace, b.sid, a.contents,a.segtype, a.extents, a.blocks*8192 "Bytes"from v$sort_usage a, v$session bwhere a.session_addr = b.saddr

USERNAME TABLESPACE SID CONTENTS SEGTYPE EXTENTS Bytes---------------------- ---------- --------- --------- ---------- ----------------SAPRTL PSAPTEMP 393 TEMPORARY SORT 64 67,108,864SAPRTL PSAPTEMP 384 TEMPORARY SORT 40 41,943,040SAPRTL PSAPTEMP 367 TEMPORARY SORT 80 83,886,080SAPRTL PSAPTEMP 395 TEMPORARY SORT 61 63,963,136SAPRTL PSAPTEMP 382 TEMPORARY SORT 67 70,254,592

---------- ----------------sum 312 327,155,712

• The index in this example is created with a parallel degree of 5 and is being sorted in PSAPTEMP

22

How Do You Monitor Index Creation or Rebuild Status? (cont.)

• With this script, you can monitor the index being created in the “target” tablespaceselect segment_name, tablespace_name, bytes, extents from dba_segmentswhere segment_type = 'TEMPORARY'ORDER BY 2

SEGMENT_NAME TABLESPACE_NAME BYTES EXTENTS--------------- --------------- ---------------- ----------31.680100 PSAPRTL 98,631,680 8463.290980 PSAPRTL 79,626,240 8197.325500 PSAPRTL 85,262,336 82208.153460 PSAPRTL 81,002,496 81237.178036 PSAPRTL 95,420,416 86

---------------- ----------sum 439,943,168 414

23

Tips for Index Creations and Rebuild

• The pga_aggreagate_target only uses 100 MB for sorts. Using _pga_max_size may help, which is a setting in BW.

• Use the parallel query for faster creation timesRun the create or rebuild index during a low peak time, because high Data Manipulation Language (DML) may cause a table lock (SAP Note 682926)

• Use ONLINE keyword so the table is not locked• Use NOLOGGING to minimize redo generation

The index will need to be dropped and recreated if using Oracle dataguardThe index will need to be dropped and recreated if a recovery is needed if a backup was not taken after the create or rebuild command

• PSAPTEMP needs to be at least 1.5 times the size of the largest index being created in order to hold the ordered data

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

9

24

Tips for Index Creations and Rebuild (cont.)• Schedule Index Rebuilds• SE38

Program RSANAORAList of Indexes in variant

• Online Index Rebuilds• If locking occurs – Kill the Oracle SQL*Net process, NOT the SAP session

25

Reactive Troubleshooting: User Performance Complaint

26

Example: BSAS SQL Statement

• ST04 Detailed Analysis Oracle Session

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

10

27

BSAS Buffer Gets

28

BSAS Solution

• There have been 13,905 executions while the database has been up for 20 days

• That means this query is executed 695 times per day or 28 times per hour

• Each execution needs to allocate 459 blocks of memory to find almost ZERO rows

• The BSAS statement has fetched almost 7 million blocks into memory with only 6 rows found out of the 13,905 executions

• Resolution:Add ZUONR to the custom index BSAS~Z1Create a new custom index as ZUONR, BUKRS, HKONT, BUDAT

29



_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

11

30

Oracle Wait Events

• Users who are “waiting” are not happy users!• Every session that waits must wait for a specific event,

and knowing where to look can answer “why” the session is waiting

• Areas to look for waits: V$SESSION_WAITV$SYSTEM_EVENTV$SESSIONV$LOCKSAP Transaction ST04 Detailed Analysis Oracle Sessions

31

Common Oracle Wait Events

• DB FILE SEQUENTIAL READSession is waiting for I/O from an explain plan using an index. A single block I/O is done.

• DB FILE SCATTERED READSession is waiting for I/O when doing a FULL TABLESCAN or FAST FULL INDEX scan. Multi-block operations are done.

• DB FILE PARALLEL WRITEThe DBWR is writing data to the datafiles.

• LOG FILE SYNCThe session is waiting for the LGWR to write the data from the log buffer to the online redo logs because of the commit.

• LATCH FREEThe session is waiting for an Oracle Latch. Use P2 value to determine latch name to resolve the issue in 9i. Oracle 10G will show the full latch name.

32

Common Oracle Wait Events (cont.)

• ENQUEUEThe session is waiting for an Oracle Enqueue, which is most likely either a(n):

ST Enqueue — Space Transaction Enqueue for extent managementTX Enqueue — Multiple sessions modifying the exact same row concurrently (usually a SELECT FOR UPDATE)

• WRITE COMPLETE WAITThe session is waiting for the DBWR to write the dirty block to disk

• FREE BUFFER WAITThe session is waiting for the DBWR to clean up dirty blocks in the Oracle buffer cache

• LIBRARY CACHE PIN or LIBRARY CACHE LOCKWaiting for exclusive lock to the library cache and may indicate a Shared Pool issue

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

12

33

Oracle Wait Events Example

• In this example on Oracle 9i, there are many users waiting for a LATCH FREE for KNB1

34

Oracle Wait Events Example (cont.)

• In this example on Oracle 10G, there are many users waiting for a LATCH on “cache buffers chains”

35

Oracle Wait Events Example (cont.)

• In this example from v$session_wait, the event and the time waiting can be seen

• Shown below, a FULL TABLESCAN (db file scattered read) can be seen. Why? FULL TABLESCANS should not be common!

• Similar information in ST04 Detailed Analysis Oracle Session

Select sid, seq#, event, p1,p2,p3, seconds_in_wait from v$session_waitwhere event not like ‘SQL%’ and event not like ‘%ipc%’and event not like ‘%timer’;

SID SEQ# EVENT P1 P2 P3 SECONDS_IN_WAIT---------- ---------- ------------------------------ ---------- ---------- ---------- ---------------

624 18902 buffer busy waits 354 44041 130 02002 41785 db file scattered read 286 73460 8 0157 30307 db file sequential read 53 320723 1 0

2245 1775 db file sequential read 354 44209 1 011 48565 log file parallel write 2 20 2 0

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

13

36

Oracle Enqueue Waits

• Usually, there are two types of Oracle Enqueues (not the same as SAP Enqueues)

TX Enqueue is a row level lock (i.e., DATA) and common in parallel jobs

NRIV LockingLocking on similar tables due to SELECT FOR UPDATE

Non-TX Enqueue (i.e., TM, ST, CI) is usually locking in Oracle kernel and is not DATA related

ST Enqueue due to Dictionary Managed Tablespaces and extent managementOracle bug

37

Oracle Enqueue Waits (cont.)

• Steps to diagnose Oracle EnqueuesSM66 can show concurrent access to a table (e.g., CE3LBOC below)

38


• Use ST04 Detailed Analysis Oracle Session Shows the SQL statement “causing” the enqueue

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

14

39


• Use transaction DB01 to see the TX enqueues and the waiter/holder

• Use the Oracle SID (System ID) shown below in DB01 to map to the session in ST04

40


• For Oracle 9i, run the following scripts in SQL*Plus to determine TYPE of enqueue. The TYPE determines in what area the problem resides.

• The example below shows two sessions with TX enqueue locking.

Prompt This will show the TYPE of enqueueSELECT sid, chr(bitand(p1,-16777216)/16777215)||chr(bitand(p1, 16711680)/65535) "Lock",to_char( bitand(p1, 65535) ) "Mode"FROM v$session_waitWHERE event = 'enqueue‘;

SID Lock Mode----- ------ ----------------------------------------

383 TX 4385 TX 4

41

• For 9i, use the following script to see the object and row of the enqueue prompt. This will show the object and row of enqueue.

• In this example, the Object 122961 is where the TX enqueue occurs.

col event format a25select a.sid, b.event, a.row_wait_obj#, a.row_wait_file#,

a.row_wait_block#, a.row_wait_row#, b.seconds_in_waitfrom v$session a, v$session_wait bwhere a.sid = b.sidand b.event = 'enqueue'order by 2, 3, 4, 5;

SID EVENT ROW_WAIT_OBJ# ROW_WAIT_FILE# ROW_WAIT_BLOCK# ROW_WAIT_ROW# SECONDS_IN_WAIT---------- --------- ------------- -------------- --------------- ------------- ---------------

381 enqueue 122961 25 25020 0 46383 enqueue 122961 25 25020 0 43385 enqueue 122961 25 25020 0 80


_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

15

42


• Oracle 10G will show the TYPE of enqueue in the event

43

General Rules for Oracle Waits

• Determine the “WAIT” event for the sessions, then a course of action

Full tablescans are not common and probably should be fixedLarge index RANGE scans may be slow due to single block fetch from db file sequential read

• Transactional enqueues (TX) cause row-level ENQUEUE waits that are usually waiting for the commit due to high concurrency. Try to commit more frequently.

Determine the TYPE of enqueueIf it is TX, then it is most likely application concurrencyIf it is NOT TX, then it may be in the Oracle kernel and more research is needed

Concurrent “Select for UPDATE” statements are a common cause of TX enqueues

44



_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

16

45

Oracle Tracing Overview

• Use Oracle tracing methods to:Determine if the Database Writer (DBWR) is writing inefficientlyDetermine if the Log Writer (LGWR) is slowing down on the write timesDetermine actual “fetch” times for Oracle blocks from the disk to the buffer cache. Use the results to ascertain if the disk subsystem is slow.Correlate Oracle’s trace with SAP’s ST05 trace to find similar poor performanceTrace other Oracle background processes, if needed, to determine what is causing them to hang or spin

46

Event 10046 Levels

• Oracle provides event 10046 to trace user and background processes to determine what they are WAITING for

• Event 10046 has different tracing levels: Level 1 (default) — Standard SQL_TRACE functionalityLevel 4 — SQL_TRACE plus tracing BIND VARIABLESLevel 8 — SQL_TRACE plus WAIT EVENTSLevel 12 — SQL_TRACE plus BIND VARIABLES plus WAIT EVENTS

47

Tracing User SessionsSQL> alter session set events '10046 trace name context forever,

level 12';

Session altered.

SQL> select count(*) from dirty_blocks;

COUNT(*)----------

386544

SQL> alter session set events '10046 trace name context off';

Session altered.

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

17

48

User Trace File

• The user trace file is located in: /oracle/<SID>/saptrace/usertrace

select count(*) from dirty_blocksEND OF STMTPARSE

#1:c=0,e=1496,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=4,tim=4001773990916BINDS #1:EXEC #1:c=0,e=189,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4,tim=4001773991256WAIT #1: nam='SQL*Net message to client' ela= 2 p1=1650815232 p2=1 p3=0WAIT #1: nam='db file scattered read' ela= 199 p1=66 p2=24963 p3=8WAIT #1: nam='db file scattered read' ela= 86 p1=66 p2=24971 p3=8WAIT #1: nam='db file scattered read' ela= 82 p1=66 p2=24979 p3=8WAIT #1: nam='db file scattered read' ela= 78 p1=66 p2=24987 p3=8WAIT #1: nam='db file scattered read' ela= 78 p1=66 p2=24995 p3=8WAIT #1: nam='db file scattered read' ela= 78 p1=66 p2=25003 p3=8WAIT #1: nam='db file scattered read' ela= 78 p1=66 p2=25011 p3=8

49

Event 10046 Tracing Information

• Wait information: nam is what event is being waited onela is the elapsed time for the operation

In Oracle 7/8, this is in units of hundredths of a secondIn Oracle 9i onwards, this is in microseconds

p1 is the additional information for that wait event (e.g., for I/O waits the p1 value is the file number)p2 is the additional information for that wait event (e.g., for I/O waits the p2 value is the block number)p3 is the additional information for that wait event (e.g., for I/O waits the p3 value is the number of blocks read by the operation)

50

10046 Trace for an SAP User

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

18

51

10046 Trace for an SAP User (cont.)

52


• In this example, because we are tracing a session other than our own, we need to attach to the process and trace it

• To do this, the Oracle Process ID (PID) needs to be determined from the Oracle Session ID (SID) that is provided in the ST04 screen

select pid from v$process where addr =(select paddr from v$session where sid = 17);

PID----------

22

53


• Now that we have the Oracle PID, we can use another Oracle tracing method, ORADEBUG, to attach and trace what is happening with the process.

* See SAP Note 613872 for additional information on ORADEBUG

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

19

54

10046 Trace for an SAP User (cont.)SQL> oradebug setorapid 22Unix process pid: 23808, image: oracle@d1ua0002 (TNS V1-V3)SQL> oradebug unlimitStatement processed.SQL> oradebug event 10046 trace name context forever, level 12Statement processed.

Waited 30 seconds......

SQL> oradebug event 10046 trace name context offStatement processed.

55


• Below is the output from the 10046 trace of the SAP work process23732, which was running a SELECT on table ECMCA.

• Notice the elapse (ela) times to fetch the block from disk into the buffer cache are very fast!

WAIT #332: nam='db file sequential read' ela= 36 p1=39 p2=2209765 p3=1WAIT #332: nam='db file sequential read' ela= 32 p1=39 p2=2209766 p3=1WAIT #332: nam='db file sequential read' ela= 32 p1=39 p2=2209767 p3=1WAIT #332: nam='db file sequential read' ela= 31 p1=39 p2=2209768 p3=1WAIT #332: nam='db file sequential read' ela= 31 p1=39 p2=2209769 p3=1WAIT #332: nam='db file sequential read' ela= 31 p1=39 p2=2209770 p3=1WAIT #332: nam='db file sequential read' ela= 30 p1=39 p2=2209771 p3=1WAIT #332: nam='db file sequential read' ela= 9311 p1=39 p2=2209772 p3=1WAIT #332: nam='db file sequential read' ela= 34 p1=39 p2=2209773 p3=1WAIT #332: nam='db file sequential read' ela= 274 p1=39 p2=2209774 p3=1WAIT #332: nam='db file sequential read' ela= 33 p1=39 p2=2209775 p3=1WAIT #332: nam='db file sequential read' ela= 17972 p1=39 p2=2209808 p3=1

56

Tracing Background Processes

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

20

57

Background ORADEBUG

SQL> select count(*) from v$bh where dirty='Y';

COUNT(*)----------

11015

SQL> oradebug setospid 6072Oracle pid: 3, Unix process pid: 6072, image: oracle@d1ua0002 (DBW0)SQL> oradebug unlimitStatement processed.SQL> oradebug event 10046 trace name context forever, level 12Statement processed.SQL> alter system checkpoint;

System altered.

SQL> select count(*) from v$bh where dirty='Y';

COUNT(*)----------

23SQL> oradebug event 10046 trace name context offStatement processed.

58

Background 10046 Trace File

• Background trace file is located in:/oracle/<SID>/saptrace/background

WAIT #0: nam='db file parallel write' ela= 74067 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 69087 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 75128 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 72345 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 67329 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 75704 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 72595 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 78253 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 70311 p1=204 p2=0 p3=2147483647WAIT #0: nam='db file parallel write' ela= 61051 p1=152 p2=0 p3=2147483647WAIT #0: nam='rdbms ipc message' ela= 2563836 p1=256 p2=0 p3=0WAIT #0: nam='rdbms ipc message' ela= 3007678 p1=300 p2=0 p3=0WAIT #0: nam='rdbms ipc message' ela= 3007685 p1=300 p2=0 p3=0WAIT #0: nam='rdbms ipc message' ela= 3007723 p1=300 p2=0 p3=0WAIT #0: nam='rdbms ipc message' ela= 3007736 p1=300 p2=0 p3=0WAIT #0: nam='rdbms ipc message' ela= 3007715 p1=300 p2=0 p3=0

59



_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

21

60

SAP Transactions for Tuning: SM66

• Use SM66 as an indicator of normal SAP workload

61

SAP Transactions for Tuning: ST04

• Use Transaction ST04 Detailed Analysis Oracle Session as an indicator of a “normal” workload

62


• Use ST05 to trace any specific transactions that may be causing issues

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

22

63

SAP Transactions for Tuning: SE30

• Use SE30 to quickly identify if the problem is in SAP or the database layer

64


• Combination ST05 and SE30 (see SAP Note 755977)• Holds the history of each trace

65

SAP Transactions for Tuning: ST12 (cont.)

• Here we can see SE30 trace information from the ABAP Trace in ST12

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

23

66

SAP Transactions for Tuning: ST12 (cont.)

• Here we can see ST05 trace information from the ST12 trace

67

SAP Programs: RSORAVDV

• Use to see DBA and V$ view information• Run in SE38

68

SAP Programs: RSORADJV

• Use to access DBA and V$ view information• Run in SE38

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

24

69

SAP Programs: RSORADJV (cont.)

• Output from RSORADJV

70

SAP Programs: RSBDCOS0

• Output from RSBDCOS0 • Can be used to execute OS-level commands from SAP

71

OS Debugging Methods

• trussTrace Unix system callsExample:

truss –p <unix pid>• strace

Print streams informationstrace is available for Windows

• sarDisplays system activity (CPU, disk, network)Example:

sar 5 10• vmstat

Displays virtual memory statisticsExample:

vmstat 5 10

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

25

72

OS Debugging Methods (cont.)

• pingUsed to send network packets to hostExamples:

ping <hostname>ping –t <hostname>

• Use these OS tracing methods in conjunction with the 10046 tracing to confirm elapse times

73



74

Oracle Best Practices: Tablespaces

• Locally Managed Tablespaces (LMTS)Benefits:

Does not use the single ST Enqueue, which can cause performance issuesReduces DBA’s workload by configuring auto allocate or uniform extent sizes

Disadvantages:Still uses pct_used and pct_free, which are not efficientConcurrent inserts are still sent to the same free block on the freelist

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

26

75

Oracle Best Practices: Tablespaces (cont.)

• ASSM tablespacesBenefits:

Same benefits as listed on previous slide for LMTSNo longer uses the pct_used and pct_free; instead, manages the block filling based on the percentagesConcurrent inserts are automatically sent to different blocks

76

Oracle Best Practices

• AUMAutomatically configures the number of undo segments neededMinimize ORA-1555 by keeping the changed data in the undo segments based on a retention time

• PGA tuningAllows the PGA to allocate memory up to a certain value that is greater than the parameters sort_area_size and hash_area_sizeMinimizes sorts operations (I/O) to PSAPTEMP

77

Oracle Best Practices (cont.)

• Auto extendDefine only one datafile for each tablespace to auto extend in the event that the DBA cannot add space in time. This will reduce ORA-1653 and ORA-1654 space errors.

• Using parallel query Use parallel query servers to create indexes fasterUse parallel query servers to create tables faster when using “create table as select” commands

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

27

78

Oracle Best Practices (cont.)

• SPFILEUsing the SPFILE aids in the managing of the database and the parameters that are changed

• Analyze table/index <name> validate structure onlineCan be used to look for corruption without putting a lock on thetable, which happened in previous Oracle versions

• Online table reorganizationTables that do not have LONG or LONG RAW columns can now be reorganized online using DBMS_REDEFINITION package. This method is based on materialized views technology.

79



80

Resources

• Oracle/SAP Newsletterwww.oracle.com/newsletters/sap/current.html

• Wait Events SAP Note 619188service.sap.com/notes *

* Requires login credentials to the SAP Service Marketplace

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

28

81

7 Key Points to Take Home

• The most efficient I/O done, besides blocks already residing in cache, will be 3-4 blocks per row return

• Proactively monitor the system for inefficient SQL statements using ST04 and ST05 transactions

• Use the CBO 10053 tracing to understand why the optimizer is picking the index in the execution plan

• Always determine what event is causing users to wait

82

7 Key Points to Take Home (cont.)

• Use wait event tracing 10046 to prove if any background or foreground process is slowing down due to internal latching or because of I/O issues

• Use SAP transactions ST12, SE30, ST05, SM66, and ST04 to quickly identify the area of waits

• Set up the Oracle/SAP database per SAP’s Best Practices Standards (ASSM, AUM, SPFILE, etc.)

83

Your Turn! Questions?

How to contact me:Mike Kennedy

[email protected]

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

_________________________________

Notes: ______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

Notes: ______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

Notes: ______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

Wellesley Information Services, 990 Washington Street, Suite 308, Dedham, MA 02026

Copyright © 2008 Wellesley Information Services. All rights reserved.

Date post:	07-Mar-2015
Category:	Documents
Upload:	ufuk-yavuz
View:	189 times
Download:	3 times

Advanced Tuning Oracle

Documents