Tuning Oracle 10GR2 for SIEBEL Applications DOAG print · Tuning Oracle 10GR2 for SIEBEL...

Basel · Baden · Bern · Lausanne · Zurich · Düsseldorf · Frankfurt/M. · Freiburg i. Br. · Hamburg · Munich · Stuttgart · Vienna

Tuning Oracle 10GR2 for SIEBEL Applications

Milen [email protected]

Deutsche Oracle Anwenderkonferenz 2007

Nürnberg, 21-22 November 2007

Performance Tuning Oracle 10GR2 for SIEBEL Applications 2 © 2007

Agenda

Data are always part of the game.

� Introduction

� Instance Tuning

� Gathering and manipulating statistics

� SQL Tuning

� Stability of execution plans

� Simple solutions

� Conclusions


� SIEBEL Applications is a business-critical large-scale application. Itsperformance should be kept as stable as possible

� SIEBEL Database Design is enterprise-wide� The same set of tables (~ 2500) and indexes (~15000) are used by all

SIEBEL modules (Call Center, Financials, HealthCare etc)� Therefore the generated SQL queries are pretty complex

� Usually 30-40 tables are joined� Up to 60-70 table joins are not unusual� Many outer joins - almost not tunable, since the table join order must be preserved

� SIEBEL Applications is a pretty interactive application� What counts to the end-user is how fast the first lines come on the screen

� SIEBEL Applications is confugured through a special SIEBEL Configuration Tool� Code must be kept as generic as possible to preserve the idea of database

independence� No knowledge of the Orace specialties (function-based indexes, bitmap

indexes, reverse indexes, partitioning etc)

What makes SIEBEL Applications so special?


Agenda


� Introduction

� Instance Tuning


� SQL Tuning


� Simple workarounds

� Conclusions


� SIEBEL Apps server sends SQLs with a mixture of literals and bind variables

� Official SIEBEL recommendation:� CURSOR_SPACE_FOR_TIME=TRUE� Pins not only cursor head (SQL statement itself), but also the

execution plans of the statements in the Shared Pool� SHARED_POOL should be sized appropriately� Creates very long lists of chunks in Shared Pool

� increases CPU consumption� „shared pool latch“ is held longer

� Under „hard parse storm“ load causes

SharedPool Tuning (1)

ORA-04031: unable to allocate 4160 bytes of shared memory("shared pool","unknown object","sga heap(1,0)","kglsim heap")


� As of 9i the official recommendation changed� CURSOR_SPACE_FOR_TIME=FALSE� Shared Pool management was vastly improved� Oracle splits Shared Pool in subheaps, if SHARED_POOL parameter is set

big enough� the number of subheaps depends on the size of SHARED_POOL and CPU count� each subheap is managed separately as if the subheap represents the whole

Shared Pool� separate latches („row cache objects“) for each subheap

� It is impossible to control to which subheap each SQL or PL/SQL code will beassigned� ORA-04031 still possible

� _KGHDSIDX_COUNT controls the number of subheaps� 2-3 subheaps are enough, but each subheap should be at least 400MB

� DB_CACHE_ADVICE = OFF reduces Shared Pool fragmentation further� DB_CACHE_ADVICE = READY and then could be changed online if needed

SharedPool Tuning (2)


� SIEBEL Applications obeys „first records should come instantly“ principle� Almost all accesses are index-oriented� A couple of indexes of the central data model tables (like

S_CONTACT, S_EVT_ACT, S_PARTY, S_ORG_EXT) are used overand over again(concurrently)

� These indexes are ideal candidates for KEEP cache pool� Assumes static configuration of DEFALT database cache, and

SHARED_POOL� Some tables (S_CONTACT, for example) are read anyway entirely, but

over index� this is a consequence of the configuration

� optimizer_mode = first_rows_10� optimizer_index_cost_adj=1..25

� KEEP Cache should be properly sized (size of tables + size of indexes) plus some reserve for the future (20% )

Database Cache Tuning


Important database configuration parameters (1)

� The ultimate goal of database configuration is to deliver the first row on the users screen as fast as possible� This doesn‘t necessarily mean that the database resources should be used

in the most optimal way� CBO must be forced to

� use NESTED LOOPS for joins� force CBO to prefer index accesses

� Right after creation of each session SIEBEL Application Server issuesthe following SQL statements� Could not be overridden with database LOGON trigger

alter session set optimizer_mode = first_rows_10;alter session set hash_join_enabled = false;alter session set "_optimizer_sortmerge_join_enabled" = false;alter session set "_optimizer_join_sel_sanity_check" = true;


� Reducing the (hard) parse times� SIEBEL documentation suggests setting

optimizer_max_permutations =100, but this parameter is obsolete in Oracle 10G� Another parameter „_optimizer_max_permutations“ exists, is accepted,

but has no noticeable influence on hard parse time� The most critical parameter influencing the parse time is

� _b_tree_bitmap_plans=FALSE� When set a parse time of a complex SQL query ( 48 tables) dropped

from 850 seconds to 20 ms.� Another parameters reducing further hard parse time

� _always_semi_join=OFF� _always_anti_join=OFF

Important database configuration parameters (2)


� SIEBEL Documentation suggests using� optimizer_index_caching=0� optimizer_index_cost_adj=1

� There are several problems with this settings� Too far from the reality. Index access cost (in terms of ms access time) is

pretty far from 1/100 of tables block access cost� Many indexes get equally attractive (Cost=1.00 or value very close to 1). In

this case the CBO sorts all equally attractive indexes in alphabetical order and picks the first one.

� This means that index name can influence the attractivity of an index!� Used to force CBO to favor index access at almost any cost e.g. to

simulate the RBO behavior

� Settings that have proven themselves in practice� optimizer_index_caching=75� optimizer_index_cost_adj=25� db_file_multiblock_read_count=8

Parameters influencing optimizer decisions (1)


� The eliminate further to possibility CBO to take „wrong“ decisions, the following list of parameters should be set:

� _always_semi_join=OFF � _always_anti_join=OFF � _push_join_predicate=FALSE � star_transformation_enabled=FALSE� _optimizer_cost_based_transformation=OFF� _optimizer_push_pred_cost_based=FALSE

� Parameters � _always_semi_join=OFF � _always_anti_join=OFF

� force CBO to merge IN() and NOT IN() subqueries into the mainquery, replacing SEMI or ANTI joins with ordinary NESTED LOOPs

� If CBO underestimates cardinality of the subqueries, an excessive amount of (CPU) time is consumed at run-time

� reduce the hard-parse time

Parameters influencing optimizer decisions (2)


� If the database is write-intensive , DBWR and LGWR processes shouldbe configured properly� DBWR

� db_writer_processes=CPU_COUNT/2 if the filesystem doesn’t support async I/O� LGWR

� log_buffer=18MB or even more (always rounded to “multiple of granule size” –2MB), especially if many (or big) batch jobs are running

� COMMIT_WRITE=BATCH, but only for “repimpexp” executable (via logon trigger)

� Using function-based indexes� Help resolving SQL queries having ORDER BY .. DESC clause

� query_rewrite_integrity=TRUSTED� query_rewrite_enabled = TRUE

� Further reducing soft parses� session_cached_cursors=300� „Softer“ soft parses

Another important parameters


Agenda


� Introduction

� Instance Tuning


� SQL Tuning



� Conclusions


� Production SIEBEL databases are very sensitive about database statistics changes� too many session and instance level parameters skew the picture from CBO� Should not be gathered too frequently

� Every 2-6 months depending on the situation, releases� Should be saved first (with DBMS_STATS.EXPORT_SCHEMA_STATS, for

example) before refreshing them (Oracle 10g does this anyway)� Gathering statistics only on indexed columns (without histograms) is a good

start� Presence/absence of histograms on specific columns can determine the

performance of specific or whole class of SQL queries� For some tables (EIM_*) it is better NOT to have statistics� Oracle uses hard-coded defaults :AVG_ROW_LEN=100, CLUF =800

� OPTIMIZER_DYNAMIC_SAMPLING =1� Statistics on function-based indexes should be gathered explicitly� Manipulating NUM_DISTINCT column statistic is usually enough

Gathering statistics


Gathering statistics - sample

--1. Gather schema statsDBMS_STATS.GATHER_SCHEMA_STATS(OWNNAME=>'SIEBEL', ESTIMATE_PERCENT=>100, DEGREE=>4, CASCADE=> TRUE, METHOD_OPT=>'for all indexed columns size 1');

-- 2. Special histogramsDBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>'SIEBEL', TABNAME=>'S_ADDR_PER', CASCADE=>TRUE,METHOD_OPT=>'for column ZIPCODE size 254 ');

-- 3. Delete statistics DBMS_STATS.DELETE_TABLE_STATS(OWNNAME=>'SIEBEL',TABNAME=>'EIM_CONTACT');

-- 4.Manipulate statisticsDBMS_STATS.SET_COLUMN_STATS(OWNNAME=>'SIEBEL', TABNAME=>'S_PARTY_REL', COLNAME=>'PARTY_ID' ,DISTCNT=>700000);


Agenda


� Introduction

� Instance Tuning


� SQL Tuning



� Conclusions


� Some of session and instance level parameters constrain the optimizerchoices and skew the real picture of the data� Session parameters

� Instance parameters

� That is why sometimes additional changes need to be made� Create stored outlines� Manipulate statistics

SQL tuning

alter session set optimizer_mode = first_rows_10;alter session set hash_join_enabled = false;alter session set "_optimizer_sortmerge_join_enabled" = false;alter session set "_optimizer_join_sel_sanity_check" = true;

optimizer_index_caching=75optimizer_index_cost_adj=25_always_semi_join=OFF _always_anti_join=OFF


A problematic SQL

SELECTT8.CONFLICT_ID,T8.LAST_UPD,T8.CREATED,… --- ~ 200 columns

FROM …SIEBEL.S_PARTY_REL T13,…SIEBEL.S_ORG_EXT T18

WHERE T18.PR_ADDR_ID = T17.ADDR_PER_ID (+) AND …T15.BU_ID = T12.ROW_ID (+) ANDT15.BU_ID = T3.PAR_ROW_ID (+) AND T13.REL_PARTY_ID = T8.ROW_ID AND((T18.INT_ORG_FLG != 'Y' OR T18.PRTNR_FLG != 'N') AND T18.ACCNT_FLG != 'N') AND(T13.PARTY_ID = :1)

ORDER BY T18.NAME, T18.LOC ;


The situation (complex execution plan)


Symptoms caused by execution plan

� Non-selective Index for the first execution step� very high logical I/Os => high CPU consumption => bad response time


� The parameters make CBO to favour index access on tables in order to avoid SORT operation in the executon plan� „first row should be presented to user as fast as possible“ principle

� Table S_ORG_EXT seems attractive to CBO because� It has many predicated on it, although all of them are not selective (97% of

all table rows)� The presence of index on S_ORG_EXT(NAME, LOC) + ORDER BY on

S_ORG_EXT(NAME, LOC) + optimizer_mode = first_rows_10

The reasons

optimizer_mode = first_rows_10optimizer_index_caching=75optimizer_index_cost_adj=25

… AND((T18.INT_ORG_FLG != 'Y' OR T18.PRTNR_FLG != 'N') AND T18.ACCNT_FLG != 'N') AND

(T13.PARTY_ID = :1)ORDER BY T18.NAME, T18.LOC ;


� The acces on the newly created index on S_PARTY_REL. PARTY_ID should be made attractive� Creating index

� Index hint (stored outline)

� Manipulating statsitics

The Solution

create index siebel.idx_S_PARTY_REL_MKU1 on siebel.S_PARTY_REL(PARTY_ID) compute statistics;

exec dbms_stats.SET_COLUMN_STATS(ownname=>'SIEBEL', tabname=> 'S_PARTY_REL', colname=>'PARTY_ID' , distcnt=>700000 );

SELECT /*+ index (T13 IDX_S_PARTY_REL_MKU1) */


The difference �

� The index with the most selective predicate is chosen

� S_ORG_EXT is accessed over the selective PAR_ROW_ID column


� The response time dropped from ~ 2 minutes to < 1 seconds� Response time improvement by factor of at least 120

� Logical reads dropped from ~ 20 Mio to 10k� Reduced bufer cache contention� Reduced CPU consumption

� Physical reads for this SQL statement vanished� Reduced cache pollution of DEFAULT and KEEP Pool

� The big table S_ORG_EXT (690MB) and its index S_ORG_EXT_M57 (100MB) are not scanned

� S_ORG_EXT is accessed over pretty selective S_ORG_EXT_U3 index� S_PARTY_REL table (80MB) is (via index) accessed instead

� Sorting data� Sort operation is not a problem, since the resultset to be sorted

consists of only 10-20 rows

Results


Agenda


� Introduction

� Instance Tuning


� SQL Tuning



� Conclusions


� Stored Outlines help to „freeze“ the execution plan� Require increased administration , since after each new application

release the SQL statements generated by SIEBEL Apps could beslightly different (new column, reordering of columns), this ignoringthe stored outlines

� spaces, tabs, newline constraints were relaxed in 9i � The most frequently used hints are INDEX, CARDINALITY and

LEADING� In 10gR2 LEADING hint accepts many parameters, allowing us to specify

not only the first table to be used, but also the second, third etc tables� /* +LEADING (T1 T5 T11 T12 T7) */

� Could be tested on integration/test system and then via exp/imptransferred to the production database

� SQL Profiles could also be used, but they are subject of additional licence that should be obtained

Stored Outlines


Agenda


� Overview

� Instance Tuning


� SQL Tuning


� Simple Workarounds

� Conclusions


Activities Screen


� Quite often user wants to see last records of one of the main screens (Activities, Contacts, Accounts etc)� Tables S_EVT_ACT , S_CONTACT are usually big ( a couple of

million records)� SIEBEL Application Server sends SQL request to Oracle � Oracle gets all records from the main table (S_EVT_ACT, for

example), sorts them, so that that last records are presented to the user first, and sends to result to the Application Server

� The User scrolls a couple of pages down (each page is ~ 30 records), until he/she (eventually) finds the needed information

� User switches to another screen, thus using maximal first sorted 1000 records. The rest (a couple of million records) will be discarded and will never be used by the user� Only 4-5 of user sessions of this type could bring the whole database to

knees� The only remedy so far is to force index usage to avoid the monster sort

The Problem


“The patch” – additional menu for time selection


� Benefits� ResultSet is constrainted as early as possible (via predicate)� S_EVT_ACT could be partitioned (partition pruning)� No unnessary polution of cache with index blocks on

S_EVT_ACT.CREATED)� Stored outlines /SQL Profiles are not needed anymore

The “pacthed” SQL

SELECT T23.CONFLICT_ID,.... --- another ~ 250 columnsFROM

...SIEBEL.S_EVT_ACT T23

WHERE...

(T23.PRIV_FLG=:s3 OR T23.PRIV_FLG IS NULL AND(T21.CON_ID=:s5)AND T23.CREATED >= :s6ORDER BY T23.CREATED DESC;


Alternative „patch“ – dummy query + saved queries


Core Messages…

Knowledge transfer is only the beginning. Knowledge application is what counts.

� Correctly configuring Oracle forSIEBEL is not an easy task� Can be pretty rewarding, though

� In-depth knowledge and understandingof Oracle and especially CBO isrequired to get optimal performance, especially if the application is heavilycustomized

� Knowledge is good, but the real poweris the combination of knowledge and experience

Basel · Baden · Bern · Lausanne · Zurich · Düsseldorf · Frankfurt/M. · Freiburg i. Br. · Hamburg · Munich · Stuttgart · Vienna

Thank you!

?www.trivadis.com

Date post:	22-Mar-2020
Category:	Documents
Upload:	others
View:	11 times
Download:	0 times

Tuning Oracle 10GR2 for SIEBEL Applications DOAG print · Tuning Oracle 10GR2 for SIEBEL...

Documents