PRACTICAL APPLICATION PERFORMANCE TUNING: APPLYING … · 2019. 12. 6. · PROFILE DB TIME USING...

PRACTICAL APPLICATION PERFORMANCE TUNING: APPLYING THEORY IN PRACTICEDAVID KURTZUKOUG TECHFEST 19

Copyright © 2019 Accenture All rights reserved. | 2

WHO AM I

Accenture Enkitec Group

Performance tuning

• PeopleSoft ERP

• Oracle RDBMS

Book

• blog.go-faster.co.uk

• blog.psftdba.com

OakTable


WHY ARE WE HERE?

Fundamental theory

• How does a feature work?

• What are the underlying mechanics?

• Make reasonable predictions about how it might work for me

A practical example of performance tuning

• Did it work as we expected?

• Or did something go wrong?

Copyright © 2019 Accenture All rights reserved. |

• Used lots of database features

• Need to understand

• the database feature before using it,

• Why might it help us?

• how far can you push the feature?

• what does too far look like?

• How different database features might interact with each other?

• How the feature will interact with the application?

• How can you configure the application to take advantage of the feature?

• http://blog.psftdba.com/2017/10/nvision-performance-tuning.html

4

WHY THIS CASE STUDY?

Lots of different database features:

• Instrumentation

• Partitioning

• Hybrid Columnar Compression

• Materialized Views

• Bloom Filter

• Statistics

• Parameter Settings

• Parallel Query

• SQL Profiles

http://blog.psftdba.com/2017/10/nvision-performance-tuning.html


Case Study

• Insurance company

• PeopleSoft Financials

• GL (nVision) reporting

• Oracle 12.1.0.2

• 2-node RAC

• Supercluster (an engineered system)

• Performance Objective

• Suite of ~600 GL reports

• Run every night

• Complete by start of US east coast day

5

A CASE STUDY

It helps if you understand a bit about the application.

• Nobody expects the DBA to be an accountant or a functional specialist

• Need a common language with users

Oracle:

• DBA ➔ Data Scientist

• Getting involved with applications and developers

• But that should be the case anyway


Think of GL a fact table in a data warehouse

• Every transaction creates (or contributes to) two GL entries.

• Dimensions:

• Ledger

• Fiscal Year and Accounting Period

• Currency

• Hierarchical attributes: Account, Department, Business Unit etc

• Example: Account:

• Various related accounts grouped together

• Electricity, Gas, Oil, Other Heating Costs

6

GENERAL LEDGER REPORTING

GL reporting is a typical example of data warehouse queries

• Some companies ETL GL to a data warehouse

• But, accountants like a real-time view at month-end

Fact tables

• Ledger, Budget Ledger, Summary Ledgers

Dimensions

• Various hierarchical attributes

• In PeopleSoft, the hierarchies are described by 'Trees'


INSTRUMENTATION


ADDITIONAL INSTRUMENTATION

PeopleSoft Application Instrumented

• Calls DBMS_APPLICATION_INFO.SET_MODULE

• Enable in Configuration

• PIA: Component/Page

• AE: Action=Program.Section.Step.StepType

COBOL/SQR/nVision not instrumented

• Psftapi package/trigger: http://www.go-faster.co.uk/scripts.htm#psftapi.sql

• Trigger on PSPRCSRQST fires when process sets its own status to 7=Processing

• Module=Process Name, Action=Process Instance

• Stores Process Instance in package global variable (session private)

• Further enhanced in nVision with FGA and PL/SQL Handler…

http://www.go-faster.co.uk/scripts.htm#psftapi.sql


FINE-GRAINED AUDIT

nVision reads parameters from PS_NVS_REPORT on startup

FGA handler extracts Layout ID and Business Unit AND adds it to Action

Hence I can obtain Report ID

BEGIN

DBMS_FGA.ADD_POLICY(

object_schema => 'SYSADM',

object_name => 'PS_NVS_REPORT',

policy_name => 'PS_NVS_REPORT_SEL',

handler_schema => 'SYSADM',

handler_module => 'AEG_FGA_NVISION_HANDLER',

enable => TRUE,

statement_types => 'SELECT',

audit_trail => DBMS_FGA.DB + DBMS_FGA.EXTENDED);

END;

/


MODULE & ACTION ➔ ACTIVE SESSION HISTORY

Process Business

MODULE ACTION CLIENT_ID Instance Unit REPORT_ID

------------ -------------------------------- ------------ --------- -------- ----------

RPTBOOK PI=1780508:USGL113S:US001 USNVISION 1780508 US001 USGL113S

RPTBOOK PI=1780509:USGL010E:US002 USNVISION 1780509 US002 USGL010E

RPTBOOK PI=1780502:USGL91PF:US001 USNVISION 1780502 US001 USGL91PF

RPTBOOK PI=1780502:USGL91FR:US001 USNVISION 1780502 US001 USGL91FR

RPTBOOK PI=1780502:USGL91US:US001 USNVISION 1780502 US001 USGL91US

RPTBOOK PI=1780502:USGL91DM:US002 USNVISION 1780502 US002 USGL91DM

RPTBOOK PI=1780506:USEXP2AM:US001 USNVISION 1780506 US001 USEXP2AM

RPTBOOK PI=1780500:USGL113S:US003 USNVISION 1780500 US003 USGL113S

RPTBOOK PI=1780509:USGL010E:US000 USNVISION 1780508 US000 USGL010E


OTHER RECOMMENDATIONS

Index FGA table

CREATE INDEX sys.fga_log$_obj$name

ON sys.fga_log$ (obj$name, sessionid, entryid)

Regularly purge FGA table

Move SYS.AUD$ and SYS.FGA_LOG$ from the SYSTEM tablespace to another ASSM tablespace using the instructions in Oracle support note 1328239.1

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1328239.1


PROFILING DB TIME USING ASH


PROFILE DB TIMEUSING ASH

Instrumentation create a relationship between the application and monitoring information in ASH and SQL Statement statistics

Look for statements / execution plans / module / actions that consume the most time

Either

• within all nVision report programs

• Or within a specific nVision Reportbook (group of reports run in a single process) that we know takes a long time from Process Scheduler data

• Or from specific report (extracted from Action)

Check that what you find matches what the users report!


PROFILE DB TIME FOR NVISION REPORTWITH x AS (

SELECT /*+leading(x h) use_nl(h) */ r.prcsinstance, h.event, h.sample_id

, CASE WHEN h.module IS NULL THEN REGEXP_SUBSTR(h.program, '[^@]+',1,1)

WHEN h.module LIKE 'PSAE.%' THEN REGEXP_SUBSTR(h.module, '[^.]+',1,2)

ELSE REGEXP_SUBSTR(h.module, '[^.@]+',1,1) END AS module

, h.action, h.sample_time, h.sql_id, h.sql_plan_hash_value, h.force_matching_signature, h.sql_exec_id, p.sql_id sql_id2

FROM dba_hist_snapshot x

, dba_hist_Active_Sess_history h

LEFT OUTER JOIN dba_hist_sqltext p ON p.sql_id = h.sql_id

, psprcsrqst r

WHERE x.dbid = h.dbid

AND x.instance_number = h.instance_number

AND x.snap_id = h.snap_id

AND x.end_interval_time >= r.begindttm

AND x.begin_interval_time <= NVL(r.enddttm,SYSDATE)

AND h.sample_time BETWEEN r.begindttm AND NVL(r.enddttm,SYSDATE)

AND (UPPER(program) LIKE 'PSNVS%' OR program like 'oracle%')

AND r.oprid = 'XXRUNCNTRL'

AND r.runcntlid = 'NVS_RPTBOOK_42'

AND r.prcsinstance = REGEXP_SUBSTR(h.action,'[[:digit:]]+')

AND h.action like 'PI=%:%:%' /*process instance number*/

AND x.begin_interval_time >= TRUNC(SYSDATE)-1

), y as(

SELECT prcsinstance, sql_plan_hash_value, force_matching_signature, sql_id, module, action

, count(distinct sql_id||sql_exec_id) execs

, sum(10) ash_secs

, 10*count(sql_id2) awr_secs

, 10*count(distinct sample_id) elapsed_secs

, min(sample_time) min_sample_time, max(sample_time) max_sample_time

FROM x

GROUP BY prcsinstance, sql_id, sql_plan_hash_value, force_matching_signature, module, action

)…

blog.psftdba.com @davidmkurtz


PROFILE DB TIME FOR NVISION REPORT

…

, z1 as (

select /*+MATERIALIZE*/ prcsinstance, row_number() over (partition by prcsinstance, force_matching_signature, sql_plan_hash_value order

by awr_secs desc) fms_seq

, module, action, force_matching_signature, sql_id, sql_plan_hash_Value

, count(distinct sql_id) over (partition by prcsinstance, force_matching_signature, sql_plan_hash_value) sql_ids

, sum(execs) over (partition by prcsinstance, force_matching_signature, sql_plan_hash_value) fms_execs

, sum(ash_secs) over (partition by prcsinstance, force_matching_signature) fms_ash_secs

, sum(awr_secs) over (partition by prcsinstance, force_matching_signature, sql_plan_hash_value) fms_awr_secs

, sum(elapsed_secs) over (partition by prcsinstance, force_matching_signature, sql_plan_hash_value) elap_secs

from y

), z as (

Select z1.*

, (select /*+RESULT_CACHE*/ object_name

from dba_hist_sql_plan p

where p.plan_hash_value = z1.sql_plan_hash_value

AND p.object_name IS NOT NULL

AND not p.object_name LIKE 'PSTREESELECT__'

AND p.object_type IN('MAT_VIEW REWRITE','TABLE')

AND rownum = 1) table_name

from z1

where fms_seq = 1

)

Select z.* from z

where fms_ash_secs >= 30

ORDER BY prcsinstance, fms_ash_secs DESC, force_matching_Signature, elap_secs DESC



PROFILE OF TOP QUERIES BY FMS AND PLAN

FMS FMS

Force Matching SQL Plan SQL FMS ASH AWR Elapsed

FMS_SEQ MODULE ACTION Signature SQL_ID Hash Value IDs Execs Secs Secs Seconds TABLE_NAME

---------- ------------ -------------------------------- --------------------- ------------- ---------- ------ ------ ------- ------ ------- ------------------

1 RPTBOOK PI=3546157:GLXX123N:10008 11748073145320522155 bd4t1fjhaq9w5 752306875 3 60 12090 11720 11720 PS_FT_SUM_XXXXX_MV

1 RPTBOOK PI=3546157:GLXX123Q:10008 8hphv1gytafak 3953228627 1 2 12090 370 370 PS_FT_SUM_XXXXX_MV

1 RPTBOOK PI=3546157:GLXX123N:10008 6172504123947024582 awd164fz5bbay 2969036184 17 248 11580 7740 5190 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123D:10008 711mk5kh1k1qm 1067239509 2 59 11580 170 1340 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123Q:10008 7vgbybgsyprv9 0 1 1 11580 0 20

1 RPTBOOK PI=3546157:GLXX123Q:10008 7vgbybgsyprv9 1123984571 1 1 11580 0 20

1 RPTBOOK PI=3546157:GLXX123Y:10008 16625752171077499412 033mkuqw7ayzg 3799839508 238 238 9820 0 2470 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123R:10008 cnrnznp0b8dmh 655052913 60 60 9820 0 600 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123Y:10008 c2jb189gs67y5 4219255878 37 37 9820 0 370 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123M:10008 08rjnhd7caj1j 3962810546 33 33 9820 0 330

1 RPTBOOK PI=3546157:GLXX123R:10008 0bbtnsh57118a 0 25 25 9820 0 250

1 RPTBOOK PI=3546157:GLXX123Y:10008 3yhbt7y615xbt 2364386617 9 9 9820 0 90 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123R:10008 08v42qtstm5uc 3779330357 5 5 9820 0 50 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123R:10008 4y8zb6j0f09ah 1391982532 2 2 9820 0 20

1 RPTBOOK PI=3546157:GLXX123M:10008 18328048413129054545 gyawtzvdywq17 3576637887 286 286 9100 0 2860 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123M:10008 4m5frz30vr84x 2513743895 76 76 9100 0 760

1 RPTBOOK PI=3546157:GLXX123R:10008 cst84drr7uz07 200447031 71 71 9100 0 710 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123Y:10008 1cnhh1ysxdvx8 1031631451 17 17 9100 0 170 PS_S_LEDGER_XXXXX

1 RPTBOOK PI=3546157:GLXX123R:10008 0nzf6202akwxk 1426946062 1 1 9100 0 10 PS_S_LEDGER_XXXXX

…

********************* ------ -------

sum 1120 35120


WHAT IS THE APPLICATION DOING?


INTRODUCTION TO NVISION

• PeopleSoft Query plug-in to Excel

• Most commonly used in General Ledger report

• but can be used in any PeopleSoft application.

• In GL

• Ledger is a data warehouse of financial information

• Financial Amounts

• Business Attributes

• Hierarchical, held in trees.

• Business Unit, Account, Dept ID etc

• Time Attributes: Fiscal Year & Accounting Period


BUSINESS_UNIT

CHARTFIELD1 PSTREESELECT05

PSTREESELECT10 PSTREESELECT10PS_LEDGER

FISCAL_YEARACCOUNTING_PERIOD

CURRENCY_CD

ACCOUNT

20

GL LEDGER REPORTING DATA MODEL


NVISION SQL

SELECT L.TREE_NODE_NUM,L2.TREE_NODE_NUM,SUM(A.POSTED_TOTAL_AMT)

FROM PS_LEDGER A

, PSTREESELECT05 L1

, PSTREESELECT10 L

, PSTREESELECT10 L2

WHERE A.LEDGER='ACTUALS'

AND A.FISCAL_YEAR=2019

AND A.ACCOUNTING_PERIOD BETWEEN 1 AND 11

AND L1.SELECTOR_NUM=30982 AND A.BUSINESS_UNIT=L1.RANGE_FROM_05

AND L.SELECTOR_NUM=30985 AND A.CHARTFIELD1=L.RANGE_FROM_10

AND L2.SELECTOR_NUM=30984 AND A.ACCOUNT=L2.RANGE_FROM_10

AND A.CURRENCY_CD='USD'

GROUP BY L.TREE_NODE_NUM,L2.TREE_NODE_NUM

PS_LEDGER

Dimension trees extracted to

tree selector table

PSTREESELECT05

because 5 character

dimension attribute

Always specify Ledger, FY

AND AP

Dimension extracts uniquely

identified by selector number

Selectors joined to fact table

Other filters

Aggregate attributes


A NON-TRIVIAL PROBLEM

Tree Selectors are populated at nVision and PS/Query runtime

• (when PS/Query uses the IN TREE operator)

Many different selectors on different trees, on different ledger dimension columns with the same length end up in same table

• Static selectors: Extracted once, and kept (until the tree changes)

• Dynamic selectors: Extracted every time report runs, and deleted when nVision ends.

• But debris can be left behind if an nVision terminates abnormally.

• Application does not maintain the optimizer statistics.

• Different selectors on different portions or different trees → skew

• The number of dimensions, and therefore the number of selector joins, can vary.

• Then you need to look up rows on ledger using various combinations of different columns.


PARTITIONING LEDGER, LEDGER BUDGET, SUMMARY LEDGER TABLES


ORACLE PARTITIONING

Only available on Oracle Enterprise Edition

Cost Option

Can partition in 2-dimensions only

A single period query will only reference that one partition

A year-to-date query will only reference those partitions in that year


CANDIDATE PARTITION COLUMNS

nVision queries always contain predicates on

• LEDGER,

• FISCAL_YEAR

• ACCOUNTING_PERIOD

SELECT … SUM(A.POSTED_BASE_AMT)

FROM PS_LEDGER A, …

WHERE A.LEDGER='ACTUALS'


AND A.ACCOUNTING_PERIOD BETWEEN 1 AND 11

…


RANGE PARTITIONING

Most customers have monthly accounting periods.

• 0 = brought forward

• 1-12 = monthly periods

• 998, 999 = carry forward and adjustments (now just 995)

Range partition on combination of FISCAL_YEAR and ACCOUNTING_PERIOD.

• Two columns in one range

• 14 periods per year, for last year, current year and future years

• Annual partitions for prior years

• Same approach for LEDGER, LEDGER_BUDG and summary ledger tables.


SUBPARTITIONING?

It depends on the data

• LEDGER column is often a good candidate. (This is what we did)

• A single value list sub-partition for each larger ledger,

• A default sub-partition for the rest.

• BUSINESS_UNIT

• In a multi-national system list partitions containing the top business units for each region, and one for the rest.

• Improvement reporting and batch processes by having different regions mostly posting to different partitions.

• Can have different sub-partitions in different partitions, so can change in different FISCAL_YEARs as the business changes.

• We didn't sub-partition summary ledgers.


HCC COMPRESSION ON ENGINEERED SYSTEM

CREATE TABLE sysadm.ps_ledger (...)

TABLESPACE GLLARGE

PCTFREE 10 PCTUSED 80

PARTITION BY RANGE(FISCAL_YEAR,ACCOUNTING_PERIOD)

SUBPARTITION BY LIST (LEDGER)

(PARTITION ledger_2016 VALUES LESS THAN (2017,0) PCTFREE 0 COMPRESS FOR QUERY HIGH

,PARTITION ledger_2017 VALUES LESS THAN (2018,0) PCTFREE 0 COMPRESS FOR QUERY HIGH

,PARTITION ledger_2018_bf VALUES LESS THAN (2018,1) PCTFREE 0 COMPRESS FOR QUERY LOW

…

,PARTITION ledger_2019_11 VALUES LESS THAN (2019,12)

,PARTITION ledger_2019_12 VALUES LESS THAN (2019,13)

,PARTITION ledger_2019_cf VALUES LESS THAN (2020,0)

…

) ENABLE ROW MOVEMENT

/

Previous years compressed

query high

Last year compressed query

low

Current/Future partitions not

compressed


PARTITION MAINTENANCE

Add new partitions each fiscal year

• I do not create MAXVALUE partitions, thus forcing need to add partition, rather than forget to do it

• Budgets are usually created a year ahead.

• Now also in same LEDGER table in latest version of product

• so advantage in sub-partitioning on LEDGER column

As partitions become historical and application ceases to post to them

• Rebuild

• Do not reserve free space: PCTFREE 0

• Introduce Compression

• Increase HCC for older years? QUERY LOW→HIGH?

• Coalesce monthly partitions into 1 annual partition?

• Collect and Lock Statistics?

Archive/Purge data by exchanging/dropping old partitions

Usually locally partitioned indexes (except for non-partitioned global unique index)


PARTITION SCRIPTING

Warning: You are probably facing into manual scripting

In PeopleSoft, partitioning is a frequent challenge, and it is platform specific.

• Financials: GL

• HR: Global Payroll, and Time and Labor

PeopleSoft partition maintenance in Application Designer is poor

• http://blog.psftdba.com/2015/02/peopletools-854-tableindex-partitioning.html

• Application Designer can preserve existing partitioning

• Use GFC_PSPART package to generate partition DDL

• Utility: https://github.com/davidkurtz/gfcbuild

• Documentation: http://www.go-faster.co.uk/gfc_pspart.manual.pdf

http://blog.psftdba.com/2015/02/peopletools-854-tableindex-partitioning.html

https://github.com/davidkurtz/gfcbuild

http://www.go-faster.co.uk/gfc_pspart.manual.pdf


HYBRID COLUMNAR COMPRESSIONMATERIALIZED VIEWS


SUMMARY LEDGERS, HYBRID COLUMNAR COMPRESSION, MATERIALIZED VIEWS

This system uses 'summary ledger' tables for some reporting

• Smaller reporting tables, fewer attribute columns, more aggregation, fewer rows

• Maintained by an overnight process that incrementally refreshes certain accounting periods

• It sounds a bit like a materialized view!

Cannot:

• Replace with materialized view because cannot replace refresh with a single query

• Introduce compression because rows deleted and reinserted into summary ledger tables


BUILD A MATERIALIZED VIEW ON EACH SUMMARY LEDGER

• Simply copy the summary ledger table into a materialized view

• I found it easier to use a pre-built table, and then reference that in the MV

• Daily full refresh of each materialized view after each summary ledger refresh – done in direct path mode

• No materialized view log on summary ledger, so summary ledger refresh not impacted

• PCTFREE 0 - Do not reserve any free space in MV

• Partition by FISCAL_YEAR/ACCOUNTING_PERIOD

• No candidate for sub-partitioning

• Hybrid Columnar Compression – Query Low

• No indexes – because we will only Smart Scan


MATERIALIZED VIEWS ON SUMMARY LEDGERS

• Enable Query ReWrite

• Set Query ReWrite Integrity Trusted/Stale Tolerated

Rewrite is a cost based transformation.

• The MV is very much smaller and therefore cheaper to access.

• The database will naturally SmartScan the Materialized View

• If we really need to use an index we will stay with the summary ledger.


BLOOM FILTERS


• Bloom Filters by Christian Antognini, 2008

• See also Bloom Filters @ DOAG Webinar 2016

• Space/Time trade-offs in hash coding with allowable errors by Burton H. Bloom, 1970

38

WHAT IS A BLOOM FILTER?

Hash value calculated for each dimension join column value

Each hash value sets a bit in a map

Same hash function used to calculate hash value of fact join column

Rows filtered against bitmap

False positives possible because multiple input values can have same hash value

Only works with equality joins between tables

Then do exact match with hash join to remove false positives, but now on a smaller data set

On Exadata, Bloom filters pushed to storage cells


©Christian Antognini

https://antognini.ch/papers/BloomFilters20080620.pdf

https://www.doag.org/formes/pubfiles/8319505/BloomFilters20160812.pdf

http://crystal.uta.edu/~mcguigan/cse6350/papers/Bloom.pdf


PEOPLESOFT TREES AND NVISION SQL

How you build application SQL matters. It has consequences for SQL performance.

The are PeopleSoft 'tree performance options' configuration settings that affect the generated SQL.


HEIRARCHICAL ACCOUNTING DATA

Accountants group and 'roll-up' related data for reporting.

Taking ACCOUNTs as an example (not PeopleSoft)

• Transactions are allocated to different accounts for reporting purposes

• Related or similar accounts are often grouped together

• 7000-7999: Overheads

• 7000-7099: Staff Costs

• 7100-7199: Building Costs

• 7200-7299: Energy/Fuel Costs

Within the range there will be individual accounts

• 7200: Electricity

• 7201: Gas…

• 7202: Oil…


PEOPLESOFT TREES

In PeopleSoft each hierarchy is represented in a 'tree' structure

In a tree of accounts

• A tree node may represent a range of accounts as above

• Or can have individual nodes for each account

That has consequences for the SQL that is built later in nVision

• A ranged tree will have

• ACCOUNT BETWEEN RANGE_FROM_10 AND RANGE_TO_10

• A single value tree will have

• ACCOUNT = RANGE_FROM_10

BUSINESS_UNIT

CHARTFIELD1 PSTREESELECT05

PSTREESELECT10 PSTREESELECT10PS_LEDGER

FISCAL_YEARACCOUNTING_PERIOD

CURRENCY_CD

ACCOUNT


Static Selectors

• The entire tree is extracted into a tree selector

• It remains extracted until the tree changes and it needs to be extracted again

• Report on the parts of the tree of interest by specifying node(s) on the tree from/below which to report

• TREE_NODE_NUM BETWEEN …

• Can only be represented as a ranged tree because underlying dimension might change

• Always get

• ACCOUNT BETWEEN RANGE_FROM_10 AND RANGE_TO_10

43

TREE SELECTOR TYPES

Dynamic Selectors

• Extracted on demand at report runtime

• Deleted again when report completes

• Only the part of the tree on which I want to report is extracted

• I can choose to force the selector to contain single values even if the tree is ranged

• Thus I can get

• ACCOUNT = RANGE_FROM_10

• And this is what I need if I want Bloom Filters to work.


I can filter the fact (ledger) table by a dimension

I can additionally group by the dimension, or not.

If I am only filtering by the dimension:

• I could replace the join to the dimension with a list of literal values generated from the dimension.

• That is what the 'use literal values if feasible' performance option does

Works well on Exadata because predicates pushed to storage cell

• Above about 1000 nodes in tree time to evaluate predicates becomes significant

Not effective on non-engineered system for smaller non-ranged trees.

44

FILTERING AND GROUPING BY TREE

SELECT L2.TREE_NODE_NUM,L3.TREE_NODE_NUM

,SUM(A.POSTED_TOTAL_AMT)

FROM PS_LEDGER A

, PSTREESELECT05 L1

, PSTREESELECT10 L2

, PSTREESELECT10 L3

, PSTREESELECT10 L

WHERE A.LEDGER='S_ICMGCONS'


AND A.ACCOUNTING_PERIOD BETWEEN 1 AND 11 AND

L1.SELECTOR_NUM=30369 AND A.BUSINESS_UNIT=L1.RANGE_FROM_05 AND

L2.SELECTOR_NUM=30370 AND A.CHARTFIELD1=L2.RANGE_FROM_10 AND

L3.SELECTOR_NUM=30371 AND A.ACCOUNT=L3.RANGE_FROM_10 AND

L.SELECTOR_NUM=30372 AND A.CHARTFIELD1=L.RANGE_FROM_10

AND A.CURRENCY_CD='USD'

GROUP BY L2.TREE_NODE_NUM,L3.TREE_NODE_NUM


COST-BASED OPTIMIZER STATISTICS


• If you don't tell the optimizer the truth about your data, then you really cannot expect it to produce a sensible answer

• The trouble is that even if you do tell it the truth, then it may still not produce a sensible answer!

46

GENERAL PRINCIPLES

• Statistics gathering where you need it, at the right time.

• Build it into the application if necessary

• Eg PeopleSoft: %UpdateStats macro

• calls DBMS_STATS to collect statistics during batch runtime.

• A broadly sensible approach, but there some exceptions

• Eg Oracle E-Business Suite: FND_STATS package

• wrapper for DBMS_STATS

• 19c Exadata

• Real-Time Stats

• High-Frequency Automatic Statistics Collection


LEDGER STATISTICS

Financial processes insert new rows as they post to the ledger

• Mostly batch processes

• Mostly row-by-row

• nVision does not alter the contents

• Generally, overnight maintenance window is adequate to maintain the statistics

• Summary ledger table statistics updated by application

• Summary ledger materialized views collect real-time statistics on initial direct-path load


HISTOGRAMS ON LEDGER

Useful on low cardinality columns (<= 254 distinct values)

• LEDGER, BUSINESS_UNIT, CHARTFIELD1/2/3, CURRENCY_CD, STATISTICS_CD

Prevent histograms on very high cardinality columns > 1000 distinct values

• POSTED_TOTAL_AMT, PROJECT_ID, OPERATING_UNIT

• (this list will vary from customer to customer)

• Consider Top-n histograms (>=12c)

• Or perhaps just set all other columns to SIZE 1 (no histograms)

Control by defining a table preference


HISTOGRAMS AND EXTENDED STATISTICSON LEDGER

Tempting to have histograms on FISCAL_YEAR and ACCOUNTING_PERIOD

• Bug 6972291 Column group selectivity is not used when there is a histogram on one column

• https://antognini.ch/2014/02/extension-bypassed-because-of-missing-histogram/

• But not unless have extended statistics with histograms on combination FISCAL_YEAR and ACCOUNTING_PERIOD

• FOR COLUMNS SIZE 254 FISCAL_YEAR ACCOUNTING_PERIOD …

• FOR COLUMNS SIZE 2048 (FISCAL_YEAR, ACCOUNTING_PERIOD) …

• FOR ALL COLUMNS SIZE 1

https://support.oracle.com/epmos/faces/DocContentDisplay?id=6972291.8

https://antognini.ch/2014/02/extension-bypassed-because-of-missing-histogram/


TREE SELECTOR STATISTICSWHAT IS THE CHALLENGE?

We need dynamic selectors to produce equality joins to be able to use Bloom filters.

Lots of concurrent reports, lots of selectors being inserted into/deleted from selector tables.

Each selector is given a unique selector number

• Selectors numbers sequentially. Selector number increases.

• Statistics are not collected during nVision reports, and we cannot add code to the application so to do.

• SELECTOR_NUM goes outside range of statistics until next maintenance window

• Different extracts from different trees all mixed up in each selector table.


Optimizer Dynamic Sampling

• Don't have explicit statistics

• Lock and delete statistics

Inconsistent results

• Optimizer couldn't decide which selector to start with

Also tried OPTIMIZER_DYNAMIC_SAMPLING=4

• Ran into a parse problem with 'Use Literal Values'

None of this helps when lots of trees are extracted into the same selector table

51

TREE SELECTOR STATISTICSOPTIMIZER DYNAMIC SAMPLING

19c new features might be interesting

• Real-time statistics

• Column statistics really are real time

• Might help with SELECTOR_NUM going out of range

• High-Frequency Automatic Statistics Collection

• Will bring the statistics up to date more frequently

• But not to use to today on 12.1


Interval range partitioning on SELECTOR_NUM

• Interval of 1

• Starting with 1

• Every selector number goes into a different partition

nVision queries have equality predicate on SELECTOR_NUM

• Single partition queries

• Use partition statistics accurately describe selector

52

INTERVAL PARTITIONING FOR TREE SELECTOR TABLES

There are problems with approach

• Complexity

• Maximum number of partitions: 1048576

• Therefore maximum value of SELECTOR_NUM is also 1048576

• Need to reset selector number every 3 months

• Interval partitioning is not designed for frequent creation of new partitions in the same table

• LIBRARY CACHE LOCK


Only one process can insert a new partition into the same table concurrently.

• Created multiple copies of tree selectors in multiple schemas

• Created synonyms and grants for all referenced SYSADM tables and views in all schemas

• ALTER SESSION SET CURRENT_SCHEMA=…

• Hash process instance number to get schema

• Started at 16 schemas, went to 32

• Reduced Library Cache Lock until it was not significant

53

INTERVAL PARTITIONINGLIBRARY CACHE LOCKING

This is what happens when you abuse a feature

• You have to do a second work around (multiple schemas) for the first work around (interval partitioning)


LOGGING SELECTOR USAGE


LOGGING SELECTOR USAGE

PeopleSoft has to know when to refresh static selectors from trees

• Static selectors are tracked in PSTREESELCTL

• VERSION compared with PSTREEDEFN.VERSION

Dynamic selectors are not tracked, created AND deleted at run time

Tree selector table will grow over time as debris left behind by failed nVisions

Created logging table: XX_NVS_TREESLCTLOG

• PL/SQL Packaged procedure: XX_NVISION_SELECTORS

• Compound triggers on PSTREESELECTnn tables

• Collect session instrumentation: MODULE, ACTION, CLIENT_INFO

• Purge process for dynamic selectors driven by log

• Statistics management in triggers


SELECTOR LOG TABLE

Column Name Data Type Description

SELECTOR_NUM NUMBER Unique identifier for tree selector records.

PROCESS_INSTANCE NUMBER PeopleSoft process instance number for nVision/Query

LENGTH NUMBER Length of tree selector

NUM_ROWS NUMBER Number of rows inserted into tree selector.

Counted by the AFTER ROW part of the triggers.

TIMESTAMP TIMESTAMP Time when rows inserted

MODULE VARCHAR2(64) Module of session inserting selector rows.

ACTION VARCHAR2(64) Action of session inserting selector rows

CLIENTINFO VARCHAR2(64) Client info of session inserting selector rows. This will include

• PeopleSoft Operator ID,

• Name of the application server or process scheduler domain,

• Name of machine where the client process is executing,

• Name of the client executable process.

STATUS_FLAG VARCHAR2(1) I=Selectors Inserted

S=Static Selectors Inserted

D=Selectors Deleted

X=Selectors Deleted AND Partition Dropped

TREE_NAME VARCHAR2(18) Name of tree from which selector extracted.

Obtained by querying statement from V$SQL.

OWNER_ID VARCHAR2(8) Schema under which nVision report run

PARTITION_NAME VARCHAR2(128) Name of partition where selectors stored

JOB_NO NUMBER Database Job number to collect statistics on partition.


COMPOUND INSERT TRIGGER

CREATE OR REPLACE TRIGGER sysadm.pstreeselect10_insert

FOR INSERT ON sysadm.PSTREESELECT10 compound trigger

l_err_msg VARCHAR2(100 CHAR);

AFTER EACH ROW IS

BEGIN

sysadm.xx_nvision_selectors.rowins(:new.selector_num,:new.range_from_10,:new.range_to_10);

EXCEPTION WHEN OTHERS THEN NULL;

END after each row;

AFTER STATEMENT IS

BEGIN

sysadm.xx_nvision_selectors.logins(10,'SYSADM');

EXCEPTION WHEN OTHERS THEN

l_err_msg := SUBSTR(SQLERRM,1,100);

dbms_output.put_line('Error:'||l_err_msg);

END after statement;

END;

/

Count the rows being

inserted. Do this in memory

with package global variables.

Update the log


COMPOUND DELETE TRIGGER

CREATE OR REPLACE TRIGGER sysadm.pstreeselect10_delete

FOR DELETE ON sysadm.PSTREESELECT10 compound trigger

l_err_msg VARCHAR2(100 CHAR);

AFTER EACH ROW IS

BEGIN

sysadm.xx_nvision_selectors.rowdel(:old.selector_num);

EXCEPTION WHEN OTHERS THEN NULL;

END after each row;

AFTER STATEMENT IS

BEGIN

sysadm.xx_nvision_selectors.logdel(10);

EXCEPTION WHEN OTHERS

THEN

l_err_msg := SUBSTR(SQLERRM,1,100);

dbms_output.put_line('Error:'||l_err_msg);

END after statement;

END;

/

Get the selector being

deleted

Update the log


SELECTOR PURGE

• Trigger to purge when nVision no longer processing*

• Daily purge process clears any selectors from on-line processes after 2 days

• Selector log is not purged because it contains useful information about report AND tree size AND usage.


SYSTEM WIDE TREE ANALYSIS

Num Min Average Median Max Std Dev Sum Num

TREE_NAME Len Uses Rows Rows Rows Rows Rows Rows Procs DTL_FIELDNAME T T T

------------------ --- ------ ------ ------- ------ ------ ------- --------- ----- ------------------ - - -

…

CORP_ACCT 10 5 1147 2839 2616 6668 2263 14194 1 ACCOUNT J D S

FUNCTION 10 480 9 32 35 35 8 15474 43 CHARTFIELD2 L D S

INT_SUP 6 7 225 2463 2838 2838 987 17243 1 PRODUCT L D S

STAT_PRODUCT 6 8 2889 2889 2889 2889 0 23112 1 PRODUCT J D S

AFFILIATE 5 43 215 576 509 938 223 24789 15 AFFILIATE L D S

INT_GAAP_CON 5 62 82 486 522 730 225 30153 10 BUSINESS_UNIT L D S

BU_GAAP_CON 5 96 44 619 614 731 115 59461 48 BUSINESS_UNIT L D S

STAT_ACCOUNT 10 45 23 4204 6516 6516 2905 189182 6 ACCOUNT J D S

INT_REP1 10 135 149 1563 1664 1664 379 211005 1 CHARTFIELD1 L D S

COMBO_CODE 10 172 17 1592 1532 2430 809 273846 18 CHARTFIELD1 L D S

UKGL_ACCOUNT 10 2586 2 1713 1147 7797 1793 4430262 110 ACCOUNT J D S

Account trees join because

nearly 8000 rows

Always 2889 rows, whole

tree, not selective. Join.


TREE SELECTOR LOG

Analysis can inform decisions about tree performance options

Sometimes difficult to make decisions for whole of nVision reporting.

May need to override tree level options in specific layouts

Drives statistics collection


SELECTOR STATISTICS MAINTENANCE1ST ATTEMPT

Collect statistics in after statement trigger

• Cannot run DBMS_STATS.GATHER_TABLE_STATS because it has a commit

• Cannot run it in an autonomous transaction because then it cannot see the rows inserted because they have not been committed

• Tried submitted a job to database job scheduler to collect statistics

• Use DBMS_JOB.SUBMIT, so request only active after insert that fired trigger commits

Partially successful

• Accurate partition statistics put in place that did generally produce good execution plans

• But it could take time for job to start, and first few queries might produce poor execution plans


SELECTOR STATISTICS MAINTENANCE2ND ATTEMPT

Selector logging tells us

• How many rows inserted for a given selector

• Min/max value for join column RANGE_FROM_nn

• We know those values will be unique

• And we know which partition that selector is in

• Estimate number of blocks and clustering factor

So I can calculate sensible partition statistics

• Update using DBMS_STATS.SET_TABLE_STATISTICS

• That still has to happen in an autonomous transaction

Testing showed that we got better performance if we still scheduled statistics collection

• Real statistics better than calculated statistics

• Calculated statistics better than no statistics and/or Optimizer Dynamic Sampling


SQL PROFILES


SQL PROFILES

I had resisted SQL profiles due to dynamic nature of SQL.

• It would be a continuously moving target.

coe_xfr_sql_profile.sql

• Takes full profile of hints from a captured execution plan → SQL Profile

SQL Patches 11g/12c

• Inject a set of hints into a SQL Statement

• match by SQL_ID. No forced matching

• Can do the same with SQL Profiles, and make it force match

• Favourite hints

• REWRITE, PX_JOIN_FILTER, LEADING

• OPT_PARAM('parallel_degree_policy','AUTO'), OPT_PARAM('parallel_min_time_threshold',2),

OPT_PARAM('parallel_degree_limit',4)


DATABASE PARAMETERS FOR NVISION


DATABASE PARAMETERS FOR NVISION

By trigger, data driven, just for nVision.

Avoided retesting for whole database.

PRCSTYPE PRCSNAME OPRID RUNCNTLID KEYWORD PARAM_NAME PARMVALUE

-------------------- ------------ ---------- ------------------------ -------- ------------------------------ ---------

nVision-ReportBook RPTBOOK SET parallel_degree_policy auto

nVision-ReportBook RPTBOOK SET parallel_degree_limit 4

nVision-ReportBook RPTBOOK SET _optimizer_skip_scan_enabled FALSE

nVision-ReportBook RPTBOOK SET parallel_min_time_threshold 1

nVision-ReportBook RPTBOOK SET parallel_degree_level 150


CONCLUSION


CONCLUSION

Every decision we made required that we understood how the feature worked.

You can criticise the complexity

• Library Cache Locking on Interval Partitioning is a good example of pushing a feature too far.

• However, the GL reports are available at the start of the business day in continental Europe.

Most of what I have discussed I have got from other people and their publications

• Instrumentation with Triggers and FGA (Tyler McPheeters)

• Analysis of ASH and SQL drove decisions about partitioning (Graham Wood).

• HCC Query Low (Roger MacNicol)

• Bloom Filtering (Christian Antognini), requires accurate statistics (JL).

• Max partitions limit (Connor, AskTOM KISS series)

• Setting own stats on partitions with PL/SQL (JL)

• SQL Profiles implementation based on coe script (Carlos Sierra)

• Automatic parallelism hint (Mauro Pagano)


You have already missed

• Statement Parsing

• Index Compression

• ASH (Craig Schallahammer)

• Stats # 1(Neil Chandler)

• Optimizer Latest (Nigel Bayliss)

• Count Distinct (Roger MacNicol)

70

#PTK @ UKOUG TECHFEST19

Still to come

• Troubleshooting Exec Plans (JL)

• Indexing (Richard Foote)

• Defaults (Timur Akhmadeev)

• Plan Stability (Nigel Bayliss)

• Stats # 2(Neil Chandler)

• Optimizer 19c (Christian Antognini)

• 19c RTS and HFASC (DMK)

• 19c New Features – including Automatic Indexing – (Richard Foote)

• Optimizer Panel Group (Antognini, Bayliss, Colgan, Foote)


QUESTIONS

Date post:	30-Mar-2021
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

PRACTICAL APPLICATION PERFORMANCE TUNING: APPLYING … · 2019. 12. 6. · PROFILE DB TIME USING...

Documents