Post on 18-Jan-2018
description
transcript
#.1
Average Active Sessions (AAS)The Golden Metric ?
Kyle Haileyhttp://perfvision.com
#.2205/03/23 Copyright 2006 Kyle Hailey
In this SessionIn this Session1. AAS
Single Metric Shows DB Performance Two methods to calculate
Sampling Time statistics
2. Yardstick Max CPU CPU Count To measure AAS against
3. Subcomponents CPU Waits Time series
#.3305/03/23
Database Performance Database Performance How quick can you find
Bottleneck in DBIf DB is idleCurrent DB Load
what is DB Load ?
What do you use?Statspack/AWRV$active_session_historyAlerts
what do you alert on ?
what the *!####!*!*? is the database doing ?!
#.4
Active SessionActive Session
Copyright 2006 Kyle Hailey
Database Machine
SQL*Plus
Shadow
Process
SQL*Plus
Shadow
Process
Application Server Machine
Shadow
Process
Shadow
Process Shadow
Process
Application
SGA
#.5
idle
Active SessionActive Session
Copyright 2006 Kyle Hailey
SQL*Plus (ie application)
idle idle idle
Have a coffeewaiting waitingTyping waitingtyping typing
Query 1 Query 2 Query 3
Database (shadow process)
TimeActive Active Active
Shadow
Process
#.6
Active SessionActive Session
Database
User 1User 2User 3User 4
Active Sessions
=
1234
Graph represents # of sessions active, but also represents amount of time active in the database
#.7
Active SessionActive Session
Copyright 2006 Kyle Hailey
Active Sessions
Users Waiting
For Every Active Session there is a user (or application) waiting
1234
1234
#.8
Measuring Active Sessions
Copyright 2006 Kyle Hailey
Fast query run often
Fast query run rarely
Slow query
If happens a lot or for long … we’ll catch it, guaranteed
User 1
User 2
User 3
Sampling Every Second
. . .
#.9Copyright 2006 Kyle Hailey
Sampling is like taking PicturesSampling is like taking Pictures
#.10
idleidle
Active SessionActive Session
Copyright 2006 Kyle Hailey
idleidle idleidle idleidleQuery 1 Query 2 Query 3Database
To execute a query , some CPU will be used but we also might spend time waiting for IO or on waiting for concurrency resources like latches
CPU IO Wait
idleidleidleidle idleidle idleidleDatabaseQuery 1 Query 2 Query 3
Activity can thus further be broken down into the type of activity: CPU, IO or WAIT
Work Work LatencyLatencyContentionContention
#.11
Session 1
Session 2
Session 3
Session 4
TIME
Graphical ASHGraphical ASH
#.12Copyright 2006 Kyle Hailey
Graph of User StatesGraph of User States
#.13Copyright 2006 Kyle Hailey
One Second GraphOne Second Graph
#.14Copyright 2006 Kyle Hailey
15 Second Averages15 Second Averages
#.15Copyright 2006 Kyle Hailey
Maximum CPU LineMaximum CPU Line
#.16Copyright 2006 Kyle Hailey
Idle UsersIdle Users
#.17Copyright 2006 Kyle Hailey
OEM Perf Page OEM Perf Page
#.18
AAS in OEMAAS in OEM
LOAD
AAS
#.19Copyright 2006 Kyle Hailey
The Power ASH gives AASThe Power ASH gives AASDB Home DB Home
PerformancePerformance
Top ActivityTop Activity
Based on Based on ASHASH
Based on TIMEBased on TIME(events Statistics)(events Statistics)
AAS=db AAS=db time/elapsed timetime/elapsed time
AAS=count active AAS=count active users /samplesusers /samples
#.20Copyright 2006 Kyle Hailey
DB TIME = area under the curveDB TIME = area under the curve
DB Time = DB Time = active sessions(ti) * ΔtΔt nn
ΣΣ00
Height = # of SessionsHeight = # of SessionsWidth = secondsWidth = secondsArea under curve = DB TimeDB Time
DB Time = sum of active time in databaseDB Time = sum of active time in database
#.212105/03/23 Copyright 2006 Kyle Hailey
AAS SourcesAAS Sources
1. Manually from v$sysstat (9i : v$system_event )
2. Statspack Need several calculations
3. AWR One calculation
4. OEM 10g Directly displayed
AAS = DB Time/Elapsed Time
#.22Copyright 2006 Kyle Hailey
1. Manually1. Manually
DB Time (DBT) = Time Spent in Database
DB TIME (10g) =
DB TIME (9i) = Select sum(time_waited) from v$system_event where event not in ( ... idle events …);+Select value from v$sysstat where name = ‘CPU used by this session’;
select value from v$sysstat where name = ‘DB time’;‘DB time’
still need to take delta valuesNote: stats$idle_event : 70 v$event_name.wait_class=‘Idle’ :62
AAS = DB TIME / Elapsed Time
#.232305/03/23 Copyright 2006 Kyle Hailey
2. Statspack AAS 2. Statspack AAS Look for
Elapsed TimeTop 5 Timed Events
Start at line 52 of about 1300Start at line 52 of about 1300
#.242405/03/23 Copyright 2006 Kyle Hailey
2. Statspack AAS2. Statspack AAS
Elapsed Time
Look at Top 5 Timed EventsTop 5 Timed EventsTop 5 Timed Events~~~~~~~~~~~~~~~~~~ % Total~~~~~~~~~~~~~~~~~~ % TotalEvent Waits Time (s) Call TimeEvent Waits Time (s) Call Time
--------------------- --------- --------- ---------------------------- --------- --------- -------
buffer busy waits 2,748 250 78.72buffer busy waits 2,748 250 78.72CPU time 32 10.16CPU time 32 10.16free buffer waits 1,588 15 4.63free buffer waits 1,588 15 4.63write complete waits 10 8 2.51write complete waits 10 8 2.51log buffer space 306 5 1.51log buffer space 306 5 1.51
STATSPACK report for
DB Name DB Id Instance Inst Num Release RAC Host------- ----------- -------- -------- ---------- --- -------
LABSF03 1420044432 labsf03 1 10.1.0.2.0 NO labsfr
Snap Id Snap Time Sessions Curs/Sess
--------- ------------------ -------- ---------
Begin Snap: 1 03-Apr-06 12:34:06 18 5.6 End Snap: 2 03-Apr-06 12:34:36 18 4.8 Elapsed: 1.00 (mins)
#.252505/03/23 Copyright 2006 Kyle Hailey
2. Statspack AAS2. Statspack AAS
DBTIME= CPU + WAITSCPU = 32 WAITS = 250+15+8+5 = 278 secs
DBTIME=320 Elapsed Time = 60 secs 320 secs / 60 secs
AAS = 5.1
Top 5 Timed EventsTop 5 Timed Events Event Time (s) Event Time (s) ----------------- ---------------------- -----buffer busy waits 250buffer busy waits 250CPU time 32CPU time 32free buffer waits 15free buffer waits 15write complete waits 8write complete waits 8log buffer spacelog buffer space 55
#.262605/03/23
3. AWR Report3. AWR ReportAAS = DB Time/Elapsed Time
23.56/59.66 = 0.39
AAS= 0.39
#.272705/03/23
4. OEM 10g4. OEM 10g
AAS = ~0.75
•OEM 10g
#.282805/03/23 Copyright 2006 Kyle Hailey
Got AAS, Now What ?Got AAS, Now What ?
We Need one more item:
CPU Count
# of CPUs available on SystemShared with other applications
Need to track CPU used on the system as wellOn dual & quad cores, lower the CPU countRepresents max active sessions that can do work
#.292905/03/23 Copyright 2006 Kyle Hailey
CPU CountCPU Count # of CPUs available in
Statspack 10g AWR report OEM 10g
Statspack 9i # of CPUs missing # of CPUs
SQLPLUS> show parameters cpu_count
NAME VALUE------------------ ----------cpu_count 2
#.303005/03/23 Copyright 2006 Kyle Hailey
What’s the DB Doing?!What’s the DB Doing?!
It’s 2am … your manager calls
Whip out the stethoscope:
AASwhat the *!####!*!*? is the database doing ?!
#.313105/03/23 Copyright 2006 Kyle Hailey
AAS FormulasAAS FormulasUse CPU count as yardstick:
AAS < 1 Database is not blocked
AAS ~= 0 Database basically idleProblems are in the APP not DB
AAS < # of CPUsCPU availableAre any single sessions 100% active?
AAS > # of CPUsCould have performance problems
AAS >> # of CPUSThere is a bottleneck
Ideal world – one databasesolution track CPU at OS
AAS > 1 still want to know if a single user is 100% active
#.323205/03/23 Copyright 2006 Kyle Hailey
Available CPU vs AASAvailable CPU vs AASAAS far above available CPU => problem
AAS = 5.1
# of CPU = 2
AAS = 0.39
# of CPU = 2
AAS < 1 , database is fine
Statspack DBTIME= CPU + WAITS
CPU = 32 WAITS = 250+15+8+5 = 278 secs
DBTIME=320 Elapsed Time = 60 secs 320 secs / 60 secs
AAS = 5.1
AWR Report
AAS = 0.75
# of CPU = 2
AAS < 1 , database is fine
#.333305/03/23 Copyright 2006 Kyle Hailey
Going Farther with AASGoing Farther with AASAAS can tell you a lotBut it’s components tell you much moreTo go farther need the components of AAS
1.CPU 2.Wait3.Value over time
Only OEM 10g shows the value over time, Statspack and AWR are aggregated over the snapshot period
#.34Copyright 2006 Kyle Hailey
EM DB Home PageEM DB Home Page
#.35Copyright 2006 Kyle Hailey
OEM 10g Perf PagesOEM 10g Perf PagesDB Home DB Home
PerformancePerformance
AAS Point in TimeAAS Point in Time
AAS over TimeAAS over Time
#.363605/03/23
AAS Components : OEM 10gAAS Components : OEM 10g
OEM 10gPerformance Page
Available CPU AAS:CPU + WAIT
Real CPU available:
Max CPU - non instance CPU
#.373705/03/23
OEM 10gOEM 10g
Relax Relax Get to Work!Get to Work!Looks Looks OK OK
But …But …
#.38Copyright 2006 Kyle Hailey
Idle Database – Perf PageIdle Database – Perf Page Value of proving the database is
Idle It’s the Databases Fault
How many times do you hear that? Database Idle
No load on database Database “performance” is fine
Under utilized Problem lies elsewhere Saved me time and stress many
times And weeks of debate about where the
problem is coming from
#.39Copyright 2006 Kyle Hailey
More than AAS and #CPUMore than AAS and #CPUKnowing your DB Profile
#.40Copyright 2006 Kyle Hailey
When to TuneWhen to Tune1. Machine
a) CPU Response times skewed 100% CPU might be fine Users wait in queue (run queue) => machine
underpowered
b) Memory Paging Wait times skewed (ex : latch free) Erratic response times ( ex : ls )
2. Oracle1) Waits > CPU ?
tune waits2) CPU > 100% ?
tune top CPU SQL3) Else
It’s the application
Oracle Load (AAS)
Top Session Top Wait Top SQL
SQL Detail Session Detail File DetailObject Detail Wait Detail
Host
AAS > #CPU
AAS > 1Waits > CPU
CPU > Waits
CPU Memory
#.414105/03/23
Limited Analysis Limited Analysis What if you find a problem ?Of the 800 waits which in order to solve most need to know
What SQLWhich sessionsValues of P1, P2 and P3Statspack and AWR fail
AAS = DB TIME / Elapsed Time
But there is another wayBut there is another way … …
#.424205/03/23 Copyright 2006 Kyle Hailey
AAS based on ASHAAS based on ASH
ASH - Active Session History v$active_session_historySamples sessions once a second
AAS = count(*) / elapsed_secondsA statistical approximation, but surprisingly close
ASH data source empowers drilldownsTop Sql Top WaitsDetails p1,p2,p3 and more
#.43Copyright 2006 Kyle Hailey
ASHASHvsvsStatisticsStatistics
Statistics are more expensivehave lag time lack clear identification of culprits
#.44Copyright 2006 Kyle Hailey
Statistic Lag TimeStatistic Lag Time
StatisticsStatistics
Samples (ASH)Samples (ASH)
Slight LagsSlight Lags
#.45Copyright 2006 Kyle Hailey
CPU Lag ProblemCPU Lag Problem ASH is the only way to see CPU usage realtime V$sysstat reports CPU but
is only updated at the end of the call. Long calls look deceiving like no CPU is being used
Time Model also reports CPUUpdated quicker
#.46Copyright 2006 Kyle Hailey
CPU in ASH vs StatsCPU in ASH vs Stats
#.474705/03/23
2. OEM 10g : Top Activity2. OEM 10g : Top Activity•Top Activity
•Based on ASH•Enables Drilldowns
•Top SQL•Top Session
•Drill into a session• Stats• Raw waits• Open cursors• General info
•Drill into a SQL• Stats and text• Users executing• Explain plan• Tuning options
#.48Copyright 2006 Kyle Hailey
Top Activity : Based on ASHTop Activity : Based on ASH
missingmissing
Thanks Thanks To To ASHASH
#.49Copyright 2006 Kyle Hailey
AAS – %Session Time IssueAAS – %Session Time Issue
Whenever AAS > 1I want to know if any one session is 100% activeUnfortunately OEM isn’t set up for this(ASHMON, my free tool, is )
#.50Copyright 2006 Kyle Hailey
Top Activity: ASH SessionsTop Activity: ASH Sessions
Many Users Active Many Users Active
On Performance Page, no way to tell how many usersOn Performance Page, no way to tell how many users
But Top Activity Page fixes that But Top Activity Page fixes that
#.51Copyright 2006 Kyle Hailey
Top Activity: ASH SessionsTop Activity: ASH Sessions
Two Users ActiveTwo Users Active
Shown in % DB TimeShown in % DB TimeMissing % Session TimeMissing % Session Time
#.52Copyright 2006 Kyle Hailey
OEM 10g Perf PagesOEM 10g Perf PagesDB Home DB Home
PerformancePerformance
Top ActivityTop Activity
SQLSQLSessionSession
Copyright 2006 Kyle Hailey
Top ActivityTop Activity
SQLSQLSessionSession
#.53Copyright 2006 Kyle Hailey
Session : ASH ActivitySession : ASH Activity
#.54Copyright 2006 Kyle Hailey
SQL : ASH ActivitySQL : ASH Activity
#.555505/03/23 Copyright 2006 Kyle Hailey
AAS from ASHAAS from ASH1. ASHRPT
Based entirely on v$active_session_history
@?/rdbms/admin/ashrpt.sql Exec ASH_REPORT_TEXT/HTML
select * from tableselect * from table(dbms_workload_repository.ash_report_text((dbms_workload_repository.ash_report_text( (select dbid from v$database),(select dbid from v$database), 1,1, sysdate – 1/24, sysdate – 1/24, sysdate )) ;sysdate )) ;
#.565605/03/23 Copyright 2006 Kyle Hailey
1. ASHRPT1. ASHRPTASH Report For TESTDB/testdbASH Report For TESTDB/testdbDB Name DB Id Instance Inst Num Release RAC HostDB Name DB Id Instance Inst Num Release RAC Host
------------ ----------- ------------ -------- ----------- --- ------------------------ ----------- ------------ -------- ----------- --- ------------
TESTDB 2371570538 testdb 1 10.2.0.1.0 NO sdbe604aTESTDB 2371570538 testdb 1 10.2.0.1.0 NO sdbe604aCPUsCPUs SGA Size Buffer Cache Shared Pool ASH Buffer Size SGA Size Buffer Cache Shared Pool ASH Buffer Size
-------- ------------------ ------------------ ------------------ ------------------ ------------------ ------------------ ------------------ ------------------
22 1,000M (100%) 468M (46.8%) 112M (11.2%) 4.0M (0.4%) 1,000M (100%) 468M (46.8%) 112M (11.2%) 4.0M (0.4%) Analysis Begin Time: 21-Apr-06 12:00:01Analysis Begin Time: 21-Apr-06 12:00:01 Analysis End Time: 21-Apr-06 12:05:01Analysis End Time: 21-Apr-06 12:05:01 Elapsed Time: 5.0 (mins)Elapsed Time: 5.0 (mins) Sample Count: 3,716Sample Count: 3,716 Average Active Sessions: 12.39Average Active Sessions: 12.39 Avg. Active Session per CPU: 6.19Avg. Active Session per CPU: 6.19 Report Target: None specifiedReport Target: None specifiedTop User Events DB/Inst: TESTDB/testdb (Apr 21 12:00 to 12:05)Top User Events DB/Inst: TESTDB/testdb (Apr 21 12:00 to 12:05) Avg ActiveAvg ActiveEvent Event Class % Activity SessionsEvent Event Class % Activity Sessions
----------------------------------- --------------- ---------- --------------------------------------------- --------------- ---------- ----------
CPU + Wait for CPU CPU 67.98 8.42CPU + Wait for CPU CPU 67.98 8.42enq: TX - row lock contention Application 23.98 2.97enq: TX - row lock contention Application 23.98 2.97buffer busy waits Concurrency 4.66 0.58buffer busy waits Concurrency 4.66 0.58latch: cache buffers chains Concurrency 2.26 0.28latch: cache buffers chains Concurrency 2.26 0.28
#.575705/03/23 Copyright 2006 Kyle Hailey
1. ASH RPT1. ASH RPT
1) General info2) Top User Events ***3) Top Background Events4) Top Event P1/P2/P3 Values 5) Top Service/Module6) Top Client IDs7) Top SQL Command Types 8) Top SQL Statements ***
9) Top SQL using literals10) Top Sessions ***11) Top Blocking Sessions12) Top Sessions running PQs 13) Top DB Objects14) Top DB Files15) Top Latches16) Activity Over Time ***
#.585805/03/23
1. ASHRPT over Time1. ASHRPT over Time
Waits over TimeNot in AAS
Difficult but better than nothing
Compare to …
#.595905/03/23
3. Custom Scripts3. Custom ScriptsHate Graphics ?
Query v$active_session_history directlyJoin to dba_hist_active_sess_history for week of data
act.sqlLike top 5 timed events
Aveact.sqlCharts with text AAS by hour (15 minute, minute , etc)
Aveactn.sqlDitto, with top 2 wait events per bucket
Following Scripts Available on http://perfvision.com/ashscripts.php
#.606005/03/23 Copyright 2006 Kyle Hailey
3. Custom Scripts3. Custom Scripts@actAnalysis Begin Time : 2007-07-24 11:04:48Analysis End Time : 2007-07-24 11:19:45Start time, mins ago: 15Request Duration : 15Collections : 528Data Values : 3327Elapsed Time: 15 mins
WAIT_EVENT CNT % Active Ave_Act_Sess
-------------------------------------- ---------- ------------
latch free 10 .3 .02log buffer space 13 .39 .02buffer busy waits 14 .42 .03db file scattered read 15 .45 .03library cache pin 78 2.34 .15log file sync 213 6.40 .40ON CPU 726 21.82 1.38enqueue 855 25.70 1.62db file sequential read 1399 42.05 2.65
------------
sum 6.30
#.616105/03/23 Copyright 2006 Kyle Hailey
3. Custom Scripts3. Custom Scripts@aveactTM NPTS AVEACT GRAPH CPU WAITS---------------- ------ ------- ---------------------- ---- -----06-AUG 13:00:00 270 .33 +- 2 29 5906-AUG 14:00:00 1040 2.24 ++--------2--- 341 198406-AUG 15:00:00 623 6.67 ++++------2---------- 438 371806-AUG 16:00:00 1088 2.59 ++--------2---- 335 248606-AUG 17:00:00 1104 1.26 ++----- 2 349 104306-AUG 18:00:00 1093 1.38 +++---- 2 663 84206-AUG 19:00:00 1012 1.74 ++------- 2 373 138806-AUG 20:00:00 1131 .99 +---- 2 304 82006-AUG 21:00:00 1111 1.22 ++----- 2 344 101206-AUG 22:00:00 1010 1.66 ++------ 2 414 125906-AUG 23:00:00 1120 1.08 +---- 2 298 91307-AUG 00:00:00 1024 .83 +--- 2 273 57607-AUG 01:00:00 1006 1.74 ++------- 2 319 142807-AUG 02:00:00 1090 2.47 ++--------2---- 347 234507-AUG 03:00:00 687 6.59 +++-------2---------- 382 414207-AUG 04:00:00 1004 1.95 ++++++--- 2 1299 65907-AUG 05:00:00 1104 3.08 +++++-----2------ 1170 222607-AUG 06:00:00 1122 1.91 +++++++-- 2 1582 558
#.62Copyright 2006 Kyle Hailey
Aveact.sqlAveact.sql
#.63
Aveact.sqlAveact.sql
Copyright 2006 Kyle Hailey
“-” = WAIT
“+” = CPUwhich waits ? -> aveactn.sql
#.64
Aveactn.sqlAveactn.sql
Copyright 2006 Kyle Hailey
#.656505/03/23 Copyright 2006 Kyle Hailey
SummarySummary AAS: Two Sources
1. v$system_event & v$sysstat Performance page in OEM Statspack, awr report Indirect – based on time converted to AAS Accurate Lags (especially CPU) Limits analysis
2. v$active_session_history Top activity in OEM ashrpt Direct measure of AAS Real time Approximation ***Allows drilldowns***
Tuning: Machine first – CPU, Memory Oracle
AAS ~= 0 Idle AAS > 1 : possible sessions blocked AAS > # CPU : bottleneck
Oracle Load (AAS)
Top Session Top Wait Top SQL
SQL Detail Session Detail File DetailObject Detail Wait Detail
Host
AAS > #CPU
AAS > 1
Waits > CPU
CPU > Waits
CPU Memory
ADDM SQL Tuning Advisor
#.66
Q1Q1
Copyright 2006 Kyle Hailey
What is the easiest way to find the load on the database
a. top 5 timed events
b. Average Active Sessions and CPU count
c. The transaction rate
d. The commit rate
b. Average Active Sessions takes into account all the major criteria in Oracle performance - cpu usage, wait time, elapsed time
others:
a. lacks the elapsed time
c & d - the amount of load created by commits and transactions varies drastically between database to database and even between different loads on a database during the day
#.67
Q2Q2
Copyright 2006 Kyle Hailey
Average active sessions, the measurement in the performance chart in OEM 10g can be calculated by which formula(s)
a. db time / elapsed time
b. wait time / elapsed time
c. count of rows in v$active_session_history over an interval/ number of samples in interval
d. count of rows in v$active_session_history / seconds of elapsed time
answer
a,c,d
c, d are equivalent because ASH samples once a second
others
b doesn't work because it is missing CPU time
#.68
Q3Q3
Copyright 2006 Kyle Hailey
If Average Active Session is less than one, we can say
a. the database is using less than 100% CPU on machine
b. the database is using 100% CPU
c. No session is completely blocked
d. Database performance should be acceptable
answer
a, c and d
#.69
Q4 Q4
Copyright 2006 Kyle Hailey
An Average Active Session value of zero means
a. the database is down
b. the database is hung
c. the database is idle
d. the database is overloaded
answer
c
#.70
Q5Q5
Copyright 2006 Kyle Hailey
CPU is updated in v$sysstata. when call finishesb. every secondc. every five secondsd. in real time
a - which can be a problem in monitoring tools if the call last a long time showing no CPU usage until the call finishes, then reporting an unrealistic spike of CPU when the call finishes