oratopA Database Monitoring ToolJuly 2013

oratopAn Oracle Call Interface (OCI) based API that runs on Linux X86 platforms and resembles Unix “top” utility in appearance. Its data gathering is solely from the database using internal views. The utility can be run against Real Application Cluster (RAC) or none-RAC single instance database. It provides a user the ability to monitor the database in real time and it is not intended to replace Unix “top”, EMGC, or server manageability tools like ASH, AWR, ADDM, etc., that provides granular details.

INTRODUCTION ............................................................................................................................ 3

LAYOUT ......................................................................................................................................... 3

DATA DESCRIPTION .................................................................................................................... 4

Section 1: Header ............................................................................................................... 4

Section 2: Database ............................................................................................................ 5

Section 3: Database Events ............................................................................................... 6

USAGE ........................................................................................................................................... 8

Options ................................................................................................................................ 8

Exiting ............................................................................................................................... 10

CONNECTIONS ........................................................................................................................... 10

Bequeath .......................................................................................................................... 10

Remote ............................................................................................................................. 11

BATCH MODE .............................................................................................................................. 12

INTERACTIVE .............................................................................................................................. 13

TECHNICAL DETAILS ................................................................................................................. 15

CONCLUSION .............................................................................................................................. 16

2

Oratop V13.2.4

Abstract

oratop can be used to monitor databases on Linux and other platforms but the executable must run on Linux with an Oracle client. To monitor databases on other platforms simply define an alias in tnsnames.ora of the linux client and connect to the databases remotely as you would with sqlplus

INTRODUCTION

The Oracle program named oratop allows users with select privilege to a set of Oracle’s views to monitor 11.2 or higher version of oracle database activities. It runs dynamically in near real time and provides a live window to a running database. (see sample snapshot shown in Figure 1 below)

Key motivation includes:

• Monitoring current database activities,• Database performance,• Identifying contentions and bottleneck• Monitor Active Data Guard

Figure 1. oratop (default mode)

3

LAYOUT

The data-grouping and layout of the program text output is shown in Figure 2.

Figure 2. oratop layout

DATA DESCRIPTION

The “oratop” displays relevant database activity information presented in four sections.

• HEADER (Section1)Tool and database information

• DATABASE (Section2)Database instance Activity

• DB EVENTS (Section3)AWR like “Top 5 Timed Events“

• PROCESSES (Section4)Processes/Sessions information

The following are breakdown detail of the various sections data using online help (see interactive section)

4

Section 1: Header

Figure 3. Header acronyms (“1” key press)

Section 2: Database

Default IOPS mode

Figure 4. Section 2 acronyms IOPS mode (“2” key press)

5

IORL mode - With key press “r” made before going into help

Figure 5. Section 2 acronyms IORL mode (“2” key press)

Section 3: Database Events

Applicable to both, the default (Cumulative) and real-time modes.

Figure 6. Section 3 acronyms (“3” key press)

6

Section 4: Process/Session

Default USR/PROG mode

Figure 7. Section 4 acronyms default USR/PROG mode (“4” key press)

MOD/ACTN mode - With key press “m” made before going into help

Figure 8. Section 4 acronyms MOD/ACTN mode (“4” key press)

7

Alerts

Selected variables will toggle in red color if met the following conditions :

% db : values > 99%%CU : load > 2 x cpu counts & host cpu > 99HLD : load > 2 * cpu counts and aas > cpu countsIORL : value > 20ms%FR : value < 1%ASW : value = session counts, USNAAS : value > cpu countsDBW : value > 50%EVENT : Active wait eventPGA : potential unusual memory growthBLOCKER: a blocking session with wait time > 5 minutesW/T : value > 1 sec

USAGE

Figure 9. Command line help

Options

i : Interval delay time. ( requires value in seconds)Specifies the delay between update refresh. A short interval delay (<10s) is not guaranteed to return for the specified delay. This is particularly true for a loaded system. Default: 3 seconds.To overrides the default value• Command line: -i <number in seconds>• Run time: changed with the 'f' keystroke interactive commands.

8

f : Long format, (132 columns) Specifies long format for header & process section.Default: short (80 columns)To overrides the default value• Command line: -f • Run time: changed with the 'f' keystroke to toggle between short and long format.

r : IORL column of section 2. Specifies I/O selection.Default: IOPSTo overrides the default value• Command line: -r• Run time: changed with the 'r keystroke to toggle between IOPS and IORL.

d : Real-Time Top 5 Wait Events, section 3. Specifies Wait Events display mode.Default: CumulativeTo overrides the default value• Command line: -d• Run time: changed with the 'd keystroke to toggle between Cumulative and Real-Time.

m : Module/Action columns of section 4. Specifies Module/Action selection.Default: USER/PROGRAMTo overrides the default value• Command line: -m• Run time: changed with the 'm’ keystroke to toggle between MOD/ACTN and USR/PROG.

b : Batch mode operation. Runs in sequential frames. It is useful for sending output to a file. In this mode, oratop will run continuously interrupted only by the user (CTRL-C) or until the iterations limit is reached if it is set with the '-n' command-line option.Default: RefreshedTo overrides the default value• Command line: -b• Run time: N/A.

n : Maximum Number of iterations. (requires a number)Specifies the maximum number of iterations, or frames, oratop should produce before ending.Default: infiniteTo overrides the default value• Command line: -n <number of iterations>• Run time: N/A.

h : Help. Displays usage or output information.Default: N/ATo overrides the default value• Command line: –h[help]• Run time: changed with the 'h’ keystroke.

9

Exiting

To quit the program, user may press any of the following keyboard keys:

• Character "q" or "Q", • Esc key • Ctrl+c (to abort)

In all cases, proper OCI session cleanup and logout is performed.

Unprivileged user

To allow the non-privileged user to use oratop, the system administrator with DBA privileges may issue the following grants:

grant select on v_$instance to <username>;grant select on v_$parameter to <username>;grant select on v_$statname to <username>;grant select on v_$wait_chains to <username>;grant select on gv_$process to <username>;grant select on gv_$session to <username>;grant select on gv_$sysstat to <username>;grant select on gv_$sesstat to <username>;grant select on gv_$osstat to <username>;grant select on gv_$sga to <username>;grant select on gv_$system_event to <username>;grant select on gv_$session_event to <username>;grant select on gv_$session_wait to <username>;grant select on gv_$sysmetric to <username>;

Table 1. Grants required for unprivileged user

Note. Typical error for a non-granted user may receive upon connection is,

ERROR CODE = 942ORA-00942: table or view does not exist

CONNECTIONS

Shell environment settings

$ export ORACLE_HOME=<path> $ export LD_LIBRARY_PATH=$ORACLE_HOME/lib$ export PATH=$ORACLE_HOME/bin:$PATH

The program can be run with either of the following methods.

Bequeath

Local connection within the same oracle Server/client Linux environment. To run the program directly on a Linux server that hosts a database, (bypassing the listener)

10

set the Unix environment for ORACLE_SID and start the program.

$ export ORACLE_SID=<sid>$ oratop [Options] [Logon]

Remote

Front-end “oratop” client is started from a Linux host and connecting remotely via TNS to oracle back-end remote server (platform independent).

Running the program from a client Linux host with installed compatible $ORACLE_HOME via TNS to any server, (Linux or none-Linux) with databases version 11.2.0.3 onward.

The procedures require the following:

• Server SettingsThe remote database requires the following:

1. “REMOTE_LOGIN_PASSWORDFILE” parameter2. Oracle password file

Note. Connecting “ / as sysdba“ is allowed only for bequeath connections.

• Client SettingsFor remote connection, the following must be satisfied on the client $ORACLE_HOME. A TNS entry to the remote database must exist in $TNS_ADMIN/tnsnames.ora.

11

<tns_alias> =(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = <Server_host>)(PORT = <port>)) (CONNECT_DATA =(SERVICE_NAME = <db_service> )))

Table 2. Remote connection tnsnames.ora

To run “oratop” from the client home, set client environment variables and start it

$ export TNS_ADMIN=<$ORACLE_HOME/network/admin>$ oratop [Options] [Logon]

Batch mode

Figure 10. Batch mode redirected to output text file.

12

INTERACTIVE

The following keyboard keys will toggle display mode at run time,

“h” - Online help

Figure 11. Run time help menu

“f” - Long/short Format

Figure 12. Long format (132 columns)

13

“d” - Real-Time/Cumulative

Figure 13. Top5 Wait Events in Real-Time

“r” - IORL/IOPS

Figure 14. Display I/O latency instead of IOPS.

14

“m” - Module-Action/User-program

Figure 15. Display Module/Action instead of User/Program.

TECHNICAL DETAILS

The “oratop” program is written in c using OCI API. It is not intrusive to a database server, and it leaves small footprints on the server, namely the sqls used by the program. It employs Unix “termio, “ioctl” libraries, and VT100 escape characters to achieve cursor control, font colors, program exit control and terminal (xterm) resizing capability.

Overhead

The oratop program is a very light process that uses minimal CPU and Memory. Server process resources are server dependent.

Limitations

• The program is not portable; it runs on Linux platforms only• The program is compatible with oracle client version11.2 and 12cR1 • Requires server to have been started with the following parameters:

statistics_level=TYPICAL1

timed_statistics=TRUE

1 Statistics_level cannot be set to “BASIC” since it will disable Manageability feature.

15

Caveats

The program may exhibit anomalies at run time, most of which are expected since an event may occur while the program is in the middle of executing/ fetching operations.

Blank ScreenOn a busy server, the following operations may leave the terminal blank for a short period:

• A program initialization• A terminal resizing (shrink/expand the terminal)• An instance joining/leaving the cluster• Quitting interactive keys menus• Upon exiting the program

Sluggish response to keyboard key pressPressing a keyboard key to interact with the program may appear to have slow response on a busy server.

Abnormal terminationLike any SQL session, the program may fail due to a server error, and the particular error will be displayed.

Abnormal exitIn some situation (corner cases) the program may leave the terminal (xterm) in an undesirable setting. To reset the terminal to its original settings, the user may issue the Linux command “reset”.

CONCLUSION

oratop utility provides a dba user a quick overview of a running database status and activity. It has the capability to provide needed information such as overall database performance bottleneck, identifying potential blocker(s), and detecting possible process that may have a memory leak. The program also reports badly performing sql as well. Furthermore, it helps in tuning some aspects of the database like user concurrency.

Modifications:

Version 13.2.4 – July 2013, Top waits change for cpu stats by statsname and waits by sum. Version 13.2.3 – June 2013, All instances in section 2 will be listed implicitly in batch mode.Version 13.2.2 – June 2013, Introduced new switches, IOPS/IORL, MOD,ACTN/USR,PROG.Version 13.2.0 – May 2013, Long format and Top 5 dynamic wait events. Version 13.1.0 – Mar 2013, sqlplus connection like and the doc were updated from June 2011. Version 12.1.0 – Dec 2012, added help key “h”.Versions 1-12 – Mar 2010, created.

16