Post on 19-May-2015
transcript
www.SageLogix.com
Paper #536
Oracle Log MinerToo Clever For Words
Tim Gorman(tim@sagelogix.com)
PrincipalSageLogix, Inc.
IOUG-A “Live 2003”IOUG-A “Live 2003”
www.SageLogix.com
AgendaAgenda
Basic overview of Log MinerWhat is it?
How to use it?
Why use it?
Oracle9i New FeaturesEnhancements to support Log Miner as a major component of Data Guard’s Logical Standby Database and Streams feature
www.SageLogix.com
What is Log MinerWhat is Log Miner
Log Miner is a mechanism for examining redo log files (online or archived) from any Oracle8, Oracle8i, or Oracle9i database
The database which generated the logs does not have to be the database used to examine them
Oracle server processes perform the I/O on redo log files in a Log Miner session
Session initiated by identifying the log files to be read
Data is retrieved on demand by SQL queries within the session on the view V$LOGMNR_CONTENTS
Easy and safeCan examine logs in another database altogether
Only need to generate snapshots of database data dictionary periodically
www.SageLogix.com
Uses for Log MinerUses for Log Miner
Debugging or auditing DML or DDL actions performed within a specified time periodRecovering dropped tables
by finding the exact SCN of the DROP commandto allow recovery of a CLONE database the precise SCN-1, instead of an approximate time of day
Recovering deleted or updated databy finding the relevant REDO_SQL statement(s) and running the associated UNDO_SQL command(s)
Database Replication componentOracle9i Data Guard Logical Standby DatabaseOracle9i Streams
www.SageLogix.com
Using Log MinerUsing Log Miner
PL/SQL packages supplied with the RDBMS to query information from (online or archived) redo log files
Oracle data dictionary must have previously been exported
using the DBMS_LOGMNR_D.BUILD procedure
contents must be valid from when redo logs were generated
exporting allows point-in-time snapshots of data dictionary
Package DBMS_LOGMNR_D supplied with Oracle8i or Oracle9i
Can be installed on lower version databases, such as Oracle8
Filename: “%OH%/rdbms/admin/dbmslmd.sql”
www.SageLogix.com
Using Log MinerUsing Log MinerPackage DBMS_LOGMNR_D
BUILD
New parameter OPTIONS
STORE_IN_FLAT_FILE
Also requires parameters DICTIONARY_FILENAME and DICTIONARY_LOCATION
STORE_IN_REDO_LOGS (Oracle9i)
Progress of BUILD execution now visible (via DBMS_OUTPUT) when SET SERVEROUTPUT ON enabled in SQL*Plus
SET_TABLESPACE procedure (Oracle9i)
By default, internal tables used by Log Miner reside in the SYSTEM tablespace
SET_TABLESPACE changes to the specified tablespace and moves the tables (if already present somewhere else)
www.SageLogix.com
Using Log MinerUsing Log Miner
SET_TABLESPACE should be a one-time only occurrence
Moving Log Miner tables from SYSTEM to TOOLS tablespace, for example
BUILD should be used regularly to ensure that any future Log Miner sessions will have accurate data dictionary mapping information available
Using Log Miner while viewing data without data dictionary translations is miserable!
All schemas are named “UNKNOWN”
All tables are named “OBJ #”
All columns are named “COL #”
www.SageLogix.com
Using Log MinerUsing Log Miner
Creating a Log Miner sessionRedo log files must first be added to a list
DBMS_LOGMNR.ADD_LOGFILE(file-name, options)
options include:
NEW (clear list and add new file)
ADDFILE (add file to existing list)
REMOVEFILE (remove file from existing list)
www.SageLogix.com
Using Log MinerUsing Log Miner
Then, the Log Miner session must be startedto populate the V$LOGMNR_CONTENTS view with information from the files in the list
Procedure DBMS_LOGMNR.START_LOGMNR
Use parameters start-SCN/stop-SCN or start-time/stop-time to restrict to certain redo records
default: no restrictions, use the full list of files
The redo trail contains only numeric IDs for database objects (not symbolic names) so data dictionary info is necessary to translate to human-readable form:
Online data dictionary
Flat-file text extract of data dictionary
Redo stream contains data dictionary extracts (Oracle9i only on both source and mining side)
www.SageLogix.com
Using Log MinerUsing Log MinerOPTIONS parameter in START_LOGMNR procedure:
Oracle8i:USE_COLMAPSKIP_CORRUPTION
Oracle9i:SKIP_CORRUPTIONPRETTY_SQLDICT_FROM_ONLINE_CATALOGCOMMITTED_DATA_ONLYDDL_DICT_TRACKINGDICT_FROM_REDO_LOGSNO_SQL_DELIMITERCONTINUOUS_MINE
Procedure END_LOGMNRFinishes a Log Miner session
www.SageLogix.com
Using Log MinerUsing Log MinerUSE_COLMAP functionality in Oracle8i
Requires use of logmnr.opt fileMust be located in the same location (directory) as the data dictionary flat-fileFormat of column-mapping entries:colmap = schema table (1, column
[, 2, column [, …]] )maps specified columns to five place-holder columns in the end of the V$LOGMNR_CONTENTS view
PHn_NAME VARCHAR2(32)PHn_REDO VARCHAR2(4000)PHn_UNDO VARCHAR2(4000)
select * from v$logmnr_contentswhere seg_owner = ‘SCOTT’ and seg_name = ‘EMP’and ph1_name = ‘SAL’ and ph1_redo = ‘100000’and ph1_undo = ‘75000’;
www.SageLogix.com
Using Log MinerUsing Log Miner
Oracle9i does it differently, better…Function MINE_VALUE return VARCHAR2
Parameter SQL_REDO_UNDOFlag: constants REDO_VALUE or UNDO_VALUE
Parameter COLUMN_NAMEFully-qualified column name
Returns string with column valueDates always in format DD-MON-YYYY HH24:MI:SS.SS
Returns NULL if column not present or column has NULL value
How to tell the difference, when NULL is returned?
Function COLUMN_PRESENT return NUMBERParameter SQL_REDO_UNDOParameter COLUMN_NAMEReturns 1 if column is present, 0 if not present
www.SageLogix.com
Using Log MinerUsing Log Miner
So, what would this Oracle9i query return?
select dbms_logmnr.mine_value(redo_value,
'SCOTT.EMP.SAL')
from v$logmnr_contents
where dbms_logmnr.mine_value(redo_value,
'SCOTT.EMP.SAL') = ‘0’
or (dbms_logmnr.mine_value(redo_value,
'SCOTT.EMP.SAL') IS NULL
and dbms_logmnr.column_present(redo_value,
'SCOTT.EMP.SAL') = 1)
www.SageLogix.com
Using Log MinerUsing Log MinerV$LOGMNR_CONTENTS view
SCN NUMBER(15) system change numberTIMESTAMP DATE timestamp of redo vectorTHREAD# NUMBER redo log thread numberLOG_ID NUMBER redo log sequence #XIDUSN NUMBER XID (transaction ID) rollback
segment #XIDSLOT NUMBER XID transaction table slotXIDSQN NUMBER XID sequence # of slotRBASQN NUMBER RBA (redo byte address) log
seq #RBABLK NUMBER RBA block within fileRBABYTE NUMBER RBA byte offset within blockUBAFIL NUMBER UBA (undo byte address) file #UBABLK NUMBER UBA block within fileUBAREC NUMBER UBA record within blockUBASQN NUMBER UBA sequence of blockABS_FILE# NUMBER absolute file#REL_FILE# NUMBER relative (to tablespace) file#
www.SageLogix.com
Using Log MinerUsing Log MinerV$LOGMNR_CONTENTS view (cont’d)
DATA_BLK# NUMBER data block address (block #)
DATA_OBJ# NUMBER data object#DATA_DOBJ# NUMBER data block data object#SEG_OWNER VARCHAR2(30) segment ownerSEG_NAME VARCHAR2(81) segment nameSEG_TYPE NUMBER type of segmentSEG_TYPE_NAME VARCHAR2(32) name of type of segmentTABLESPACE_NAME VARCHAR2(30) segment’s tablespaceROW_ID VARCHAR2(18) row IDSESSION# NUMBER session IDSERIAL# NUMBER serial# of sessionUSER_NAME VARCHAR2(30) Oracle account nameSESSION_INFO VARCHAR2(4000) additional infoROLLBACK NUMBER 0=commit, 1=rollbackOPERATION VARCHAR2(30) SQL command typeSQL_REDO VARCHAR2(4000) SQL statementSQL_UNDO VARCHAR2(4000) “reverse” SQL stmtINFO VARCHAR2(32) informational msg
www.SageLogix.com
Enhanced data analysis
Enhanced data analysis
In Oracle9i, LogMiner has been enhanced to provide comprehensive Log Analysis for (almost) all types of data:
Index-organized tables
Clustered tables
Chained and migrated rows
LOBs and LONGs
direct-path loaded data
scalar object types
All DDL commands
Still missing:Collection object types (i.e. VARRAYs and NESTED TABLES)
www.SageLogix.com
Enhanced data analysis
Enhanced data analysis
To support redo logfile-based applications (such as logical standby databases), PK values and/or before-images may need to be added to the redo stream
Database supplemental loggingMinimal
Allows Log Miner to group REDO logs for individual DML statementsALTER DATABASE ADD SUPPLEMENTAL LOG
DATAPrimary key logging
Allows identification of rows logically rather than using ROWIDSRequirement for using Logical Standby database:
ALTER DATABASE ADD SUPPLEMENTAL DATA (PRIMARY KEY, UNIQUE KEY) COLUMNS;
www.SageLogix.com
Enhanced data analysis
Enhanced data analysis
Supplement logging (cont’d)Table supplemental logging
Logs column before-image values
An application might require that the before-image of the entire row be logged
Not just the columns being changed
www.SageLogix.com
New V$ viewsNew V$ views
V$LOGMNR_DICTIONARYShows info about dictionary being used (flat-file or online)
V$LOGMNR_LOGSShows info about log files being analyzed
V$LOGMNR_LOGFILENot documented! Shows info about logfiles being analyzed
V$LOGMNR_PARAMETERSShows options chosen for current Log Miner session
Parameter values passed to START_LOGMNR
V$LOGMNR_SESSIONNot documented! Information about each Log Miner session, including SESSION_ID for joining back to the V$SESSION view.Can be queried from outside an active Log Miner session.
www.SageLogix.com
New V$ viewsNew V$ viewsV$LOGMNR_PROCESS
Not documented! Information about the Oracle server process underlying the current Log Miner session.Cannot be queried from outside an active Log Miner session.
V$LOGMNR_TRANSACTIONNot documented! Information about the transaction listed within the current Log Miner session.Cannot be queried from outside an active Log Miner session
V$LOGMNR_REGIONNot documented! Information about internal memory structures used within the current Log Miner session.Cannot be queried from outside an active Log Miner session.
www.SageLogix.com
New V$ viewsNew V$ views
V$LOGMNR_CALLBACKNot documented! Information about internal memory structures used within the current Log Miner session.
Cannot be queried from outside an active Log Miner session.
V$LOGMNR_STATSNot documented! Usage statistics about each Log Miner session, intended to be joined to V$LOGMNR_SESSION and/or V$SESSION.
www.SageLogix.com
DocumentationDocumentation
Oracle8i/9i Server Administration manualChapter on “Using LogMiner to Analyze Online and Archived Redo Logs”
Oracle8i/9i Supplied Packages Reference manual
Chapters on “DBMS_LOGMNR” and “DBMS_LOGMNR_D”
Oracle9i Server Administration manualSection on “Database Supplemental Logging” and “Table Supplemental Logging”
Oracle9i Data Warehousing Guide manualChapter 15 on “Change Data Capture” (or “Streams”)
Oracle9i Supplied Packages Reference manualChapters on “DBMS_LOGMNR_CDC_PUBLISH” and “DBMS_LOGMNR_CDC_SUBSCRIBE”
www.SageLogix.com
DocumentationDocumentation
Metalink note #148616.1 – Oracle9i Log Miner New Features
Good overview with examples
PL/SQL source files located in $ORACLE_HOME/rdbms/admin
File dbmslm.sql
Package header source for DBMS_LOGMNR
File dbmslmd.sql
Package header source for DBMS_LOGMNR_D
File dbmslms.sql
Package header source for DBMS_LOGMNR_SESSION
More streamlined repackaging of DBMS_LOGMNR
Bug #2137007 filed to add some documentation this to standard doc set – still not visible (Feb 2003)
www.SageLogix.com
Q&AQ&APaper #536
Slides downloadable fromhttp://www.EvDBT.com/papers.htm
And http://www.SageLogix.com
Tim Gorman