FMW 11g Diagnostic Framework

7/25/2019 FMW 11g Diagnostic Framework

1/38

FMW Diagnostic Framework


2/38

22

Diagnostic Framework (DFW) Background

Database group has built a generic diagnosticframework to be adopted by all Oracle products

Primary goal of this framework is to reduce time toresolution of bugs/issues

Aids in detecting, diagnosing, and resolving problems F ! has adopted the concepts and parts of the

generic framework, and added "ava/F ! specifice#tensions

$argeted problems are critical errors caused by codebugs, metadata corruption, customer data corruption,deadlocked threads, and inconsistent state

%opyright & 2'(( Oracle and/or its affiliates) All rights reserved)


3/38

**

DFW Goals

First+failure diagnosis imiting damage and interruptions after a problem is

detected -educing problem diagnostic and resolution time .implifying customer interaction with Oracle .upport



4/38

DFW Tools and Tec nologies

Automatic capture of diagnostic data upon first failure .tandardi0ed log formats Diagnostic rules

1ncident detection log filter 1ncident packaging service 1P.3 1ntegration with !eb ogic Diagnostics Framework

! DF3 4 .upport !orkbench 1.. viewer



5/38

55

DFW !co"e and rele#ance to la$ered "roducts

DF! is applicable for 6 Oracle !eb ogic .erver with "-F ayered Fusion iddleware products that run on top of

Oracle !eb ogic .erver such as 6 Oracle 71 Oracle .OA .uite Oracle 7P4 Process anager Oracle 7A

4tc8) %omponents such as O9. and !eb %ache are

currently not covered)



6/38

::

De%initions Pro&lem 6 A problem is a critical error in the product) 4ach problem

has a uni;ue problem id and a key)

< Problem key 6 A te#t string that includes an error code

Incident 6 An incident is a single occurrence of a problem)

< Incident package ' An incident package package3 is a collection ofdata about incidents for one or more problems)

< Incident Packing Service 6 4nables creating and manipulating anincident package)

< Incident flood control 6 A flood controlled incident is one which doesnot generate an incident dump to prevent information overload)



7/38==

De%initions (contd )

We& ogic Diagnostic Framework (W DF) ' ! DFconsists of a set of components that work together to collect, archive,and access diagnostic information about a !eb ogic .erver instanceand its applications)

Har#ester' 9arvester in the ! DF component responsible forcollecting metrics from specified bean attributes)

Watc es 6 A watch is a rule which monitors weblogic server runtime

beans or log messages for a specified condition)

*oti%ication 6Once the watch condition is satisfied an associatednotification can be raised using . $P, .> P, " ?, or " . )



8/38@@

De%initions (contd )

+utomatic Diagnostic ,e"ositor$ (+D,) 6 is a file+based repository for diagnostic data such as traces, dumps, the alertlog, health monitor reports, and more)

+D,-I + $he AD- %ommand 1nterpreter AD-%13 is a utility thatenables you to investigate problems, and package and upload first+failure diagnostic data to Oracle .upport, all within a command+lineenvironment) AD-%1 also enables you to view the names of the tracefiles in the AD-, and to view the alert log with ? tags stripped, withand without content filtering)

W !T ' DF! has ! .$ commands for ;uerying Problems/1ncidentsacross ! . servers, view incident dump files, create manual incidents,e#ecute dump manually etc



9/38

DFW -om"onents

+utomatic Diagnostic ,e"ositor$ .a#a Diagnostic Dum"s .a#a Diagnostics Data /0tractor

Diagnostic Framework M&eans Diagnostic Framework (+D,) -ommand ine

Inter%ace (- I)



10/38('('

+utomatic Diagnostic ,e"ositor$ 1 +D,

.tores diagnostic data trace, dumps, metadata3 for

all Oracle products inside a single file+systemdirectory hierarchy AD- data is highly structured

etadata e)g) problem/incidents3 stored in

relationsDumps for incidents are performed in separate files

4ach !eb ogic .erver stores diagnostic data insubdirectories of its own home directory within the

AD-



11/38((((

!am"le +D, Director$ !tructure %or We& ogic



12/38(2(2

+D, Home 2 +D, Base

+D, ome + is the root directory for all diagnosticdata for a particular instance of an Oracleproduct/component)

+D, Base + AD- homes for differentproducts/components are grouped together under thesame root directory called the AD- base)



13/38(*(*

.a#a Diagnostic Dum"s

Perform targeted diagnostic data collection .imple "ava interface 1mplemented by components and applications -egistered at runtime %an be run manually and automatically Out Of $he 7o# Diagnostic dumps include6

$hreads 4#ecution %onte#t all active ones3 Active 9$$P re;uests %lass histogram D . etrics ogs by 4%1D3 ogs by timestamp up to +/B 5 minutes3 ! DF .erver 1mage



14/38((

.a#a Diagnostic Dum"s la$out


An incident on !eb ogic .erver may include the

following dumps6>ame6 Cvm)threadsDump Files6 Cvm threads( i()dmp

>ame6 dms)metricsDump Files6 dms metrics2 i()dmp

>ame6 odl);uicktraceDump Files6 odl ;uicktrace* i()dmp

>ame6 odl)logsDump Arguments6 ecidE'c a5'bdcb e (e(6+2f2@ 25f6(*2@ 'c@fb*6+@'''+'''''''''''''a dDump Files6 odl logs i()dmp

>ame6 odl)logsDump Arguments6 timestampE(*(:=2@=5=' 'Dump Files6 odl logs5 i()dmp

>ame6 wls)imageDump Files6 diagnostic image Admin.erver 2'(( ' 22 2( 5 (=)0ip


15/38(5(5

.a#a Diagnostics Data /0tractor (DD/)

Active in each !eb ogic .erver

-ule driven incident detection and creation 1ncident detection

"ava)util)logging log filter ! DF Diagnostic odule " ? notifications

1ncident creation Default set of dumps to e#ecute -ule based dump e#ecution

"ava AP1 for creating incidents athers incident meta data correlation keys, component info,

etc)3 Flood control .upport for creating manual incidents



16/38(:(:

.a#a Diagnostics Data /0tractor (-ontd )

Out Of $he 7o# rules detect 1>%1D4>$ 4--O- level logmessages

Out Of $he 7o# ! DF !atch/>otification %onfiguration6 Out of memory errors

Gncaught e#ceptions $hread deadlocks .tuck threads



17/38(=(=

Management Beans (MBeans) 3 .M4

DF! provides 7eans that can be used to configurethe Diagnostic Framework

%an configure an ! DF !atch and >otification tocreate an incident

4nable or disable the detection of incidents throughthe log files

4nable or disable flood control and setting parametersfor flood control

.i0e management to configure ma# si0e for allincidents



18/38

(@(@

Management Beans



19/38

((

+D,-I -ommand3 ine 5tilit$

AD- %ommand 1nterpreter

Gtility that enables users to investigate problems, andpackage

A list of common AD-%1 commands is included in asubse;uent slide)



20/38

2'2'

W !T and DFW

listProblems 3Problem 1d Problem Hey( 74A+('('2' I9$$PJ

list1ncidents idEK(L31ncident 1d Problem Hey 1ncident $ime2 74A+('('2' I9$$PJ Fri Feb 2: (*6 26'( PD$ 2'('( 74A+('('2' I9$$PJ $ue Feb 2* ':6(=6* PD$ 2'('



21/38

2(2(

W !T 2 DFW

show1ncident idEM(M31ncident 1d6 (Problem 1d6 (Problem Hey6 74A+('('2' I9$$PJ

1ncident $ime6 $ue Feb 2* ':6(=6* PD$ 2'('4rror essage 1d6 74A+('('2'4#ecution %onte#t6 ''''14#;GvyAhH75"N 4yf(AfdC:'''' iFlood %ontrolled6 false

Dump Files 6 dms ecidct#( i()dmp Cvm threads2 i()dmpdms metrics* i()dmp odl logs i()dmp odl logs5 i()dmpdiagnostic image Admin.erver 2'(' '2 2* ': (= 2)0ipreadme)t#t



22/38

2222

W !T 2 DFW

e#ecuteDump nameEMdms)metricsM,outputFileEM/tmp/dumpout)t#tM, idEM(M3

Dump file dms metrics( i()dmp added to incident (


Incident reation 6 og %ilter


23/38

2*%opyright & 2'(( Oracle and/or its affiliates) All rights reserved)

Incident -reation 6 og %ilter

loada""lication,ules1register dum"s

. 5 17Dlogging 0ml

Incident Filter

create

log message

in#oke

DD/ loadDe%aultrules

,egister De%aultdum"s

-reate incident8

+D,

create incident

/0ecutedum"s

/0ecutedum"s

Write message

!er#er log


24/38

22

Incident -reation Generated &$ W DF Watc*oti%ication

Fusion iddleware configures a ! DF Diagnosticsodule F !DF!

1t contains a set of !atch and >otification rules fordetecting a specific set of critical errors and creatingan incident for each occurrence of those errors)

7y default it contains the following watch conditions6 Deadlock .tuck$hread Gnchecked4#ception such as >ullPointer4#ception,

.tackOverflow4rror, or OutOf emory4rror %ustom !atch conditions can be added



25/38

2525

Incident -reation Generated &$ W DF Watc*oti%ication


Managed !er#er

Diagnostic,ules

Watc*oti%ication

W DF DiagnosticImage Bean

DFW Diagnostics Data /0tractor +D,

+""lication Dum" -om"onent Dum"

9

: ;

=

?

,egister

*oti%$

/0ecute /0ecute

In#oke

,egister

Write

Write

-reateIncident

@


26/38

2:2:

Incident -reation *oti%ication &$ W DF Watc*oti%ication



27/38

2@2@

Incident Packaging !er#ice

Package and intelligently gather 1ncident data -elated incidents packaged based on6

4%1D $imestamp %omponent

Prompt the user to dump additional data 1P. actions3 1ntrusive dumps %onfiguration info $est cases

1nitiate and package -DA report



28/38

*'*'

/M !u""ort Work&enc

anage reported problems .ystem alerted or user+reported anage life+cycle and history of problems found .ingle target view, group view, stack view, enterprise view

Present patches and workarounds when available %reate .ervice -e;uests Packages data using 1P. Gpload incident information and test cases to Oracle

Provide guided resolution for non+bug problems Automatically track and synchroni0e .ervice -e;ueststatus from Oracle


Work&enc 3 Domain !ummar$


29/38

*(

Work&enc 3 Domain !ummar$


Work&enc 3 Pro&lems


30/38

*2

Work&enc 3 Pro&lems


W k& 3 P &l D il


31/38

****

Work&enc 3 Pro&lem Details


Work&enc 3 Incident Details


32/38

**

Work&enc 3 Incident Details



33/38

*5*5

Work&enc 3 Package



34/38

*:*:

+D,-I -ommands

%ommon AD-%1 commands include6

94 P .4$ 7A.4 .4$ 9O 4PA$9 .9O! A 4-$ .9O! P-O7 4 .9O! 1>%1D4>$ .9O! $-A%4F1 4 1P. %-4A$4 PA%HA 4 1P. 4>4-A$4 PA%HA 4 PG- 4



35/38

*=*=

Manual Incident creation and "ackaging

create1ncident message1dEM D.+5'5''M,descriptionEMsample incidentM31ncident 1d6 ( Problem 1d6 Problem Hey6 D.+5'5'' I A>GA J

1ncident $ime6 2*rd February 2'(' ((6556 5 $ 4rror essage 1d6 D.+5'5''Flood %ontrolled6 false

! 9O 4/wlserver (')*/server/adr/adrci .et AD- 7A.4 and AD- 9O 4PA$9 variables 1P. %-4A$4 PA%HA 4 1>%1D4>$ (

%reated package ( based on incident id (, correlation level typical 1P. 4>4-A$4 PA%HA 4 ( in /tmp

enerated package ( in file /tmp/74A**=!eb 2'(''22*(*2*(5 %O ()0ip,mode complete



36/38

*@*@

,D+ 2 DFW

,emote Diagnostic +ssistant is a command line diagnostic tool whichprovides a comprehensive picture of the middleware environment)

1f the diagnostic dump information captured in the incident package

proves to be insufficient for analy0ing the root cause, an -DA reportmay be generated as additional input for support)



37/38

**

+dditional ,e%erences DFW inks

DF! ProCect page DF! .upport 1nternals DF! %ustomer Documentation DF! 1nternal Dump Documentation DF! Functional specification DF! !eb ogic .erver port functional specification DF! Developers guide DF! "avaDoc AD- %ustomer Documentation

FMW Diagnostics

Diagnostics .py Application F ! Diagnostics Ora$weet roup KfmwdiagL F ! Diagnostics %onnect roup ailing list6 fmwdiag ww

%opyright & 2'(' Oracle and/or its affiliates) All rights reserved)


38/38

Date post:	26-Feb-2018
Category:	Documents
Upload:	andreea-mihaela
View:	225 times
Download:	0 times

FMW 11g Diagnostic Framework

Documents