Instalacion Agente Java Introscope 9.5

Java Agent Implementation Guide Release 9.1.0.1

CA Application Performance Management

This documentation, which includes embedded help systems and electronically distributed materials, (hereinafter referred to as the “Documentation”) is for your informational purposes only and is subject to change or withdrawal by CA at any time.

This Documentation may not be copied, transferred, reproduced, disclosed, modified or duplicated, in whole or in part, without the prior written consent of CA. This Documentation is confidential and proprietary information of CA and may not be disclosed by you or used for any purpose other than as may be permitted in (i) a separate agreement between you and CA governing your use of the CA software to which the Documentation relates; or (ii) a separate confidentiality agreement between you and CA.

Notwithstanding the foregoing, if you are a licensed user of the software product(s) addressed in the Documentation, you may print or otherwise make available a reasonable number of copies of the Documentation for internal use by you and your employees in connection with that software, provided that all CA copyright notices and legends are affixed to each reproduced copy.

The right to print or otherwise make available copies of the Documentation is limited to the period during which the applicable license for such software remains in full force and effect. Should the license terminate for any reason, it is your responsibility to certify in writing to CA that all copies and partial copies of the Documentation have been returned to CA or destroyed.

TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENTATION “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT WILL CA BE LIABLE TO YOU OR ANY THIRD PARTY FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENTATION, INCLUDING WITHOUT LIMITATION, LOST PROFITS, LOST INVESTMENT, BUSINESS INTERRUPTION, GOODWILL, OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH LOSS OR DAMAGE.

The use of any software product referenced in the Documentation is governed by the applicable license agreement and such license agreement is not modified in any way by the terms of this notice.

The manufacturer of this Documentation is CA.

Provided with “Restricted Rights.” Use, duplication or disclosure by the United States Government is subject to the restrictions set forth in FAR Sections 12.212, 52.227-14, and 52.227-19(c)(1) - (2) and DFARS Section 252.227-7014(b)(3), as applicable, or their successors.

Copyright © 2012 CA. All rights reserved. All trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.

CA Technologies Product References

This document references the following CA Technologies products and features:

■ CA Application Performance Management (CA APM)

■ CA Application Performance Management ChangeDetector (CA APM ChangeDetector)

■ CA Application Performance Management ErrorDetector (ErrorDetector)

■ CA Application Performance Management for CA Database Performance (CA APM for CA Database Performance)

■ CA Application Performance Management for CA SiteMinder (CA APM for CA SiteMinder)

■ CA Application Performance Management for CA SiteMinder Web Access Manager (CA APM for CA SiteMinder Web Access Manager)

■ CA Application Performance Management for CA SYSVIEW® (CA APM for CA SYSVIEW)

■ CA Application Performance Management for IBM CICS Transaction Gateway (CA APM for IBM CICS Transaction Gateway)

■ CA Application Performance Management for IBM WebSphere Application Server (CA APM for IBM WebSphere Application Server)

■ CA Application Performance Management for IBM WebSphere Distributed Environments (CA APM for IBM WebSphere Distributed Environments)

■ CA Application Performance Management for IBM WebSphere MQ (CA APM for IBM WebSphere MQ)

■ CA Application Performance Management for IBM WebSphere Portal (CA APM for IBM WebSphere Portal)

■ CA Application Performance Management for IBM WebSphere Process Server (CA APM for IBM WebSphere Process Server)

■ CA Application Performance Management for IBM z/OS® (CA APM for IBM z/OS)

■ CA Application Performance Management for Microsoft SharePoint (CA APM for Microsoft SharePoint)

■ CA Application Performance Management for Oracle Databases (CA APM for Oracle Databases)

■ CA Application Performance Management for Oracle Service Bus (CA APM for Oracle Service Bus)

■ CA Application Performance Management for Oracle WebLogic Portal (CA APM for Oracle WebLogic Portal)

■ CA Application Performance Management for Oracle WebLogic Server (CA APM for Oracle WebLogic Server)

■ CA Application Performance Management for SOA (CA APM for SOA)

■ CA Application Performance Management for TIBCO BusinessWorks (CA APM for TIBCO BusinessWorks)

■ CA Application Performance Management for TIBCO Enterprise Message Service (CA APM for TIBCO Enterprise Message Service)

■ CA Application Performance Management for Web Servers (CA APM for Web Servers)

■ CA Application Performance Management for webMethods Broker (CA APM for webMethods Broker)

■ CA Application Performance Management for webMethods Integration Server (CA APM for webMethods Integration Server)

■ CA Application Performance Management Integration for CA CMDB (CA APM Integration for CA CMDB)

■ CA Application Performance Management Integration for CA NSM (CA APM Integration for CA NSM)

■ CA Application Performance Management LeakHunter (CA APM LeakHunter)

■ CA Application Performance Management Transaction Generator (CA APM TG)

■ CA Customer Experience Manager (CA CEM)

■ CA Embedded Entitlements Manager (CA EEM)

■ CA eHealth® Performance Manager (CA eHealth)

■ CA Insight™ Database Performance Monitor for DB2 for z/OS

■ CA Introscope® (CA Introscope)

■ CA SiteMinder®

■ CA Spectrum® Infrastructure Manager (CA Spectrum)

■ CA SYSVIEW® Performance Management (CA SYSVIEW)

Contact CA Technologies

Contact CA Support

For your convenience, CA Technologies provides one site where you can access the information you need for your Home Office, Small Business, and Enterprise CA Technologies products. At http://ca.com/support, you can access the following:

■ Online and telephone contact information for technical assistance and customer services

■ Information about user communities and forums

■ Product and documentation downloads

■ CA Support policies and guidelines

■ Other helpful resources appropriate for your product

Providing Feedback About Product Documentation

If you have comments or questions about CA Technologies product documentation, you can send a message to [email protected].

If you would like to provide feedback about CA Technologies product documentation, complete our short customer survey, which is available on the CA Support website at http://ca.com/docs.

http://www.ca.com/support

mailto:[email protected]

http://www.ca.com/docs

http://www.ca.com/docs

Product Documentation

CA APM documentation includes information for CA APM, CA Introscope, CA CEM, and CA APM extensions and integrations.

You can view and search all the titles in the CA APM documentation set from the CA APM bookshelf on the CA Support Online (CSO) website.

The following list shows the documentation specific to CA APM.

■ CA APM Release Notes — Release summary information for CA APM.

Note: In previous releases, this document was titled CA APM ReadMe.

■ CA APM Readme — Important last-minute release information for CA APM; available on the CA APM software download site or CA Support site.

Note: In previous releases, this document was titled CA APM Known Issues.

■ CA APM Installation and Upgrade Guide — Installation requirements; installing CA APM, including installing CA Introscope, Enterprise Manager, APM database, Workstation, WebView, CA CEM, TIM; upgrading from previous releases.

■ CA APM Overview Guide — A broad overview of CA APM components and architecture. Explains terms and concepts used in a CA APM deployment.

■ CA APM Configuration and Administration Guide — Combines configuration and administration information for CA Introscope and for CA CEM. CA Introscope and CA CEM properties are documented in the appendix.

■ CA APM Security Guide — Choosing and configuring CA APM, CA Introscope, and CA CEM security solutions. Includes information about Embedded Entitlements Manager.

■ CA APM Sizing and Performance Guide — Sizing, tuning, and capacity planning for your CA APM deployment and components.

■ CA APM Transaction Definition Guide — Transaction definition processes and procedures for CA APM; describes the necessary steps to record, define, and verify customer transactions.

The following list shows the documentation specific to CA Introscope. CA APM documentation is also pertinent for CA Introscope.

■ CA APM Java Agent Implementation Guide — Installation, configuration, and use of the CA APM Java Agent.

■ CA APM .NET Agent Implementation Guide — Installation, configuration, and use of the CA APM .NET Agent.

■ CA APM Environment Performance Agent Implementation Guide — Implementing Environment Performance Agent (EPAgent) with CA Introscope to monitor system information, including process availability, disk statistics, web application server and web server logs, Solaris KStat and HTTP service availability. The guide provides instructions for installing, configuring, and using EPAgent.

■ CA APM Workstation User Guide — Using CA Introscope dashboards, Investigator tree and application triage map, Transaction Tracer, and reporting. Includes CA Introscope metrics overview and descriptions.

■ CA APM WebView User Guide — Using WebView to view CA Introscope data in dashboards and the Investigator.

■ CA APM ChangeDetector User Guide — Using CA APM ChangeDetector to monitor and report changes in application files and configuration.

■ CA APM Transaction Generator Implementation Guide — Using the CA APM Transaction Generator (CA APM TG) to monitor the availability, health, and performance of web sites and web services from the perspective of a user attempting to access web sites. How to use the CA APM TG Agent to generate synthetic transactions that you can monitor using CA APM.

The following list shows the documentation specific to extensions and integrations.

■ CA APM API Reference Guide — Contains data and components managed within CA APM that is exposed to consumers with an application programming interface (API).

■ CA APM Catalyst Connector Guide — Installing and using the Catalyst Connector for CA APM.

■ CA APM for CA SiteMinder SNMP Collector Guide — Installing and configuring CA APM for CA SiteMinder SNMP Collector. Understanding the associated metrics.

■ CA APM for CA SiteMinder Web Access Manager Guide — Installing, configuring, and using CA APM for CA SiteMinder Web Access Manager.

■ CA APM for IBM CICS Transaction Gateway Guide — Installing, configuring, and using CA APM for IBM CICS Transaction Gateway.

■ CA APM for IBM WebSphere Application Server for Distributed Environments Guide — Installing, configuring, and using CA APM for IBM WebSphere Distributed Environments.

■ CA APM for IBM WebSphere Application Server for z/OS Guide — Installing, configuring, and using CA APM for IBM WebSphere for z/OS.

■ CA APM for IBM WebSphere MQ Guide — Using CA Introscope to view metrics from IBM WebSphere MQ.

■ CA APM for IBM WebSphere Portal Guide — Installing, configuring, and using CA APM for IBM WebSphere Portal.

■ CA APM for IBM z/OS Guide — Installing, configuring, and using CA APM for IBM z/OS.

■ CA APM for Microsoft SharePoint Guide — Installing and configuring CA APM for Microsoft SharePoint to monitor your SharePoint components during development, QA, staging, and production.

■ CA APM for Oracle Databases Guide — Installing, configuring, and using CA APM for Oracle Databases.

■ CA APM for Oracle WebLogic Portal Guide — Installing, configuring, and using CA APM for Oracle WebLogic Portal.

■ CA APM for Oracle WebLogic Server Guide — Installing, configuring, and using CA APM for Oracle WebLogic Server.

■ CA APM for SOA Implementation Guide — Installation and configuration information for using CA APM for SOA and SOA platform extensions with CA Introscope.

■ CA APM for Web Servers Guide — Installing, configuring, and using CA APM for Web Servers.

■ CA APM Integration for CA CMDB Guide — Installing, configuring, and using CA APM for CA CMDB.

■ CA APM Integration for CA Infrastructure Management Guide — Installing, configuring, and using CA APM for CA Infrastructure Management.

■ CA APM Integration for CA NSM Guide — Installing, configuring, and using CA APM for CA NSM.

■ CA Cross-Enterprise Application Performance Management Integration Guide — Installing, configuring, and using the SYSVIEW Agent extension, which allows you to manage application performance of distributed applications accessing mainframe back ends and trace transactions from distributed applications to mainframe CICS transactions. Allows you to monitor the health metrics of critical mainframe components.

■ CA Introscope SAP NetWeaver Conversion Guide — Using CA APM for SAP NetWeaver.

■ CA Introscope WebView User Guide for SAP — This guide is for SAP WebView users.

Note: In some product documentation, screenshots contain a logo or other reference to ‘Wily’, which CA Technologies has replaced with ‘APM.’ Ignore ‘Wily’ logos, as they have been removed in the current APM Workstation. References to ‘Wily’ in diagrams refer to CA Introscope.

Documentation Changes

The following documentation updates have been made since the last release of this documentation:

■ Java agent enhancements

– The Java agent architecture has been modified to provide more efficient correlation of traces in cross-process transactions.

– The Java agent directory structure (see page 38) has been modified to enable easier upgrades.

– Administrators can remotely issue commands from the Java agent to generate thread dumps (see page 67) as needed, directly from the Workstation user interface.

– The GC Monitor (see page 245) reports detailed information about Garbage Collectors and Memory Pools to detect memory-related issues that could adversely affect performance.

■ Java agent properties

New properties:

– introscope.agent.threaddump.enable

– introscope.agent.threaddump.deadlockpoller.enable

– introscope.agent.threaddump.deadlockpollerinterval

– introscope.agent.threaddump.MaxStackElements

– introscope.agent.primary.net.interface.name

Changed properties:

– introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds—This property can now specify the number of seconds between attempts by the Java agent to reconnect to:

■ the primary Enterprise Manager based on the Java agent profile introscope.agent.enterprisemanager.connectionorder property value.

■ any Enterprise Manager based on the loadbalancing.xml configuration.

■ Documentation changes

– CA APM product names have been updated to reflect the current naming conventions.

– This guide is renamed from the Introscope Java Agent Guide to the CA APM Java Agent Implementation Guide.

– Virtual agents are not agents; they are conglomerations of agent metrics, therefore Enterprise Manager-related entities. Information about configuring and using virtual agents has been moved to the CA APM Configuration and Administration Guide.

Understanding Directory and File Name Conventions

This guide uses the following conventions in file names and directory paths:

Convention Refers to

<Agent_Home>

The top-level directory where the CA Introscope agent is installed. This directory is typically named wily.

<APM_Db_Home> The top-level directory where the APM database is installed, when referring to Database-only installations.

<AppServer_Home> The top-level directory where your application server is installed. This directory is often the same as <Agent_Home>.

<EM_Home> The top-level directory where the Enterprise Manager is installed.

<ProductName_Home> The installation directory of a third-party product or type of application. For example, it is the installation directory of the application server if you are using WebLogic.

<version> Version-specific identifier included in file names or displayed in the user interface.

For example, the following file name:

com.wily.introscope.soa.dependencymap_<version>.jar

represents a version-specific file name, such as:

com.wily.introscope.soa.dependencymap_9.1.0.jar

<File_Name><VersionNumber><Operating System or other identifier>.FileType

A file name that includes specific identifying information.

For example, if you extract files from a tar package for CA APM ChangeDetector 9.1.0.0 on a UNIX operating system, then download this file:

ChangeDetector9.1.0.0unix.tar

But, view it in this guide as follows:

ChangeDetector<VersionNumber>.unix.tar

Convention Refers to

Forward slash (/) path separators

The path separator used in directory names on your operating environment.

The forward slash (/) is used on UNIX platforms and in examples throughout this guide, but use the separator appropriate for your operating system.

Dollar sign ($) environment variables

The environment variable notation used on your operating system.

The dollar sign ($) is used in UNIX environments and in examples throughout this guide, but you use the character appropriate to your environment.

Contents 13

Contents

Chapter 1: Introduction to the Java Agent 25

About Introscope and Java Agents ............................................................................................................................. 25

Planning a Java Agent Deployment ............................................................................................................................ 26

Install and Evaluate the Default Functionality .................................................................................................... 26

Determine Configuration Requirements ............................................................................................................. 27

Define a Baseline Agent Profile with Appropriate Configuration Properties ...................................................... 27

Evaluate Agent Performance Overhead .............................................................................................................. 28

Validate and Deploy the Agent Configuration .................................................................................................... 28

Deploying the Java Agent ........................................................................................................................................... 28

Chapter 2: Installing and Configuring the Java Agent 29

View the Product Compatibility Guide ....................................................................................................................... 29

Before You Install the Agent ...................................................................................................................................... 30

Select the method for installing the Java agent ......................................................................................................... 30

Install the Java Agent Interactively ..................................................................................................................... 31

Install the Java Agent Silently .............................................................................................................................. 33

Install Manually Using Installation Archives........................................................................................................ 37

About the Java Agent Directory Structure ................................................................................................................. 38

How to instrument applications ................................................................................................................................. 39

Configuring the Application Server to Start the Java Agent ....................................................................................... 40

Configure Apache Tomcat to use the Java agent ................................................................................................ 40

Configure JBoss to use the Java agent ................................................................................................................ 41

Configure Oracle WebLogic to use the Java agent .............................................................................................. 42

Configure WebLogic with JRockit JVM to Use the Java Agent ............................................................................ 46

Configure WebLogic with JRocket JVM to View Socket Metrics ......................................................................... 47

Configure IBM WebSphere to use the Java agent............................................................................................... 47

Configure Oracle Application Server to use the Java agent ................................................................................ 55

Configure GlassFish 2.1 to use the Java agent .................................................................................................... 56

Configure SAP Netweaver 7.1 to use the Java agent .......................................................................................... 56

Configuring the connection to the Enterprise Manager ............................................................................................ 57

Connect to the Enterprise Manager Using a Direct Socket Connection ............................................................. 57

Connect to the Enterprise Manager with HTTP tunneling .................................................................................. 58

Configure a proxy server for HTTP tunneling ...................................................................................................... 59

Connect to the Enterprise Manager with HTTPS tunneling ................................................................................ 59

Connect to the Enterprise Manager over SSL ..................................................................................................... 60

Configure Agent Load Balancing ......................................................................................................................... 61

14 Java Agent Implementation Guide

Upgrading multiple agent types ................................................................................................................................. 61

Uninstalling the Java agent ........................................................................................................................................ 62

Uninstalling the Java agent from z/OS ................................................................................................................ 63

Chapter 3: Configuring Agent Properties 65

How to modify communication with Enterprise Manager ......................................................................................... 65

How to configure backup Enterprise Managers and failover properties ................................................................... 65

How to enable and use additional GC metrics ........................................................................................................... 67

How to enable and configure thread dumps ............................................................................................................. 67

Chapter 4: AutoProbe and ProbeBuilding Options 71

AutoProbe and ProbeBuilding overview .................................................................................................................... 71

Unsupported instrumentation methods ............................................................................................................. 72

Configuring ProbeBuilding ......................................................................................................................................... 72

Full or typical tracing options .............................................................................................................................. 72

Dynamic ProbeBuilding ....................................................................................................................................... 73

Dynamic ProbeBuilding vs. dynamic instrumentation ........................................................................................ 75

ProbeBuilding class hierarchies (JVM 1.5) .......................................................................................................... 76

Removing line numbers in bytecode ................................................................................................................... 79

Chapter 5: ProbeBuilder Directives 81

ProbeBuilder Directives Overview ............................................................................................................................. 81

Components traced by the default PBDs ............................................................................................................ 82

Default PBD Files ................................................................................................................................................. 83

Default PBD Files From Previous Releases .......................................................................................................... 86

Default PBL files .................................................................................................................................................. 86

Default tracer groups and toggles files ............................................................................................................... 86

Turning tracer groups on or off ........................................................................................................................... 95

Adding classes to a tracer group ......................................................................................................................... 96

EJB naming .......................................................................................................................................................... 98

Using the IntroscopeAgent.profile, PBLs, and PBDs together .................................................................................... 99

Applying ProbeBuilder Directives ............................................................................................................................... 99

Using JVM AutoProbe ....................................................................................................................................... 100

Using the ProbeBuilder Wizard or command-line ProbeBuilder ...................................................................... 100

Instrumenting with new and changed PBDs ..................................................................................................... 100

Creating custom tracers ........................................................................................................................................... 102

Using a custom BlamePointTracer tracer for common metrics ........................................................................ 102

Directive names and arguments used in tracer syntax ..................................................................................... 103

Commonly used tracer names and examples ................................................................................................... 105

Advanced single-metric tracers......................................................................................................................... 107

Contents 15

Skip directives ................................................................................................................................................... 110

Counting object instances ................................................................................................................................. 110

Turning on InstrumentPoint directives ............................................................................................................. 111

Combining custom tracers ................................................................................................................................ 111

Instrumenting and inheritance ......................................................................................................................... 111

Java 1.5 annotations ......................................................................................................................................... 112

Using Blame Tracers to mark blame points.............................................................................................................. 113

Blame Tracers .................................................................................................................................................... 113

High agent CPU overhead from deep nested frontend transactions ................................................................ 114

Custom FrontendMarker directive.................................................................................................................... 114

Blame Tracers in standard PBDs ....................................................................................................................... 115

Boundary Blame and Oracle backends ............................................................................................................. 115

Chapter 6: Java Agent Naming 117

Understanding the Java Agent name ....................................................................................................................... 117

How the agent determines its name ................................................................................................................. 118

How Introscope resolves agent naming conflicts ............................................................................................. 119

Agent naming considerations for clustered applications ......................................................................................... 120

Specifying an agent name using a Java system property ......................................................................................... 120

Specifying an agent name using a system property key .......................................................................................... 121

Obtaining an agent name from the application server ............................................................................................ 121

Application servers that support agent naming ................................................................................................ 121

Automatic agent naming .......................................................................................................................................... 122

Automatic agent naming and renamed agents ................................................................................................. 123

Advanced automatic agent naming options ..................................................................................................... 123

Enabling cloned agent naming in clustered environments ...................................................................................... 124

Cloned agent naming scenario .......................................................................................................................... 125

Configuring unique names for application instances ........................................................................................ 125

Application triage map and the agent name ............................................................................................................ 125

Chapter 7: Java Agent Monitoring and Logging 127

Configuring agent connection metrics ..................................................................................................................... 127

Socket metrics .......................................................................................................................................................... 128

Restricting socket and SSL metric collection ..................................................................................................... 128

Fine-tuning socket and SSL metric collection .................................................................................................... 129

SSL, NIO, and socket tracing in the application triage map .............................................................................. 129

Turning off socket and SSL metric collection .................................................................................................... 130

Backwards compatibility ................................................................................................................................... 131

Configuring logging options ..................................................................................................................................... 131

Running the agent in verbose mode ................................................................................................................. 132

Redirecting agent output to a file ..................................................................................................................... 132


Changing the name or location of the agent logfile .......................................................................................... 133

Agent log files and automatic agent naming .................................................................................................... 133

Rolling up logs by date or size ........................................................................................................................... 134

Managing ProbeBuilder Logs ................................................................................................................................... 135

Command-line ProbeBuilder and ProbeBuilder Wizard log name and location ............................................... 135

AutoProbe log name and location .................................................................................................................... 136

Chapter 8: Configuring LeakHunter and ErrorDetector 137

LeakHunter ............................................................................................................................................................... 137

How LeakHunter works ..................................................................................................................................... 138

What LeakHunter tracks in Java ........................................................................................................................ 138

What LeakHunter does not track ...................................................................................................................... 139

System and version requirements .................................................................................................................... 139

Enabling and disabling LeakHunter .......................................................................................................................... 140

Configuring LeakHunter properties .......................................................................................................................... 140

Ignoring collections that cause poor performance ........................................................................................... 142

Running LeakHunter ................................................................................................................................................. 143

Identifying potential leaks with collection IDs ......................................................................................................... 143

LeakHunter log file ................................................................................................................................................... 144

Potential leak first identified log entry ............................................................................................................. 144

Identified potential leak stops leaking log entry ............................................................................................... 145

Identified potential leak is leaking again log entry ........................................................................................... 146

LeakHunter timeout log entry ........................................................................................................................... 146

Using LeakHunter ..................................................................................................................................................... 146

ErrorDetector ........................................................................................................................................................... 147

Types of errors .................................................................................................................................................. 147

How ErrorDetector works ................................................................................................................................. 148

Enabling ErrorDetector in the Java Agent ................................................................................................................ 148

Configuring ErrorDetector options ........................................................................................................................... 149

Advanced error data capture ................................................................................................................................... 150

Defining new error types.......................................................................................................................................... 150

ExceptionErrorReporter .................................................................................................................................... 150

MethodCalledErrorReporter ............................................................................................................................. 151

ThisErrorReporter ............................................................................................................................................. 151

HTTPErrorCodeReporter ................................................................................................................................... 152

Using error tracer directives with caution ........................................................................................................ 152

Using ErrorDetector ................................................................................................................................................. 152

Chapter 9: Configuring Boundary Blame 153

Understanding Boundary Blame .............................................................................................................................. 153

Using URL groups ..................................................................................................................................................... 153

Contents 17

Defining keys for URL groups ............................................................................................................................ 154

Defining membership of each URL group ......................................................................................................... 154

Define the name for a URL group ..................................................................................................................... 155

Advanced naming techniques for URL groups (optional) ................................................................................. 155

Running the URLGrouper .................................................................................................................................. 159

Using Blame tracers ................................................................................................................................................. 159

Chapter 10: Configuring Transaction Trace Options 161

Controlling automatic Transaction Tracing behavior ............................................................................................... 161

Transaction Trace component clamp ................................................................................................................ 161

Transaction trace sampling ............................................................................................................................... 162

Agent heap sizing .............................................................................................................................................. 162

Configuring cross-process Transaction Tracing ........................................................................................................ 163

Extending transaction trace data collection ............................................................................................................. 163

About User ID data ............................................................................................................................................ 163

About servlet request data ............................................................................................................................... 164

Configuring Agent to collect additional transaction trace data ........................................................................ 164

Disabling the capture of stalls as Events .................................................................................................................. 166

Chapter 11: Configuring the Introscope SQL Agent 167

The SQL Agent overview .......................................................................................................................................... 167

The SQL Agent files ................................................................................................................................................... 168

SQL statement normalization ................................................................................................................................... 168

How poorly written SQL statements create metric explosions......................................................................... 168

SQL statement normalization options .............................................................................................................. 170

Turn Off Statement Metrics ..................................................................................................................................... 176

SQL metrics............................................................................................................................................................... 176

Chapter 12: Enabling JMX Reporting 179

Java Agent JMX support ........................................................................................................................................... 179

Introscope support for WebLogic 9.0 JMX metrics ........................................................................................... 180

Default JMX metric conversion process ................................................................................................................... 180

Using primary key conversion to streamline JMX metrics ....................................................................................... 181

Managing metric volume with JMX filters ............................................................................................................... 182

JMX filters for WebLogic ................................................................................................................................... 182

Configuring JMX reporting ....................................................................................................................................... 183

Enabling JSR-77 data for WAS 6.x ............................................................................................................................. 184


Chapter 13: Configuring Platform Monitoring 187

Understanding platform monitors ........................................................................................................................... 187

Enabling platform monitors on Windows Server 2003 ............................................................................................ 189

Enabling platform monitors on AIX .......................................................................................................................... 189

Disabling platform monitors .................................................................................................................................... 190

Configure Permissions to Access Platform Monitors on HP-UX ............................................................................... 191

Troubleshooting platform monitoring ..................................................................................................................... 191

Troubleshooting platform monitoring on Windows ......................................................................................... 192

Appendix A: Java Agent Properties 193

Configuring the IntroscopeAgent.profile location ................................................................................................... 194

Command-line property overrides ........................................................................................................................... 195

Agent failover ........................................................................................................................................................... 196

introscope.agent.enterprisemanager.connectionorder ................................................................................... 196

introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds ............................................................ 197

Agent HTTP tunneling .............................................................................................................................................. 198

Agent HTTP tunneling for proxy servers ........................................................................................................... 198

Agent HTTPS tunneling...................................................................................................................................... 200

Agent memory overhead ......................................................................................................................................... 201

introscope.agent.reduceAgentMemoryOverhead ............................................................................................ 202

Agent metric aging ................................................................................................................................................... 202

Configuring agent metric aging ......................................................................................................................... 203

Agent metric clamp .................................................................................................................................................. 206

introscope.agent.metricClamp ......................................................................................................................... 206

Agent naming ........................................................................................................................................................... 207

introscope.agent.agentAutoNamingEnabled .................................................................................................... 208

introscope.agent.agentAutoNamingMaximumConnectionDelayInSeconds .................................................... 208

introscope.agent.agentAutoRenamingIntervalInMinutes ................................................................................ 209

introscope.agent.agentName ........................................................................................................................... 209

introscope.agent.agentNameSystemPropertyKey ............................................................................................ 210

introscope.agent.disableLogFileAutoNaming ................................................................................................... 210

introscope.agent.clonedAgent .......................................................................................................................... 211

introscope.agent.customProcessName ............................................................................................................ 211

introscope.agent.defaultProcessName ............................................................................................................. 212

introscope.agent.display.hostName.as.fqdn .................................................................................................... 212

Agent recording (business recording) ...................................................................................................................... 212

introscope.agent.bizRecording.enabled ........................................................................................................... 213

Agent thread priority ................................................................................................................................................ 213

introscope.agent.thread.all.priority .................................................................................................................. 214

Agent to Enterprise Manager connection ................................................................................................................ 214

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT ................................................................. 214

Contents 19

introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT ................................................................. 215

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT .................................................. 215

introscope.agent.enterprisemanager.transport.tcp.local.ipaddress.DEFAULT ................................................ 216

introscope.agent.enterprisemanager.transport.tcp.local.port.DEFAULT ......................................................... 216

Application triage map ............................................................................................................................................. 216

introscope.agent.appmap.enabled ................................................................................................................... 217

introscope.agent.appmap.metrics.enabled ...................................................................................................... 217

introscope.agent.appmap.queue.size ............................................................................................................... 218

introscope.agent.appmap.queue.period .......................................................................................................... 218

introscope.agent.appmap.intermediateNodes.enabled................................................................................... 219

Application Triage Map and Catalyst Integration ..................................................................................................... 219

Configure the Ability to Send Information ........................................................................................................ 219

Configure a List of Available Networks ............................................................................................................. 220

Application triage map business transaction POST parameters .............................................................................. 221

introscope.agent.bizdef.matchPost .................................................................................................................. 221

Known limitations ............................................................................................................................................. 222

Application triage map managed socket configuration ........................................................................................... 223

introscope.agent.sockets.managed.reportToAppmap ..................................................................................... 224

introscope.agent.sockets.managed.reportClassAppEdge................................................................................. 224

introscope.agent.sockets.managed.reportMethodAppEdge ............................................................................ 225

introscope.agent.sockets.managed.reportClassBTEdge ................................................................................... 225

introscope.agent.sockets.managed.reportMethodBTEdge .............................................................................. 226

AutoProbe ................................................................................................................................................................ 226

introscope.autoprobe.directivesFile ................................................................................................................. 226

introscope.autoprobe.enable ........................................................................................................................... 227

introscope.autoprobe.logfile ............................................................................................................................ 227

Bootstrap Classes Instrumentation Manager ........................................................................................................... 228

introscope.bootstrapClassesManager.enabled ................................................................................................ 228

introscope.bootstrapClassesManager.waitAtStartup ....................................................................................... 228

CA CEM Agent Profile Properties ............................................................................................................................. 229

introscope.autoprobe.directivesFile ................................................................................................................. 229

introscope.agent.remoteagentconfiguration.allowedFiles .............................................................................. 230

introscope.agent.remoteagentconfiguration.enabled ..................................................................................... 231

introscope.agent.decorator.enabled ................................................................................................................ 232

introscope.agent.decorator.security ................................................................................................................ 232

introscope.agent.cemtracer.domainconfigfile.................................................................................................. 233

introscope.agent.cemtracer.domainconfigfile.reloadfrequencyinminutes ...................................................... 233

Configure the Session ID Collection .................................................................................................................. 234

ChangeDetector configuration ................................................................................................................................. 235

introscope.changeDetector.enable................................................................................................................... 235

introscope.changeDetector.agentID ................................................................................................................. 235

introscope.changeDetector.rootDir .................................................................................................................. 236


introscope.changeDetector.isengardStartupWaitTimeInSec............................................................................ 236

introscope.changeDetector.waitTimeBetweenReconnectInSec....................................................................... 236

introscope.changeDetector.profile ................................................................................................................... 237

introscope.changeDetector.profileDir .............................................................................................................. 237

introscope.changeDetector.compressEntries.enable ....................................................................................... 238

introscope.changeDetector.compressEntries.batchSize .................................................................................. 238

Cross-process tracing in WebLogic Server ............................................................................................................... 238

introscope.agent.weblogic.crossjvm ................................................................................................................. 239

Cross-process transaction trace ............................................................................................................................... 239

introscope.agent.transactiontracer.tailfilterPropagate.enable ........................................................................ 239

Dynamic instrumentation ........................................................................................................................................ 240

introscope.autoprobe.dynamicinstrument.enabled ......................................................................................... 240

autoprobe.dynamicinstrument.pollIntervalMinutes ........................................................................................ 240

introscope.autoprobe.dynamicinstrument.classFileSizeLimitInMegs .............................................................. 241

introscope.autoprobe.dynamic.limitRedefinedClassesPerBatchTo .................................................................. 241

introscope.agent.remoteagentdynamicinstrumentation.enabled ................................................................... 242

introscope.autoprobe.dynamicinstrument.pollIntervalMinutes ...................................................................... 242

ErrorDetector ........................................................................................................................................................... 243

introscope.agent.errorsnapshots.enable .......................................................................................................... 243

introscope.agent.errorsnapshots.throttle ........................................................................................................ 243

introscope.agent.errorsnapshots.ignore.<index> ............................................................................................. 244

Extensions ................................................................................................................................................................ 244

introscope.agent.extensions.directory ............................................................................................................. 244

introscope.agent.common.directory ................................................................................................................ 245

GC Monitor ............................................................................................................................................................... 245

introscope.agent.gcmonitor.enable.................................................................................................................. 245

Java NIO .................................................................................................................................................................... 246

Buffers ............................................................................................................................................................... 246

Channels ............................................................................................................................................................ 247

NIODatagramTracing metrics ............................................................................................................................ 247

Restricting Java NIO metrics ............................................................................................................................. 248

JMX ........................................................................................................................................................................... 252

introscope.agent.jmx.enable ............................................................................................................................ 252

introscope.agent.jmx.ignore.attributes ............................................................................................................ 253

introscope.agent.jmx.name.filter ..................................................................................................................... 253

introscope.agent.jmx.name.jsr77.disable ......................................................................................................... 254

introscope.agent.jmx.name.primarykeys ......................................................................................................... 255

introscope.agent.jmx.excludeStringMetrics ..................................................................................................... 256

LeakHunter ............................................................................................................................................................... 256

introscope.agent.leakhunter.collectAllocationStackTraces .............................................................................. 257

introscope.agent.leakhunter.enable ................................................................................................................. 258

introscope.agent.leakhunter.leakSensitivity..................................................................................................... 258

Contents 21

introscope.agent.leakhunter.logfile.append .................................................................................................... 259

introscope.agent.leakhunter.logfile.location .................................................................................................... 259

introscope.agent.leakhunter.timeoutInMinutes .............................................................................................. 260

introscope.agent.leakhunter.ignore.<number> ............................................................................................... 260

Logging ..................................................................................................................................................................... 261

log4j.logger.IntroscopeAgent ............................................................................................................................ 262

log4j.appender.logfile.File................................................................................................................................. 263

log4j.logger.IntroscopeAgent.inheritance ........................................................................................................ 263

log4j.appender.pbdlog.File ............................................................................................................................... 264

log4j.appender.pbdlog ...................................................................................................................................... 264

log4j.appender.pbdlog.layout ........................................................................................................................... 265

log4j.appender.pbdlog.layout.ConversionPattern ............................................................................................ 265

log4j.additivity.IntroscopeAgent.inheritance ................................................................................................... 266

Metric count ............................................................................................................................................................. 266

introscope.ext.agent.metric.count ................................................................................................................... 267

Multiple inheritance ................................................................................................................................................. 267

introscope.autoprobe.hierarchysupport.enabled ............................................................................................ 268

introscope.autoprobe.hierarchysupport.runOnceOnly .................................................................................... 268

introscope.autoprobe.hierarchysupport.pollIntervalMinutes .......................................................................... 269

introscope.autoprobe.hierarchysupport.executionCount ................................................................................ 269

introscope.autoprobe.hierarchysupport.disableLogging .................................................................................. 270

introscope.autoprobe.hierarchysupport.disableDirectivesChange .................................................................. 270

Platform monitoring ................................................................................................................................................. 270

introscope.agent.platform.monitor.system ...................................................................................................... 271

Remote configuration .............................................................................................................................................. 271

introscope.agent.remoteagentconfiguration.enabled ..................................................................................... 271

introscope.agent.remoteagentconfiguration.allowedFiles .............................................................................. 272

Security ..................................................................................................................................................................... 272

introscope.agent.decorator.security ................................................................................................................ 272

Servlet header decorator ......................................................................................................................................... 273

introscope.agent.decorator.enabled ................................................................................................................ 273

Socket metrics .......................................................................................................................................................... 273

introscope.agent.sockets.reportRateMetrics ................................................................................................... 274

introscope.agent.io.socket.client.hosts ............................................................................................................ 274

introscope.agent.io.socket.client.ports ............................................................................................................ 275

introscope.agent.io.socket.server.ports ........................................................................................................... 275

SQL Agent ................................................................................................................................................................. 275

introscope.agent.sqlagent.normalizer.extension ............................................................................................. 276

introscope.agent.sqlagent.normalizer.regex.matchFallThrough ...................................................................... 277

introscope.agent.sqlagent.normalizer.regex.keys ............................................................................................ 278

introscope.agent.sqlagent.normalizer.regex.key1.pattern .............................................................................. 278

introscope.agent.sqlagent.normalizer.regex.key1.replaceAll .......................................................................... 279


introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat ................................................................... 279

introscope.agent.sqlagent.normalizer.regex.key1.caseSensitive ..................................................................... 280

introscope.agent.sqlagent.sql.artonly .............................................................................................................. 280

introscope.agent.sqlagent.sql.rawsql ............................................................................................................... 280

introscope.agent.sqlagent.sql.turnoffmetrics .................................................................................................. 281

introscope.agent.sqlagent.sql.turnofftrace ...................................................................................................... 281

SSL communication .................................................................................................................................................. 282

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT ................................................................. 282

introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT ................................................................. 283

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT .................................................. 283

introscope.agent.enterprisemanager.transport.tcp.truststore.DEFAULT ........................................................ 284

introscope.agent.enterprisemanager.transport.tcp.trustpassword.DEFAULT ................................................. 284

introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT .......................................................... 284

introscope.agent.enterprisemanager.transport.tcp.keypassword.DEFAULT ................................................... 285

introscope.agent.enterprisemanager.transport.tcp.ciphersuites.DEFAULT ..................................................... 285

Stall metrics .............................................................................................................................................................. 285

introscope.agent.stalls.thresholdseconds ........................................................................................................ 286

introscope.agent.stalls.resolutionseconds ....................................................................................................... 286

Thread dumps .......................................................................................................................................................... 287

introscope.agent.threaddump.enable .............................................................................................................. 287

introscope.agent.threaddump.deadlockpoller.enable ..................................................................................... 288

introscope.agent.threaddump.deadlockpollerinterval ..................................................................................... 288

introscope.agent.threaddump.MaxStackElements .......................................................................................... 289

Transaction tracing ................................................................................................................................................... 289

introscope.agent.transactiontracer.parameter.httprequest.headers .............................................................. 290

introscope.agent.transactiontracer.parameter.httprequest.parameters ........................................................ 290

introscope.agent.transactiontracer.parameter.httpsession.attributes ............................................................ 291

introscope.agent.transactiontracer.userid.key ................................................................................................. 291

introscope.agent.transactiontracer.userid.method ......................................................................................... 292

introscope.agent.transactiontrace.componentCountClamp ............................................................................ 293

introscope.agent.crossprocess.compression .................................................................................................... 294

introscope.agent.crossprocess.compression.minlimit...................................................................................... 295

introscope.agent.crossprocess.correlationid.maxlimit ..................................................................................... 296

introscope.agent.transactiontracer.sampling.enabled ..................................................................................... 296

introscope.agent.transactiontracer.sampling.perinterval.count ...................................................................... 297

introscope.agent.transactiontracer.sampling.interval.seconds ....................................................................... 297

introscope.agent.transactiontrace.headFilterClamp ........................................................................................ 298

introscope.agent.ttClamp ................................................................................................................................. 299

URL grouping ............................................................................................................................................................ 299

introscope.agent.urlgroup.keys ........................................................................................................................ 300

introscope.agent.urlgroup.group.default.pathprefix........................................................................................ 300

introscope.agent.urlgroup.group.default.format ............................................................................................. 300

Contents 23

WebSphere PMI ....................................................................................................................................................... 301

introscope.agent.pmi.enable ............................................................................................................................ 302

introscope.agent.pmi.enable.alarmManager ................................................................................................... 302

introscope.agent.pmi.enable.bean ................................................................................................................... 303

introscope.agent.pmi.enable.cache .................................................................................................................. 303

introscope.agent.pmi.enable.connectionPool .................................................................................................. 304

introscope.agent.pmi.enable.hamanager ......................................................................................................... 304

introscope.agent.pmi.enable.j2c ...................................................................................................................... 305

introscope.agent.pmi.enable.jvmpi .................................................................................................................. 305

introscope.agent.pmi.enable.jvmRuntime ....................................................................................................... 306

introscope.agent.pmi.enable.objectPool .......................................................................................................... 306

introscope.agent.pmi.enable.orbPerf ............................................................................................................... 307

introscope.agent.pmi.enable.scheduler ........................................................................................................... 307

introscope.agent.pmi.enable.servletSessions................................................................................................... 308

introscope.agent.pmi.enable.system ................................................................................................................ 308

introscope.agent.pmi.enable.threadPool ......................................................................................................... 309

introscope.agent.pmi.enable.transaction ......................................................................................................... 309

introscope.agent.pmi.enable.webApp .............................................................................................................. 310

introscope.agent.pmi.enable.webServices ....................................................................................................... 310

introscope.agent.pmi.enable.wlm .................................................................................................................... 311

introscope.agent.pmi.enable.wsgw .................................................................................................................. 311

introscope.agent.pmi.filter.objref .................................................................................................................... 312

WLDF metrics ........................................................................................................................................................... 312

introscope.agent.wldf.enable ........................................................................................................................... 313

Appendix B: Alternative Methods for Instrumentation 315

Deploying the Java Agent on other application servers ........................................................................................... 315

Configuring Sun ONE 7.0 .......................................................................................................................................... 316

Configuring Oracle 10g 10.0.3 .................................................................................................................................. 317

Configuring WebLogic Server ................................................................................................................................... 318

Configuring HTTP servlet tracing .............................................................................................................................. 318

Creating an AutoProbe connector file ...................................................................................................................... 319

Running the AutoProbe Connector for a JVM ................................................................................................... 319

About running ProbeBuilder manually ..................................................................................................................... 322

AutoProbe for WebSphere 6.1 and 7.0 for z/OS ...................................................................................................... 322

Appendix C: Using the PBD Generator 325

About the CA PBD Generator ................................................................................................................................... 325

Configuring the PBD Generator ................................................................................................................................ 326

Required PBD Generator parameters ............................................................................................................... 326

Using the PBD Generator ......................................................................................................................................... 326


Appendix D: Using the Network Interface Utility 329

Determine Network Interface Names ...................................................................................................................... 329

Index 331


Chapter 1: Introduction to the Java Agent

This section contains the following topics:

About Introscope and Java Agents (see page 25) Planning a Java Agent Deployment (see page 26) Deploying the Java Agent (see page 28)

About Introscope and Java Agents

CA Introscope is an enterprise application performance management solution that enables you to monitor complex web applications in production environments 24x7, detect problems before they affect your customers, and resolve these issues quickly and collaboratively. A key part of the architecture for this solution is the low overhead agent.

The agent is a data gathering component of Introscope that collects detailed performance information about applications and the computing environment as transactions are executed. The Java agent collects this information from applications and resources running on Java Virtual Machines (JVMs) and sends the information to the Enterprise Manager for further processing.

The Java agent inserts probes into the bytecode of the components that make up the JVM that the application uses. Inserting the probes into the bytecode is part of the instrumentation process that enables the monitoring of your applications.

Instrumenting an application also requires tracers that are defined in ProbeBuilder Directive (PBD) files. The instructions or directives in PBD files identify the application components to monitor. The tracers identify the metrics the agent should collect from which probes as the application runs in the JVM. You can control what is monitored by changing the PBD files to suit your environment.

When you install the Java agent, several default PBD files are deployed to enable default monitoring of your environment. You can modify the default monitoring to achieve the balance of visibility and performance that you require. After the application is instrumented, the Java agent collects the data you are interested in and reports the data to the Enterprise Manager. The Enterprise Manager then processes and stores the data for real-time and historical reporting. You can then view and work with the collected data using the Introscope Workstation to create alerts or take responsive action.

Planning a Java Agent Deployment


For application management, the key activities are:

■ Deploy agents to monitor the performance and availability of application servers.

■ Test, adjust, and optimize monitoring of application components.

■ Customize the agent profile to control agent operations as needed.

■ Create application- or agent-specific metric groupings, dashboards, alerts, and actions

■ Investigate, triage, and diagnose application issues.


When planning a deployment, your goal is to develop the right balance between agent overhead and the visibility of application performance. Low overhead agents allow for real-time monitoring of all transactions in production environments. Keeping overhead low helps performance and availability of critical applications and server resources. However, keeping overhead low does not provide enough information to diagnose problems when they occur. Therefore, it is common to deploy and adjust the agent configuration throughout the lifecycle of an application. In addition, monitoring more components when an application is being developed or tested, then reducing the components after an application is released to production.

Install and Evaluate the Default Functionality

The first step in deploying agents involves installing and evaluating the default agent configuration. The default agent configuration demonstrates data collection for many common components of the application and the computing environment. The default agent configuration also includes several features enabled and other, less frequently used features, disabled.

Your goal is to evaluate the depth and breadth of data collection provided by default and become familiar with Introscope and how to monitor applications. After you are familiar with the performance metrics the agent provides by default, you can customize the agent to collect data, as needed.

As you evaluate the default performance, keep in mind that when the agent collects more metrics, it consumes more system resources. If the agent collects fewer metrics, you have less visibility into potential problems. As you refine the agent configuration, try to strike a balance between the depth of data collection and an acceptable level of performance. The appropriate level of instrumentation typically depends on where the agent is deployed. For example, an agent monitoring within a test environment is typically configured to collect large numbers of metrics. An agent on a production server, however, is typically configured to deliver essential information.



Determine Configuration Requirements

Before you deploy the agent, determine your data collection requirements. This information lets you tailor the data collection behaviors of the agent, and evaluate the impact on overhead through alternative configurations of the agent.

Introscope can be used throughout the lifecycle of an application. For example, from development through testing, load verification, staging, and into production. At each stage in the lifecycle, the monitoring goals, environmental constraints, and service level expectations are likely to be different. To address these differences, you configure the agent to behave differently for the type of environment that is monitored.

Your goal is to determine the appropriate trade-off between the visibility of performance details and resource overhead. Also consider the optimum level of visibility at a reasonable cost for the environment being monitored.

In preproduction application environments, such as development, you typically configure a higher level of data collection to provide deeper visibility into the application performance. In production or high-volume transaction environments, you typically reduce the metrics reported to control agent overhead. Depending on your requirements, you can also configure agent properties to control specific agent behavior. For example, track the maximum number of metrics collected and the removal of old metrics.

For your environment, determine the appropriate level of visibility and acceptable performance overhead, so you can configure the agent to match your requirements.

Define a Baseline Agent Profile with Appropriate Configuration Properties

After you identify the type of application environment to monitor, you can create a "candidate" agent configuration. Most agent operations are controlled using properties in the agent profile (IntroscopeAgent.profile). For example, the IntroscopeAgent.profile file defines ProbeBuilder Directive and ProbeBuilder List files the agent uses. The files listed in the agent profile then control the specific metrics that the agent gathers. The IntroscopeAgent.profile file also provides properties that enable or disable specific features or tune operations such as polling intervals.

Depending on your configuration and environment, you can adjust the properties in the agent profile to evaluate the impact of each change. For example, you can start with a default agent profile that does not have ChangeDetector enabled. Later, you can enable ChangeDetector properties in the profile and evaluate agent performance after the change before adding any other features.

Deploying the Java Agent


Evaluate Agent Performance Overhead

When evaluating an agent configuration, verify that the metrics collected provide sufficient visibility into application performance and availability. In addition, verify that the volume of metrics cannot impose an unacceptable load on the operating environment. The agent cannot report more metrics than are necessary to identify and localize performance and availability problems.

You can effectively evaluate agent performance by understanding the performance characteristics of the application. For example, you can load test your application before and after implementing default monitoring to verify impact.

To extend data collection in a controlled way, verify agent operation and overhead before and after implementing changes. For example, only add monitoring support for one application at a time, so you can evaluate the performance of each add-on individually.

Validate and Deploy the Agent Configuration

After verifying that the candidate agent configuration provides the visibility required, deploy the configuration across that environment.

You can deploy a validated configuration by installing the following configuration items to the target environment:

■ IntroscopeAgent.profile file

■ modified or custom PBD files

Deploying the Java Agent

Deploying the agent involves the following high level steps:

1. Install the agent on the target computer.

2. Configure the application server (see page 39) to instrument your applications.

3. Configure the startup script (see page 40) or Java command for the application server to include the agent and the location of the agent profile.

4. Configure the agent connection (see page 57) to the Enterprise Manager.

5. Restart the application server to instrument the application and start data collection.

6. Configure the IntroscopeAgent.profile (see page 65) if you want to change any properties set during installation or want to modify data collection or other agent operations.


Chapter 2: Installing and Configuring the Java Agent


View the Product Compatibility Guide (see page 29) Before You Install the Agent (see page 30) Select the method for installing the Java agent (see page 30) About the Java Agent Directory Structure (see page 38) How to instrument applications (see page 39) Configuring the Application Server to Start the Java Agent (see page 40) Configuring the connection to the Enterprise Manager (see page 57) Upgrading multiple agent types (see page 61) Uninstalling the Java agent (see page 62)

View the Product Compatibility Guide

The product Compatibility Guide provides a list of all supported operating environments.

Follow these steps:

1. Open a browser and go to http://www.ca.com/support.

The CA Support Online page appears.

2. Log in to CA Support Online.

3. Select CA Application Performance Management from the Support By Product, Select a Product page drop-down list.

The CA Application Performance Management page appears.

4. Scroll to the Product Status section and click the Application Performance Management Compatibility Guide link.

The Application Performance Management Compatibility Guide page appears.

5. Click the link that corresponds to the release you want.

The Compatibility Matrix appears.



Before You Install the Agent


Before You Install the Agent

Before you install the agent, do the following steps:

1. Review the Java agent deployment process.

2. Verify that you have a supported version of the application server.

Note: The server where you plan to install the agent must have a supported version of the JVM available locally. For application server and JVM requirements, see the CA APM Compatibility Guide.

3. Verify that you have a supported version of the JVM. If you are unable to use a supported version, use a previous version:

■ Use the 8.x Java agent for Java 1.4.x.

■ Use the 7.2 Java agent for Java 1.3.x.

4. Verify that the Introscope Enterprise Manager and Workstation components are installed. Note the host name and port number for the Enterprise Manager connection with the agent.

You can use ping or telnet to verify connectivity between the agent and the Enterprise Manager.

Note: For information about installing the Enterprise Manager, Workstation, and WebView components, see the CA APM Installation and Upgrade Guide.

More information:

View the Product Compatibility Guide (see page 29) Planning a Java Agent Deployment (see page 26) Deploying the Java Agent (see page 28)

Select the method for installing the Java agent

You can install the Java agent in one of the following ways:

■ interactively using a graphical user interface (GUI) or text-based console installer

■ silently without interactive prompts using an edited response file

■ manually by extracting and configuring files for individual application servers

In most cases, you use the interactive installer when you want to install files locally on a computer. If you install interactively, the prompts displayed are the same for the graphical or text-based interface. However, the option to choose the GUI or text-based installer depends on the operating system of the computer where you run the installer. For example, most UNIX environments support the GUI or text-based console installer, but start the console installer by default.



If you want to install files remotely on a computer or deploy a preconfigured set of installation instructions, you can run the installer silently using an edited response file.

If you do not want to run the installer interactively or silently, you can manually install and configure the agent using an installation archive. Manual installation enables you to extract specific sets of agent files from application server- and operating system-specific archives, and then manually configure deployment options. CA Technologies provides these archives to expedite deployment of the Java agent onto multiple systems that share the same type of application server and operating system.

For more information about installing interactively, silently, or manually see the appropriate section.

Install the Java Agent Interactively

You can install the Java agent interactively by selecting the Java agent installer appropriate for your operating system, starting the installer, and responding to the prompts. If you use the GUI installer, you can make selections using drop-down menus and clicking checkboxes. In the text-based console installer, you make selections by entering text.

Follow these steps:

1. Select the installation archive appropriate for your operating system. For example:

■ IntroscopeAgentInstaller<version>windows.zip to install on Microsoft Windows.

■ IntroscopeAgentInstaller<version>unix.tar to install on supported UNIX or Linux operating systems.

■ IntroscopeAgentInstaller<version>zOS.tar to install on IBM z/OS.

■ IntroscopeAgentInstaller<version>os400.zip to install on IBM OS/400 (IBM i operating system).

2. Extract the installation archive files using a command appropriate to your operating system. For example, on UNIX or Linux:

tar –xvf IntroscopeAgentInstaller<version>unix.tar

3. Follow the prompts displayed to start the installation.

4. Specify the location of the application server root directory.

If you do not want to specify an application server root directory, then accept the default response (C:\ on Windows, / on UNIX).



5. Select the application server type from the list of valid application servers:

■ Default

■ JBoss

■ Tomcat

■ WebLogic

■ WebSphere

■ Sun ONE

■ Oracle Application Server

■ Interstage

The application server you select determines the additional monitoring options available.

6. Specify the root installation directory for the Java agent. In most cases, use the application server root directory.

Within the root installation directory, the installer creates the “wily” directory that is the <Agent_Home> directory.

7. Specify whether you want to create an agent profile or use an existing agent profile.

If you create an agent profile, you are prompted to select:

■ Typical or Full instrumentation

■ Agent name and process name

■ Enterprise Manager host and port number

If you select an existing profile, you are prompted for the location of the file. Specify the fully qualified path to the file.

The settings you specify can be changed after installation if needed, by editing the IntroscopeAgent.profile file.

8. Specify the path to the Java executable that is used to launch your application server.

9. Specify the ProbeBuilding method for instrumenting your application. In most cases, use JVM AutoProbe.



10. Specify whether to enable the ChangeDetector agent extension.

If you enable ChangeDetector, you are prompted to name the ChangeDetector agent.

If you do not to enable ChangeDetector, the ChangeDetector files are installed in the <Agent_Home>/examples directory. You can enable ChangeDetector later by copying the files to the <Agent_Home>/core/ext directory and modifying the IntroscopeAgent.profile file.

Note: For more information about ChangeDetector, see the CA APM ChangeDetector Guide.

11. Select additional monitoring options to install. For example, select CA APM for SOA if you want to monitor web services and other SOA environment components.

If you do not enable other monitoring options during installation, the files are installed in the <Agent_Home>/examples directory and can be enabled later.

12. Specify the location of a “pickup folder” for .zip or .tar files that contain add-ons you want installed with the agent.

Any .zip or .tar files found in the specified location are extracted into the <Agent_Home> directory.

13. Review your selections and continue to install the agent, and then complete the installation.

Note: Installing the agent does not stop or start the application server, or configure the application server startup script. These tasks must be performed manually after installation. The specific steps depend on the type of application server you are monitoring.

Install the Java Agent Silently

In silent mode, you invoke the agent installer from a command line and specify a response file that contains installation instructions. The installation then runs in the background without displaying any information about its progress. Because this installation method enables you to install the agent without user interaction, it is most commonly used to install agents on remote computers or to install multiple agents with the same configuration.

If you have previously installed a Java agent, you can use the automatically-generated response file to install additional agents. Alternatively, you can manually edit the sample response file provided in the installation or create your own response file using a text editor.



About Automatically Generated Response Files

Any time you run the Java agent installer (interactively or silently), the installer creates a response file that records the installation options you selected. This automatically generated response file is saved in the <Agent_Home>/install directory. The file name indicates the date and time that the installer created the response file in the following format:

autogenerated.responsefile.<year>.<month>.<day>.<hour>.<minutes>.<seconds>

For example, an installation done April 30, 2005 at 7:10:05 a.m. would have an automatically generated response file with this name:

autogenerated.responsefile.2005.4.30.7.10.05

You can use this automatically generated response file for subsequent silent installations with the same settings or edit it to use new settings.

About the sample response file

If you have not previously installed the Java agent or want to use default instead of previously selected installation options, you can edit the default sample response file included with the Java agent installer. The default sample response file is located in the <Agent_Home>\install directory and named:

SampleResponseFile.Agent.txt

The sample response file provides default settings for most properties, but you must manually edit the file before you can use it for silent installation.



Configure the response file properties and install the agent

Whether you use the automatically-generated response file, the default sample response file, or create a custom response file, you must configure the properties in the file appropriately before invoking the Java agent installer in silent mode. The properties you set in the response file are equivalent to the selections you make when running the installer interactively.

Note: For additional details about setting any property, see the comments in the <Agent_Home>/install/SampleResponse.Agent.txt file.

Follow these steps:

1. Open the response file in a text editor.

2. Set the appropriate property values. The properties to configure are:

USER_INSTALL_DIR=<root_installation_directory>

Specifies the directory for installing the agent. In most cases, you should use the application server root directory.

silentInstallChosenFeatures=Agent

Specifies the components you want to install. Add Docs to this property is you want to install the agent files and documentation. For example:

silentInstallChosenFeatures=Agent,Docs

appServer=Default

Specifies the type of application server to monitor. Valid values are Default, JBoss, Tomcat, WebLogic, WebSphere, Sun, Oracle, or Interstage. The value is case-sensitive.

This setting controls the ProbeBuilder Directives (PBDs) that are installed with the agent and which additional monitoring options are valid.

(Optional) appServerHome=

Specifies the home directory of the application server. This property is not needed if you have set the USER_INSTALL_DIR property to the application server root directory.

(Optional) appServerJavaExecutable=

Specifies the path to the Java executable that is used to launch the application server.

instrumentationLevel=Typical

Specifies whether you want to use Full or Typical instrumentation. The value is case-sensitive.

agentName=Default Agent

Specifies the agent name that should be displayed in the Workstation.



processName=Default Process

Specifies the process name that should be displayed in the Workstation.

emHost=localhost

Specifies the host name of the computer running the Enterprise Manager that the agent should connect to by default.

emPort=5001

Specifies the port number the agent should use to connect to its Enterprise Manager.

(Optional) alternateAgentProfile=

Specifies the absolute path to an existing agent profile.

(Optional) pickupFolder=

Specifies the absolute path to a "pickup folder" that contains .zip or .tar files for any add-ons you want to install with the agent.

(Optional) changeDetectorEnable=false

Specifies whether to enable the ChangeDetector agent extension. If you set this property to false, ChangeDetector files are installed but not enabled. You may enable it later.

(Optional) changeDetectorAgentID=

Specifies a name for the ChangeDetector agent extension. If you enable ChangeDetector, uncomment and set this property.

(Optional) shouldEnable*

Specifies which additional CA APM monitoring solutions you want to enable. All of the properties for optional CA APM monitoring solutions are set to false by default. The options that are valid to enable depend on the application server type.

3. Save the response file and close the text editor.



4. Invoke the installer in silent mode by specifying the path to the installer executable and the absolute path to the response file.

<path_to_installer> -f <absolute_path_to_response-file>

The command you use to invoke the installer depends on the operating system where you are running the command.

For example, on Windows, the command format looks like this:

IntroscopeAgent<version>windows.exe -f C:\temp\myResponseFile.txt

On Linux or UNIX, the command format looks like this:

./IntroscopeAgent<version>unix.bin -f /

Select the appropriate command format for your operating system.

5. Check the <Agent_Home>/install/Introscope_Agent_<version>_InstallLog.log file to verify the agent has been installed successfully.

Install Manually Using Installation Archives

You can put the agent files on a system without running the Java agent installer interactively or configuring a response file. Application server-specific archives let you install the agent.

The installation archives contain all of the files that would have been installed had you run the agent installer. After you copy an archive to a computer, you extract the contents and configure the agent profile to identify the Enterprise Manager to connect to and other properties. Use these files to deploy multiple agents in a batch job, or keep the files as an archive of the default set of agent files.

To install the Java agent manually from an archive, verify that the computer where you plan to install has 35 MB of free disk space.

Follow these steps:

1. Select the appropriate installation archive for your application server and operating system.

2. Extract the contents of the archive into a location your JVM can access using a command appropriate to your operating system. For example, on UNIX or Linux:

tar -xvf IntroscopeAgentFilesOnly-NoInstaller<version><app_server>.<os>.tar

3. Open the <Agent_Home>/core/config/IntroscopeAgent.profile file in a text editor and configure the connection to the Enterprise Manager.

For more information about setting the properties for communication with the Enterprise Manager, see Configuring the connection to the Enterprise Manager (see page 57).

About the Java Agent Directory Structure


4. Configure any additional properties, then save and close the IntroscopeAgent.profile file.

5. Configure the application server with the location of the Java agent startup file and the agent profile.

6. Restart the application server.

About the Java Agent Directory Structure

When you install the agent, the following directory structure is created in the root installation directory:

wily

This directory is the <Agent_Home> directory, which contains Agent.jar that is used to start the agent.

Within the <Agent_Home> directory, additional subdirectories provide the libraries and extension files that enable various features of the Java agent.

core

■ config

Contains the IntroscopeAgent.profile, ProbeBuilder Directive files (.pbd), and ProbeBuilder List files (.pbl) files that control agent operations, metric data collection, and the instrumentation process. The specific properties defined in the IntroscopeAgent.profile and the .pbd and .pbl files that are deployed and referenced in the agent profile depend on choices you made during installation.

Within the config directory, the hotdeploy subdirectory enables you to deploy new directives without editing the IntroscopeAgent.profile or restarting applications. If files placed in the hotdeploy directory contain invalid syntax or collect too many metrics, instrumentation can fail or application performance can suffer.

■ ext

Contains the files for enabled agent extensions or features. For example, the directory contains files for the application triage map, and LeakHunter.

connectors

Contains the CreateAutoProbeConnector.jar utility that is used to configure an AutoProbe Connector to enable instrumentation in certain environments.

common

Contains the configuration and property files for the extensions.

How to instrument applications


examples

Contains folders and files for optional agent extensions, such as CA APM for SOA. If you did not enable an extension at installation, you can use the files in this directory to configure the extension at a later time.

install

Contains log files that record the installation process and the files you can use for silent installation. For example, the generated response file and the SampleResponseFile.Agent.txt file are located in this directory.

This directory is not created if you manually install the Java agent from an installation archive.

logs

Stores agent log files.

tools

Contains the URLGrouper.jar command line utility that analyzes web server log files.

This directory also includes the list of the files of extensions. For example, configurePMI.bat, CreateIU.jar, listServers.bat, MergeUtility.jar, NetInterface.jar, and setPmiModules.jacl files are located in this directory.

UninstallerData

Contains a subdirectory with the executables and associated resources used to uninstall the agent and related resources.

This directory is not created if you manually install the Java agent from an installation archive.

version

Contains version information for optional CA APM monitoring solutions. This information is installed regardless of what you enable at installation time.

How to instrument applications

After you have installed the agent, you must configure the application server to instrument your applications. The most common method for instrumenting applications is to use JVM AutoProbe and the –javaagent command line option. JVM AutoProbe instruments applications dynamically during runtime. It is suitable for all J2EE application servers that provide a hook in the bootstrap or application server class loader for Introscope to see all bytecode loaded from the file system.

Most JVM providers support the –javaagent option. If you are using a JVM that does not provide support for this option you must use an alternate method for instrumentation.

Configuring the Application Server to Start the Java Agent


Running ProbeBuilder manually is only required if you must instrument bytecode statically before an application is started. You can run ProbeBuilder manually using the ProbeBuilder wizard or from a command-line prompt. It instruments bytecode and outputs a newly named, instrumented jar or class file. This newly instrumented bytecode is then placed ahead on the classpath (or renamed in place) for the application prior to its being started.

For more information about the alternatives to using JVM AutoProbe, see Appendix B: Alternative Methods for Instrumentation.


You must configure the application server you are instrumenting to include the path to the agent’s primary .jar file and the agent profile. In most cases, you do this by editing the application server's startup script, then restarting the application server. When the application server restarts, the Java agent instruments the classes discovered for default components of the JVM and application environment. The specific steps involved depend on the application server.

Configure Apache Tomcat to use the Java agent

To configure Apache Tomcat to use the Java agent, you must edit the Tomcat startup script. By default, the Tomcat startup script is catalina.sh or catalina.bat in the $CATALINA_HOME/bin directory. Depending on the requirements of your web server, you may use a different startup script or a different location for the startup script.

Follow these steps:

1. Navigate to the directory that has the Tomcat startup script. For example:

cd /apache-tomcat-6.0.18/bin

2. Open the Tomcat startup script in a text editor. For example:

vi catalina.sh

3. Locate the command line for setting Java options and add the following command-line options to specify the path to the agent's startup .jar file and the agent profile:

-javaagent:<PathToAgentJar>

-Dcom.wily.introscope.agentProfile=<PathToAgentProfile>

For example, if using Agent.jar, add code similar to the following before the command that starts the server:

set JAVA_OPTS=%JAVA_OPTS% -javaagent:c:\apache-root\wily\Agent.jar

-Dcom.wily.introscope.agentProfile=

c:\apache-root\wily\core\config\IntroscopeAgent.profile

4. Save the startup script.

5. Restart the Tomcat server.



Tomcat PBD tracing options

After you install the Java agent on an Apache Tomcat server, the following PBDs are available to customize data collection:

■ tomcat41x.pbd

■ tomcat50x.pbd

■ tomcat55x.pbd

Configure JBoss to use the Java agent

To configure JBoss to use the Java agent, you must edit the JBoss startup script. By default, the JBoss startup script is run.sh or run.bat in the $JBOSS_HOME/bin directory. Depending on the requirements of the web server, you may use a different startup script or a different location for the startup script.

To modify the JBoss server to include the agent

1. Navigate to the directory that has the JBoss startup script. For example:

cd /jboss-4.2.3.GA/bin

2. Open the JBoss startup script in a text editor. For example:

vi run.sh

3. Locate the command line for setting Java options and add the following command-line options to specify the path to the agent's startup .jar file and the path to the agent profile:



4. For example, if using Agent.jar, add code similar to the following before the command that starts the server:

set JAVA_OPTS= -javaagent:%JBOSS_HOME%\wily\Agent.jar

-Dcom.wily.introscope.agentProfile=%JBOSS_HOME%\wily\core\config\IntroscopeAg

ent.profile %JAVA_OPTS%

5. Save the run.bat file.

6. (Optional) If you want to enable the reporting of JBoss JMX metrics, you must configure the agent profile to collect JMX metrics by opening the IntroscopeAgent.profile and setting the following property:

introscope.agent.jmx.enable=true

Note: If you want to see JMX metrics from JBoss in a JConsole by using the JMX remote management server with a platform-specific MBeanServer, you should add com.wily.use.platform.mbeanserver=true to the IntroscopeAgent.profile. This reflects a change from previous versions of Introscope where the use of the platform-specific MBeanServer was set in the command line.

7. Save the IntroscopeAgent.profile.



JBoss-specific PBDs and PBLs

When you install the Java Agent on a JBoss application server, the following JBoss-specific PBDs and PBLs are available for you to customize data collection:

■ jboss4x.pbd

■ jsf.pbd

■ jsf-toggles-full.pbd

■ jsf-toggles-typical.pbd

■ jboss-full.pbl

■ jboss-typical.pbl

Configure Oracle WebLogic to use the Java agent

To configure Oracle WebLogic to use the Java agent, you must edit the WebLogic startup script. Depending on your requirements, the startup script you use can be specific to a WebLogic domain. By default, the WebLogic startup script is startWebLogic.sh or startWebLogic.cmd in the $WEBLOGIC_HOME/samples/domain/<domain_name>/bin directory. Depending on the requirements of the application server, you may use a different startup script, such as an application-specific startup script or a different location for the startup script.

To modify the WebLogic 9.x or 10.x server to include the agent

1. Navigate to the directory that has the WebLogic startup script you want to modify. For example:

cd $WL_HOME/samples/domain/wl_server

2. Open the WebLogic startup script in a text editor. For example:

vi startWebLogic.sh

3. Locate the command line for setting Java options or the Java command line and add the following command-line options. For example: -javaagent:<PathToAgentJar> -Dcom.wily.introscope.agentProfile=<PathToAgentProfile>

4. Save and close the WebLogic startup script or application-specific startup script.

Create a startup class for WebLogic

Creating a WebLogic startup class for an application server or cluster enables the Java agent to collect additional information from the application server. If you configure a startup class, the Java agent can automatically determine its name. The startup class also enables the Java agent to report JMX metrics that are used to determine application health in the Workstation.



Follow these steps:

1. Open the WebLogic Administrative Console.

2. In the left pane, expand the Environments folder.

3. Click the Startup & Shutdown folder.

The Startup and Shutdown page opens.

4. Click Configure a New Startup Class.

The Configuration tab is shown.

5. In the Name field, enter:

Introscope Startup Class

6. In the ClassName field, enter:

com.wily.introscope.api.weblogic.IntroscopeStartupClass

7. Click Create.

The Target and Deploy tab appears.

8. Select the boxes for the servers you want to make this startup class available to.

9. Click Apply and select the Run before application deployments option.

10. Add the location of the WebAppSupport.jar to the <server or application server> startup classpath.


Enable cross-process tracing in WebLogic Server

Transaction trace sessions enable you to trace all of the operations that take place in a transaction, including transactions that cross JVM boundaries on computers with compatible JVM versions.

Cross-process transaction tracing is supported for synchronous transactions, for instance servlets to EJBs, and asynchronous transactions.

To enable cross-process transaction tracing for WebLogic

1. Create a startup class as described in Creating a startup class for WebLogic (see page 42).

2. Add "-Dweblogic.TracingEnabled=true" to the java command line for starting WebLogic Server.

3. Open the IntroscopeAgent.profile in a text editor.

4. Locate and set introscope.agent.weblogic.crossjvm property to true. For example:

introscope.agent.weblogic.crossjvm=true

5. Save and close the IntroscopeAgent.profile.



About Java agent support for JMX metrics

Introscope can collect management data that application servers or Java applications expose as JMX-compliant MBeans, and present the JMX data in the Investigator metric tree. In older versions, WebLogic provided only a single MBeanServer as a source of JMX metrics. WebLogic 9.x and later provide three:

■ RuntimeServiceMBean: per-server runtime metrics, including active effective configuration

■ DomainRuntimeServiceMBean: domain-wide runtime metrics

■ EditServiceMBean: allows user to edit persistent configuration

Introscope polls only RuntimeServiceMBean for metrics because it supports local access (an efficiency issue), and because it contains most of the data expected to be relevant. However, Introscope can support any MBean built to the Sun JMX specification. For more information on the Sun JMX specification, see http://java.sun.com/products/JavaManagement/.

To support the collection of JMX metrics, the following keywords are defined and enabled by default in the IntroscopeAgent.profile file for WebLogic:

■ ActiveConnectionsCurrentCount

■ WaitingForConnectionCurrentCount

■ PendingRequestCurrentCount

■ ExecuteThreadCurrentIdleCount

■ OpenSessionsCurrentCount

For more information see Enabling JMX Reporting (see page 179).

Introscope displays the JMX metrics it collects in the Investigator tree under the following node:

<Domain>|<Host>|<Process>|AgentName|JMX|

http://java.sun.com/products/JavaManagement/



How to Set Up Resource Metrics in Weblogic

Various agents can report resource metric categories in the CA Introscope Workstation. You can configure an Oracle WebLogic server so that a Java agent can report resource metrics.

To set up resource metrics for Oracle WebLogic, follow these steps:

1. Install CA APM for Oracle WebLogic using the instructions in the CA APM for Oracle WebLogic Server Guide.

2. Enable the CA APM for Oracle Databases option using the CA APM for Oracle Databases Guide.

3. Follow the instructions for configuring resource metrics in the CA APM Configuration and Administration Guide.

About WebLogic Diagnostic Framework (WLDF)

The WebLogic Diagnostic Framework (WLDF) is a monitoring and diagnostic framework that defines and implements a set of services that run within the WebLogic Server process and participate in the standard server life cycle. Using WLDF, you can create, collect, analyze, archive and access diagnostic data generated by a running server and the applications deployed within its containers. This data provides insight into the run-time performance of servers and applications and enables you to isolate and diagnose faults when they occur.

WLDF enables dynamic access to server data through standard interfaces and the volume of data accessed at any given time can be modified without shutting down and restarting the server.

How WLDF data is converted into Introscope metrics

In WLDF, information is organized into a set of Data Accessors, each with multiple Columns. Introscope converts this WLDF information into an Introscope-specific metric format and displays it in the Investigator under the following node:

<Domain>|<Host>|<Process>|AgentName|WLDF|

Information in the Data Accessors is defined by a domain name and one or more key/value pairs. Introscope converts Data Accessor Columns into key and value information displayed in alphabetical order in the Introscope-generated metrics. The following example shows the syntax used:

<Domain>|<Host>|<Process>|AgentName|WLDF|<domain

name>|<key1>=<value1>|<key2>=<value2>:<metric>



For example, this table shows the information for the BYTECOUNT Column from the HTTPAccessLog Data Accessor:

Domain name Key/Value pairs Metric names

WebLogic Name=HTTPAccessLog, Type=WLDFDataAccessRuntime=Accessor,

WLDFRuntime=WLDFRuntime

BYTECOUNT

The Data Accessor information in the table above would be converted to the following Introscope metric:

<Domain>|<Host>|<Process>|AgentName|WLDF|Weblogic|Name=HTTPAccessLog

|Type=WLDFDataAccessRuntime=Accessor|WLDFRuntime=WLDFRuntime:BYTECOUNT

Note that the key/value pairs are displayed alphabetically in the Introscope metric.

Enable WLDF reporting

By default, WLDF reporting is not enabled in Introscope. You can modify the agent profile to enable reporting of WLDF metrics.

To enable WLDF reporting

1. Shut down the managed application, if it is running.

2. Configure a WebLogic startup class.


4. Locate and set the following property:

introscope.agent.wldf.enable=true


Configure WebLogic with JRockit JVM to Use the Java Agent

If you are using WebLogic 9.0 or later with JRockit JVM 5.0 or later, use the following command-line options to start the JVM:

JAVA_VENDOR=Bea

JAVA_OPTIONS=%JAVA_OPTIONS% -javaagent:PathToAgentJar

-Dcom.wily.introscope.agentProfile=PathToIntroscopeAgent.profile

If you are using an older version of WebLogic or JRockit JVM, create and run an AutoProbe connector file (see page 319) to instrument applications.



Configure WebLogic with JRocket JVM to View Socket Metrics

Due to a third-party compatibility issue, running WebLogic 9.0 with JRockit JVM 1.5.0 may experience a problem with viewing metrics for socket clients. To address this problem, use the following option to turn on metrics for socket clients.

Follow these steps:

1. Open the file toggles_typical.pbl or toggles_full.pbl in a text editor.

2. Turn on the Managed Socket option to trace socket metrics. For example:

#######################

# Network Configuration

# ================

#TurnOn: SocketTracing

# NOTE: Only one of SocketTracing and ManagedSocketTracing should be 'on'.

ManagedSocketTracing is provided to

# enable pre 9.0 socket tracing.

TurnOn: ManagedSocketTracing

TurnOn: UDPTracing

3. Save your changes.

The Managed Socket option is turned on.

Configure IBM WebSphere to use the Java agent

To configure IBM WebSphere to use the Java agent, you must edit the WebSphere startup script. Depending on your requirements, the startup script you use can be specific to an application server node. By default, the WebSphere startup script is the server.xml file in the $WEBSPHERE_HOME/profiles/<App Server Name/config/cells/<Cell Name>/nodes/<Node name>/servers/server1directory. Depending on the requirements of the application server, you may use a different startup script, such as an application-specific startup script or a different location for the startup script. In addition, different combinations of WebSphere on different operating systems or using different JVM vendors or JVM versions can have special requirements. For more information, see the section that most closely describes you WebSphere Application Server environment.

WebSphere Application Server 6.1 - UNIX, Windows, OS/400, z/OS - IBM JVM 1.5

The CA Introscope dynamic instrumentation feature requires class redefinition support. Use of class redefinition can significantly impact performance when running on IBM JDK version 5. IBM has provided technical information about this performance overhead in the Java Diagnostics Guide.



CA Introscope and IBM JDK version 5 customers that would like to take advantage of dynamic instrumentation should keep this performance overhead in mind. When employing this configuration, CA Technologies recommends using the dynamic instrumentation feature only in a QA environment.

For more information about dynamic instrumentation, see Dynamic ProbeBuilding vs. dynamic instrumentation (see page 75).

When you are running WebSphere 6.1 using an IBM JVM 1.5, use the alternate versions of the Java agent .jar file and Java agent profile. These files, named AgentNoRedef.jar and IntroscopeAgent.NoRedef.profile, are located in the <Agent_Home>/core/config directory.

Note: If you are using the AllAppServer agent distribution, the alternate profile is named IntroscopeAgent.websphere.NoRedef.profile.

As a result of using the previous files and syntax, metrics are no longer reported for the following:

■ System classes

■ NIO (Sockets and Datagrams)

■ SSL

The following features are also affected:

■ Socket instrumentation reverts to pre-Introscope 9.0 ManagedSocket style.

■ Remote dynamic instrumentation is disabled

■ Changes to PBD files require that you restart the instrumented JVM before being applied

■ Deep inheritance and hierarchy support instrumentation is disabled.

When using the previous files and syntax, the agent reports class redefinition status by the following actions:

■ Adding a metric named Agent Class Redefinition Enabled under the agent node in the Investigator. Metric values are true or false.

■ Writing a log message to agent log file.

■ When class redefinition is enabled, the log message is written at the WARN level and reads:

Introscope Agent Class Redefinition is enabled. Enabling class redefinition

on IBM 1.5 JVMs is known to incur significant overhead.

■ When class redefinition is disabled, the log message is written at INFO level and reads:

Introscope Agent Class Redefinition is disabled.



■ Writing a message to standard error (only when class redefinition is enabled), which reads:

Warning: Introscope agent has been configured to support class redefinition.

IBM JVMs version 1.5 and higher are known to incur significant overhead with

redefinition enabled. To avoid this overhead please use AgentNoRedef.jar

instead of Agent.jar.

If you use a non-IBM JVM or an IBM JVM that is a version other than 1.5, the previous metric and messages are not output.

To modify the WebSphere Application Server 6.1 - UNIX, Windows, OS/400, z/OS - IBM JVM 1.5 to use the agent

1. Open the WebSphere Administrator Console.

2. Navigate to Application Servers > your server > Server Infrastructure > Java and Process Management > Process Definition > Java Virtual Machine.

3. Set the Generic JVM Argument field as follows:

-javaagent:<Agent_Home>/AgentNoRedef.jar

-Dcom.wily.introscope.agentProfile=<Agent_Home>/core/config/IntroscopeAgent.N

oRedef.profile

If you have both instrumented and non-instrumented applications on the same computer, include the -Xshareclasses:none setting in the Generic JVM Argument to avoid errors on AIX.

Note: A unique directory is required when there is more than one version of WebSphere using the same agent directory.

WebSphere Application Server 6.1 - UNIX, Windows - Sun, HP, Other JVM 1.5

If you use a non-IBM JVM or an IBM JVM that is a version other than 1.5, not all metrics and messages are output.

The following examples indicate which Java argument and .jar file to use with a specific JVM when installing the Java Agent on WebSphere 6.1.

To modify the WebSphere Application Server 6.1 - UNIX, Windows - Sun, HP, all other JVM 1.5 to use the agent




-javaagent:<Agent_Home>/Agent.jar

-Dcom.wily.introscope.agentProfile=<Agent_Home>/core/config/IntroscopeAgent.p

rofile

4. Restart the WebSphere application server.



WebSphere Application Server 7.0 - UNIX, Windows, OS/400 - JVM 1.5 or later

If you are monitoring WebSphere 7.0, verify you have build 7.0.0.8 or later installed before configuring the server to start the Java agent.

To modify WebSphere Application Server 7.0 to use the agent




-javaagent:<Agent_Home>/Agent.jar


rofile


WebSphere Application Server 7.0 - z/OS - JVM 1.5 or later

If you are monitoring WebSphere 7.0, verify you have build 7.0.0.8 or later installed before configuring the server to start the Java agent.

To modify WebSphere Application Server 7.0 to use the agent




-Xbootclasspath/a:<Agent_Home>/Agent.jar -javaagent:<Agent_Home>/Agent.jar


rofile




Modifying Java2 Security Policy

For AutoProbe to run correctly in WebSphere environments with Java2 Security enabled, it may be necessary to add permissions to your Java2 Security Policy.

To add permissions to your Java2 Security Policy

1. Open the file $WebSphere home/properties/server.policy in a text editor.

2. Add the following permissions to the file:

// permissions for Introscope AutoProbe

grant codeBase "file:${was.install.root}/-" {

permission java.io.FilePermission "${was.install.root}${/

}wily${/}-", "read";

permission java.net.SocketPermission "*", "connect,resolve";

permission java.lang.RuntimePermission "setIO";

permission java.lang.RuntimePermission "getClassLoader";

permission java.lang.RuntimePermission "modifyThread";

permission java.lang.RuntimePermission "modifyThreadGroup";

permission java.lang.RuntimePermission "loadLibrary.*";

permission java.lang.RuntimePermission "accessClassInPackage.*";

permission java.lang.RuntimePermission "accessDeclaredMembers";

};

grant {

permission java.util.PropertyPermission "*", "read,write";

};

Note: Line breaks are shown for user readability and are not needed when adding the permissions to the server.policy file.

3. Save and close the file.

Configuring a custom service in WebSphere

Creating a custom service in the WebSphere Application Server enables the Java agent to collect additional information from the application server. If you configure a custom service, the Java agent can automatically determine its name. The custom service also enables the Java agent to report JMX and Performance Monitoring Infrastructure (PMI) metrics that are used to determine application health displayed in the Introscope Workstation on the Application Overview tab.

To configure a custom service in WebSphere 6.1


2. Select the server you'd like to configure, then navigate to Server Infrastructure > Administration > Custom Services.

3. Click New to add a new Custom Service.

4. In the Classname field, enter:

com.wily.introscope.api.websphere.IntroscopeCustomService



5. In the Display Name field, enter:

Introscope Custom Service

6. In the Classpath field, enter:

<WebSphere_Home>/wily/WebAppSupport.jar

7. Click OK.


Enable the collection of WebSphere PMI metrics

Introscope can provide WebSphere performance data by extracting WebSphere Performance Monitoring Infrastructure (PMI) metrics via the PMI interface provided with WebSphere 6.1 and higher.

You must first enable PMI data collection in WebSphere before the data will be available to Introscope. In WebSphere, all performance monitoring settings are off by default.

To enable PMI reporting

1. Configure a custom service for Introscope in WebSphere.

2. Enable PMI data collection in WebSphere.

3. Enable the reporting of PMI data in the IntroscopeAgent.profile.

After you have enabled PMI data collection in WebSphere, the PMI data can be displayed as Introscope metrics. You can filter which metric categories to bring into Introscope, depending on your needs.

Note: For WebSphere on z/OS. CA Technologies recommends you us CA APM for WebSphere z/OS. CA APM for WebSphere z/OS provides WebSphere-specific PBDs and metrics that use low overhead tracer technology to retrieve WebSphere-specific metrics. CA APM for WebSphere z/OS does not require you to enable PMI in WebSphere for z/OS. You can also enable PMI reporting in WebSphere for z/OS, but this approach consumes more system resources.

Configure the reporting of WebSphere PMI metrics in Introscope

After you turn on Performance Monitoring Settings in WebSphere, you must enable PMI data collection in Introscope, and enable the metric categories you’d like to see reported.

To configure the reporting of PMI metrics in Introscope

1. Shut down your managed application.


3. Locate the property, introscope.agent.pmi.enable, under the WebSphere PMI Configurations section, and verify it is set to true.



4. Locate the following properties for high-level PMI metric categories:

introscope.agent.pmi.enable.threadPool=true

introscope.agent.pmi.enable.servletSessions=true

introscope.agent.pmi.enable.connectionPool=true

introscope.agent.pmi.enable.bean=false

introscope.agent.pmi.enable.transaction=false

introscope.agent.pmi.enable.webApp=false

introscope.agent.pmi.enable.jvmRuntime=false

introscope.agent.pmi.enable.system=false

introscope.agent.pmi.enable.cache=false

introscope.agent.pmi.enable.orbPerf=false

introscope.agent.pmi.enable.j2c=true

introscope.agent.pmi.enable.webServices=false

introscope.agent.pmi.enable.wlm=false

introscope.agent.pmi.enable.wsgw=false

introscope.agent.pmi.enable.alarmManager=false

introscope.agent.pmi.enable.hamanager=false

introscope.agent.pmi.enable.objectPool=false

introscope.agent.pmi.enable.scheduler=false

# introscope.agent.pmi.enable.jvmpi=false

Four categories—threadPool, servletSessions, connectionPool, and j2c—are set to true by default. You can enable additional metric categories by setting the appropriate properties to true or comment out categories to reduce the metrics reported.

5. Save the changes and close the file.

6. Restart the managed application.

After you’ve enabled PMI collections in Introscope, available PMI metrics are displayed under the WebSpherePMI node in the Investigator tree.

Configure the Reporting of Resource Metric Map Data for WebSphere

Various agents can report resource metric categories in the CA Introscope Workstation. After you enable PMI data collection in WebSphere, you can configure the application server to report the Resource Metric Map data.

Follow these steps:

1. Log in to the WebSphere console.

2. Select Monitoring and Tuning, Performance Monitoring Infrastructure.

3. Select the server you want to use for reporting, such as server1.

4. Click the Configuration tab, and select the Custom option.

5. Click the Custom link.

A configuration tree of options appears.



6. Select JDBC Connection Pools on the tree, and enable WaitingThreadCount on the right pane.

7. Select ThreadPools on the tree, and enable ActiveCount on the right pane.

8. Save the configuration and restart the application server.

The application server is configured to report resource metrics.

Note: For more information about the Resource Metric Map data, see the CA APM Configuration and Administration Guide.

Enable cross-process tracing in WebSphere

Transaction trace sessions enable you to trace all of the operations that take place in a transaction, including transactions that cross JVM boundaries on computers with compatible JVM versions. Cross-process transaction tracing is supported for synchronous transactions, for instance servlets to EJBs, and asynchronous transactions.

To enable cross-process transaction tracing in WebSphere

1. Create a custom service as described in Configuring a custom service in WebSphere 6.1 (see page 51).

2. Turn on the work area service.

From the administration page, Servers > Application servers, click on server1, click on Business Process Services, click on Work Area Service, check the "Enable service at server startup" box.


4. Locate and set introscope.agent.websphere.crossjvm property to true. For example:

introscope.agent.websphere.crossjvm=true


Logging considerations on WebSphere for z/OS

There are some things to consider when logging in a WebSphere z/OS environment. For more information about Java Agent logging options, see Java Agent Monitoring and Logging (see page 127).

Tagging log output as EBCDIC

WebSphere for z/OS changed its default encoding from EBCDIC CP1047 to ASCII ISO8859-1. Because z/OS is normally an EBCDIC machine, any logging data written by the Java Agent or AutoProbe must be tagged to use EBCDIC as the final output stream, instead of ASCII.



To tag data as EBCDIC instead of ASCII

1. Open the IntroscopeAgent.profile, located in the <Agent_Home>\wily\core\config directory.

2. Add these properties to the IntroscopeAgent.profile:

log4j.appender.console.encoding=IBM-1047

log4j.appender.logfile.encoding=IBM-1047


Eliminating startup timing issues with logging facilities

A new property has been added for WebSphere for z/OS 6.1 and later, which is used to eliminate any startup timing window exposures that can occur with the Introscope logging facilities.

To eliminate the timing window exposures

1. Open the IntroscopeAgent.profile, located in the <Agent_Home>\wily\core\config directory.

2. Add this property to the IntroscopeAgent.profile:

introscope.agent.logger.delay=100000

The value is in milliseconds, so the default delay in this example is 100 seconds.


Configure Oracle Application Server to use the Java agent

Oracle Application Server (OAS) uses one configuration file for the management of every application, and therefore every JVM, that is managed by the Oracle Console. This file is usually called opmn.xml.

To modify Oracle Application Server to use the Java agent

1. Shut down your application server and make a backup of opmn.xml.

2. Locate the section in the opmn.xml file for the application you want to instrument. You will most likely want to instrument more than one application at this time.

3. Under <category id="start-parameters"> for the selected application, locate the section called <data id="java-options".



4. Insert the following at the end of the java-options line, before the end "/> using the appropriate path for your environment:



For example, the entire entry for one application would be on one line:

<data id="java-options" value="-server -XX:MaxPermSize=128M -ms512M -mx1024M

-XX:AppendRatio=3

-Djava.security.policy=$ORACLE_HOME/j2ee/home/config/java2.policy

-Djava.awt.headless=true -Dhttp.webdir.enable=false

-javaagent:$ORACLE_HOME/wily/Agent.jar

-Dcom.wily.introscope.agentProfile=$ORACLE_HOME/wily/core/config/IntroscopeAg

ent.profile/>

Configure GlassFish 2.1 to use the Java agent

GlassFish is an open source application server project led by Sun Microsystems for the J2EE platform.

To configure GlassFish 2.1 for Introscope

1. In GlassFish, open the domain.xml file, located at

/appserver-install-dir/domains/domain1/config/

2. Add the full path for the Agent.jar file and the IntroscopeAgent.profile as jvm-options to the java-config element in the domain.xml file. For example:

<java-config ..........>

<jvm-options>-javaagent:/sw/wily/Agent.jar</jvm-options>

<jvm-options>-Dcom.wily.introscope.agentProfile=/sw/wily/core/config/Introsco

peAgent.profile</jvm-options>

.....>

3. Copy the WebAppSupport.jar from the <Agent_Home> root directory to the <Agent_Home>/core/config/ext directory.

4. Start the application server.

Configure SAP Netweaver 7.1 to use the Java agent

To configure JVM AutoProbe for NetWeaver 7.1

1. Start the SAP Configtool (configtool.bat).

2. Select an instance to modify.

3. In the right pane, select VM Parameters > System.

4. Create a new parameter (without providing -D). For example:

name: com.wily.introscope.agentProfile

value: <Agent_Home>/core/config/IntroscopeAgent.profile

Configuring the connection to the Enterprise Manager


5. Click the Additional tab and create a new parameter, for example:

name: -javaagent:<Agent_Home>\Agent.jar

value: <leave empty>

6. Save your changes.

7. Restart the SAP server.

8. To verify that the changes made with the Configtool were made, open the following file:

<drive>:\usr\sap\...\j2ee\cluster\instance.properties

9. Find the line beginning with ID<server_id>.JavaParameters, and confirm that it contains the information you entered above.


To report metrics, the agent must connect to an Enterprise Manager. The default communication settings enable an agent to connect to a local Enterprise Manager using port 5001. However, the agent and the Enterprise Manager do not typically reside on the same system. You can modify the default settings when you install the agent, or after installing the agent by modifying the IntroscopeAgent.profile.

Depending on your requirements, you can configure the communication between the agent and Enterprise Manager to use:

■ Direct socket connections

■ HTTP tunneling connections or HTTP tunneling through a proxy server

■ HTTP over Secure Sockets Layer connections (HTTPS)

■ Secure Socket Layer (SSL) connections

Connect to the Enterprise Manager Using a Direct Socket Connection

The most common way for the agent to connect to the Enterprise Manager is through a direct socket connection. CA Technologies recommends using a direct socket connection to the Enterprise Manager if possible.

To configure a direct socket connection from the agent to the Enterprise Manager


2. Locate the introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT property and specify the host name or IP address of the Enterprise Manager the agent should connect to by default. For example:

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT=sfcollect01

If you use a cluster with more than one Enterprise Manager, be sure to specify a Collector Enterprise Manager.



3. Set the introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT property to the Enterprise Manager listening port. For example:

introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT=5001

4. Set the introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT property to the socket factory used for connections to the Enterprise Manager. For example:

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wi

ly.isengard.postofficehub.link.net.DefaultSocketFactory

5. (Optional) Specify one or more backup Enterprise Managers for the agent to connect to, if the connection to the primary Enterprise Manager is lost.

6. Save and close the IntroscopeAgent.profile file.

Connect to the Enterprise Manager with HTTP tunneling

If a direct socket connection to the Enterprise Manager is not feasible, you can configure agents to connect to an Enterprise Manager over HTTP. This configuration allows communication to pass through firewalls that only permit HTTP traffic.

Note: HTTP tunneling imposes more CPU and memory overhead on the application server and Enterprise Manager than a direct socket connection.

To configure HTTP tunneling for an agent


2. Set the introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT property to the host name or IP address of the Enterprise Manager the agent should connect to by default. For example:

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT=webhost

3. Set the introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT property to the HTTP listening port of the Enterprise Manager’s embedded web server. For example:


This property should match the value specified in the <EM_Home>/config/IntroscopeEnterpriseManager.properties file on the Enterprise Manager for the introscope.enterprisemanager.webserver.port property. By default, this port is 8081.

4. Set the introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT property to the HTTP tunneling socket factory. For example:


ly.isengard.postofficehub.link.net.HttpTunnelingSocketFactory

5. Save and close the IntroscopeAgent.profile file.



Configure a proxy server for HTTP tunneling

You can configure the HTTP tunneled agent to connect through a proxy server to the Enterprise Manager. This configuration is necessary for a forwarding proxy server where the agent is running behind a firewall that only allows outbound HTTP traffic routed through the proxy server.

The proxy server configuration properties apply only if the agent is configured to tunnel over HTTP. The proxy server configuration applies to any configured HTTP tunneled connection on the agent, not to a single connection. This is especially important to consider when configuring failover between multiple Enterprise Managers, where the connection to each Enterprise Manager is over HTTP.

Important! HTTP/1.1 is required to enable agent HTTP tunneling. In addition, the proxy server must support HTTP Post.

To configure a proxy server for HTTP tunneling


2. Set the introscope.agent.enterprisemanager.transport.http.proxy.host property to the host name or IP address of the proxy server.

3. Set the introscope.agent.enterprisemanager.transport.http.proxy.port property to the port number of the proxy server.

4. (Optional) If the proxy server requires user credentials for authentication, set the following additional properties:

introscope.agent.enterprisemanager.transport.http.proxy.username=<user_name>

introscope.agent.enterprisemanager.transport.http.proxy.password=<user_passwo

rd>


Connect to the Enterprise Manager with HTTPS tunneling

The agent can connect to the Enterprise Manager using HTTP over Secure Sockets Layer (SSL) by configuring properties in the IntroscopeAgent.profile file.

To configure a connection through HTTPS


2. Set the introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT property to the host name or IP address of the target Enterprise Manager.

3. Set the introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT property to the HTTPS listening port of the Enterprise Manager's embedded web server. For example:




4. Set the introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT property to use an HTTP tunneling socket factory. For example:


ly.isengard.postofficehub.link.net.HttpsTunnelingSocketFactory


Connect to the Enterprise Manager over SSL

If you are using direct Secure Socket Layer (SSL) connections, you can configure the agent communication with the Enterprise Manager using SSL without HTTP tunneling.

To configure the agent to connect using SSL


2. Configure the agent to connect to the Enterprise Manager’s SSL listening port using an SSL socket factory.

3. Set the introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT property to the host name or IP address of the target Enterprise Manager.

4. Set the introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT property to the Enterprise Manager's SSL listening port.

5. Set the introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT property to the SSL socket factory. For example, set the property to:

com.wily.isengard.postofficehub.link.net.SSLSocketFactory

6. If the agent needs to authenticate the Enterprise Manager, uncomment and set the introscope.agent.enterprisemanager.transport.tcp.truststore.DEFAULT property to the location of the truststore containing the Enterprise Manager’s certificate. If you do not specify a truststore, the agent trusts all certificates.You can specify an absolute path or a path relative to the agent profile. For example:

introscope.agent.enterprisemanager.transport.tcp.truststore.DEFAULT=/certs

7. Set the introscope.agent.enterprisemanager.transport.tcp.trustpassword.DEFAULT property to specify the truststore password, if needed.

8. If the Enterprise Manager requires client authentication, set the introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT property to the location of a keystore containing the agent's certificate. You can specify an absolute path or a path relative to the agent profile. On Windows, backslashes must be escaped. For example:

introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT=C:\\keystor

e

9. Set the introscope.agent.enterprisemanager.transport.tcp.keypassword.DEFAULT property to the keystore password, if needed.

Upgrading multiple agent types


10. Set the introscope.agent.enterprisemanager.transport.tcp.ciphersuites.DEFAULT property to a comma-separated list of allowed cipher suites.

If you do not specify a value for this property, the default cipher suites are used.


Configure Agent Load Balancing

In clusters where the workload is primarily metrics reported by Introscope agents, you can optimize the overall cluster capacity by configuring MOM agent load balancing.

Note: For more information about MOM agent load balancing, see the CA APM Configuration and Administration Guide.

Upgrading multiple agent types

Some environments have thousands of agents distributed across many different application servers. For example, an environment might have 8,000 agents, with 3,000 agents on WebLogic, 2,000 on WebSphere, and 3,000 on JBoss. To make it easier to upgrade agents across multiple application servers, you can use agent superset package. Agent superset packages are operating system-specific packages that contains files for all supported application servers, except SAP NetWeaver.

The agent superset packages available are:

■ IntroscopeAgentFiles-NoInstaller<version>allappserver.windows.zip

■ IntroscopeAgentFiles-NoInstaller<version>allappserver.unix.tar

■ IntroscopeAgentFiles-NoInstaller<version>allappserver.zOS.tar

■ IntroscopeAgentFiles-NoInstaller<version>allappserver.os400.zip

Note: In these file names, <version> refers to the current release number of the Java agent.

Each package contains:

■ All application server-specific PBDs and PBLs

■ All application server agent profiles, with the application server name embedded in the file name. For example:

IntroscopeAgent.weblogic.profile

IntroscopeAgent.websphere.profile

The default IntroscopeAgent.profile is not been included. See step 3 for more information.

■ All agent .jars and platform monitors suitable for the operating system type

Uninstalling the Java agent


To upgrade multiple agent types using the superset agent packages

1. Select a superset package appropriate for the target operating system.

2. Extract the selected agent package into the application server’s home directory. Follow the manual installation instructions for Java agent installation. For more information, see Install manually using installation archives (see page 37).

Note: The extra PBDs and PBLs in the <Agent_Home>/core/config directory that refer to application servers you are not using can be safely ignored.

3. If you have not already configured an IntroscopeAgent.profile, select the appropriate IntroscopeAgent.<application_server_name>.profile, rename it to IntroscopeAgent.profile, and configure the file for use with your environment.

Note: If you have already configured an IntroscopeAgent.profile, open the corresponding IntroscopeAgent.<application_server_name>.profile file in an editor and look for new properties you may want to use. Transfer these properties to your existing IntroscopeAgent.profile.


Uninstalling the Java agent requires you to know where the Java agent was installed for each application being monitored.

If you used the Java agent installer to install the Java agent, you can use the uninstall program to remove the installed agent files. Launch the uninstaller and follow the on-screen directions.

To uninstall the Java agent from any supported JVM

1. Stop the application server.

Note: Monitored JVMs must be shut down before you run the uninstall program.

2. Open the application server startup script and remove the Java agent switches from the Java command line. Depending on the application server, the startup options include the following:

■ -Xbootclasspath

■ -javaagent:<path_to_the_agent_jar>

■ -Dcom.wily.introscope.agentProfile=<path_to_the_agent_profile>

3. Reboot the application server.

4. Run the Uninstall_Introscope_Agent program located in <Agent_Home>/UnstallerData directory.

5. Manually delete the <Agent_Home> directory.



Uninstalling the Java agent from z/OS

The recommended way to uninstall the Java agent from z/OS is to delete the <Agent_Home> directory using an rm -rf command. This is necessary because the executable uninstaller does not run properly on z/OS due to a third party bug.

Important! Only remove files after shutting down the monitored JVM.

For an active Introscope installation on z/OS, it is important to keep the UninstallerData folder intact. If you delete the UninstallerData folder, you will not be able to upgrade to future versions of Introscope. Do not delete the UninstallerData folder unless you have decided to uninstall the entire instance.


Chapter 3: Configuring Agent Properties

Most agent operations are configured using properties in the IntroscopeAgent.profile file. This section describes the most common agent properties to set. Additional properties may be appropriate for your environment. Different versions of the agent may have different properties available for you to set or different default values.


How to modify communication with Enterprise Manager (see page 65) How to configure backup Enterprise Managers and failover properties (see page 65) How to enable and use additional GC metrics (see page 67) How to enable and configure thread dumps (see page 67)

How to modify communication with Enterprise Manager

To report metrics, the agent must connect to an Enterprise Manager. After installing the agent, you can modify the communication channel used by modifying the IntroscopeAgent.profile.

Depending on your requirements, you can configure the communication between the agent and Enterprise Manager to use:

■ Direct socket connections

■ HTTP tunneling connections or HTTP tunneling through a proxy server

■ HTTP over Secure Sockets Layer connections (HTTPS)

■ Secure Socket Layer (SSL) connections

The most common way for the agent to connect to the Enterprise Manager is through a direct socket connection. CA Technologies recommends using a direct socket connection to the Enterprise Manager, if possible. If you want to use a different type of connection, see the appropriate section.

How to configure backup Enterprise Managers and failover properties

When you install the agent, you specify the Enterprise Manager host name and port number that the agent should connect to by default. Optionally, you can also specify one or more backup Enterprise Managers. If the agent loses the connection to its primary Enterprise Manager, it can then attempt to connect to one of its backup Enterprise Manager.

How to configure backup Enterprise Managers and failover properties


To enable an agent to connect to a backup Enterprise Manager, you specify the communication properties for the Enterprise Managers in the agent profile, including the order in which the agent should attempt to contact backup Enterprise Managers. If the primary Enterprise Manager is not available, the agent tries to connect to the next Enterprise Manager on its list of allowed connections. If the agent does not connect with its first backup Enterprise Manager, it tries the next Enterprise Manager in the list. It continues trying to connect to each Enterprise Managers in the list, in order, until it succeeds in connecting. If the agent is unable to connect to any of its Enterprise Managers, it waits ten seconds before retrying.

To configure one or more backup Enterprise Managers for an agent

1. Open the Introscope.Agent.profile file in a text editor.

2. Specify one or more alternative Enterprise Manager communication channels by adding the following properties to the agent profile for each backup Enterprise Manager:

introscope.agent.enterprisemanager.transport.tcp.host.NAME

introscope.agent.enterprisemanager.transport.tcp.port.NAME

Introscope.agent.enterprisemanager.transport.tcp.socketfactory.NAME

Replace NAME with an identifier for the new Enterprise Manager channel. Do not use DEFAULT or the name of an existing channel when creating a new channel. For example to create two backup Enterprise Managers:

introscope.agent.enterprisemanager.transport.tcp.host.BackupEM1=paris

introscope.agent.enterprisemanager.transport.tcp.port.BackupEM1=5001

Introscope.agent.enterprisemanager.transport.tcp.socketfactory.BackupEM1=com.

custom.postofficehub.link.net.DefaultSocketFactory

introscope.agent.enterprisemanager.transport.tcp.host.BackupEM2=voyager

introscope.agent.enterprisemanager.transport.tcp.port.BackupEM2=5002

introscope.agent.enterprisemanager.transport.tcp.socketfactory.BackupEM2=com.

wily.isengard.postofficehub.link.net.DefaultSocketFactory

3. Locate the introscope.agent.enterprisemanager.connectionorder property and set it to a comma-separated list of identifiers for the primary and backup Enterprise Managers. The order in which you list the identifiers defines the order in which they are contacted. For example:

introscope.agent.enterprisemanager.connectionorder=DEFAULT,BackupEM1,BackupEM

2

4. Locate the introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds property and specify how frequently the agent should try connecting to its primary Enterprise Manager. The default interval is 120 seconds (two minutes). For example:

introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds=120

5. Save the changes and close the IntroscopeAgent.profile file.

6. Restart the application.

How to enable and use additional GC metrics


How to enable and use additional GC metrics

Garbage collection and memory management can have a significant effect on an applications performance. Basic GC Heap metrics are available by default. There are also optional metrics that you can enable to provide additional details about garbage collection processing and memory pool usage. These additional metrics display under the GC Monitor node in the Investigator when enabled. The GC Monitor metrics report information to help you optimize memory pool allocation and garbage collection processing. Therefore, these metrics are typically enabled when developing or testing applications or when researching application performance issues. In most cases, the metrics are not used for real-time application management in a production environment and are disabled by default.

If you want to enable the GC Monitor metrics, you can open the agent profile in a text editor and set the introscope.agent.gcmonitor.enable property to true. After you enable the property, you can view details about the Garbage Collectors and Memory Pools for the JVM you are monitoring. For more information about the metrics, see the CA APM Workstation User Guide.

How to enable and configure thread dumps

Thread dumps can provide useful detailed information about what is happening within the agent JVM. Thread dump functionality is provided in the Thread Dumps tab associated with each agent node in the metric browser tree.

For information about collecting and analyzing thread dumps, see the CA APM Workstation User Guide. Setting the Thread_Dump permission allows users to see the Thread Dumps tab and use all the functionality. For more information, see the CA APM Security Guide.

Both IntroscopeAgent.profile and IntroscopeEnterpriseManager.properties properties are required to enable thread dumps. By default, the Thread Dumps tab and its functionality are enabled. However, if either or both of the thread dump enable properties are set to false, then users cannot see the Thread Dumps tab.

If you enable or disable thread dumps on a MOM, that configuration applies to all the Collectors in the cluster. Therefore, if you disable thread dumps on a MOM, thread dumps are disabled on all the Collectors too.

To enable thread dumps

1. Open the IntroscopeAgent.profile file in the <Agent_Home>/wily/core/config directory and set this property:

introscope.agent.threaddump.enable=true

2. Save and close IntroscopeAgent.profile.



3. Open the IntroscopeEnterpriseManager.properties file located in the <EM_Home>/config directory and set this property:

introscope.enterprisemanager.threaddump.enable=true

4. Save and close IntroscopeEnterpriseManager.properties.

For CA Introscope users to view the Deadlock Count metric, configure the IntroscopeAgent.profile. You can perform additional configuration to display agent Thread node metrics.

To enable Deadlock Count metric collection

1. Open the IntroscopeAgent.profile file in the <Agent_Home>/wily/core/config directory.

2. Set this property to true to enable the Deadlock Count metric to be collected.

introscope.agent.threaddump.deadlockpoller.enable=true

3. (Optional) Set the full version of the PBL to display metrics in the agent Threads node.

■ Specify the name of the PBL file in this property: introscope.autoprobe.directivesFile.

For example, to use the full version of the standard PBL for WebLogic server, set the property to:

introscope.autoprobe.directivesFile=weblogic-full.pbl

User can see metrics such as Active Threads counts and thread groups under AgentName | Threads.

4. Save and close IntroscopeAgent.profile.

Configure thread dumps using both IntroscopeAgent.profile and IntroscopeEnterpriseManager.properties properties.

To configure thread dumps

1. Open the IntroscopeEnterpriseManager.properties file located in the <EM_Home>/config directory.

2. (Optional) Set this property to save thread dump files to a specific directory on the Enterprise Manager. For example, TestThreadDumps.

introscope.enterprisemanager.threaddump.storage.dir=TestThreadDumps

3. (Optional) Set this property to purge thread dump files older than a specified number of days. For example, older than 30 days.

introscope.enterprisemanager.threaddump.storage.clean.disk.olderthan.days=30

4. (Optional) Set this property to purge thread dump files after a specified number of days. For example, after every two days.

introscope.enterprisemanager.threaddump.storage.clean.disk.freq.days=2



5. (Optional) Set this property to limit the maximum number of thread dump files that can be stored on the Enterprise Manager. For example, 5000 files.

introscope.enterprisemanager.threaddump.storage.max.disk.usage=5000

Note: If:

* the number of thread dump files stored exceeds the limit set in the introscope.enterprisemanager.threaddump.storage.max.disk.usage property

and

* there are no files older than the number of days set in the introscope.enterprisemanager.threaddump.storage.clean.disk.olderthan.days property

Then the Enterprise Manager does not store any thread dump files.

6. Save and close IntroscopeEnterpriseManager.properties.

7. Restart the Enterprise Manager.

If an Enterprise Manager goes down, you can copy thread dump files to another Enterprise Manager so users can view thread dump data.

Important! Restart the Enterprise Manager when you add files to or remove files from the thread dump directory. CA Technologies does not recommend moving thread dump files from one Enterprise Manager to another.

To copy thread dump files from one Enterprise Manager to another

1. Navigate to the <EM_Home>/threaddumps directory on the Enterprise Manager containing thread dump files (EM1).

2. Copy the thread dump files.

3. Paste the files into the <EM_Home>/threaddumps directory on the Enterprise Manager where users are to view the thread dumps (EM2).

4. Restart both Enterprise Managers EM1 and EM2.

5. If needed, establish the agent connection and enable and configure thread dumps on EM2.

EM2 users can select the agent node then click the Load Previous button in Thread Dumps tab. A list displays the thread dumps moved from EM1.


Chapter 4: AutoProbe and ProbeBuilding Options


AutoProbe and ProbeBuilding overview (see page 71) Configuring ProbeBuilding (see page 72)

AutoProbe and ProbeBuilding overview

The Java agent inserts probes into the bytecode of applications you want to monitor. ProbeBuilding is the process by which you choose which probes to insert into applications using ProbeBuilder Directives (PBDs) and ProbeBuilder Lists (PBLs). The default PBD and PBL files included with the Java agent provide a basic level of metric collection. In most cases, you will want to modify the default settings in these files to better tune metric collection specifically for your environment.

After you have configured PBDs and PBLs to insert the probes for the metrics you want to collect in your applications and environment, you use these files to instrument applications automatically using JVM AutoProbe or manually using ProbeBuilder. In most cases, you should use JVM AutoProbe to instrument applications. Configuring JVM AutoProbe, however, depends on the application environment.

To instrument your applications on JVMs 1.5 and higher, use the following methods

■ JVM AutoProbe, with -javaagent property. CA Technologies highly recommends using JVM AutoProbe to instrument applications on JVMs 1.5 or higher.

■ Manual ProbeBuilding. Manual ProbeBuilding is an advanced instrumentation technique. Contact CA Support before using this method.

Important! When instrumenting your applications, use only one method of instrumentation.

The instructions in this section assume that you have performed the installation and configuration tasks outlined in Installing and Configuring the Java Agent (see page 29).

Configuring ProbeBuilding


Unsupported instrumentation methods

The AutoProbe for Application Servers method for instrumentation is no longer supported for Java agents.You can use AutoProbe for Application Servers to instrument applications that use older versions of the JVM (1.4 and earlier) with earlier versions of the Java agent, but CA Technologies recommends that you use JVM AutoProbe for applications that use JVM 1.5 or later.

For more information, see The Java Agent and Application Server AutoProbe (see page 315).


The instrumenting process is performed using ProbeBuilding technology, in which probes defined in ProbeBuilder Directive (PBD) files identify the metrics an agent will gather from web applications and virtual machines at run-time.

By default, AutoProbe will use the typical PBD set provided with the Java agent, which results in the collection of a moderate number of metrics. The following sections have instructions on how to customize the metric collection level and how to configure optional ProbeBuilding behaviors.

■ Full or typical tracing options (see page 72)

■ Dynamic ProbeBuilding (see page 73)

■ ProbeBuilding class hierarchies (JVM 1.5) (see page 76)

■ Removing line numbers in bytecode (see page 79)

Full or typical tracing options

In Introscope, ProbeBuilder List (PBL) files govern which tracer groups are used in the instrumentation process. The introscope.autoprobe.directivesFile property specifies one or more PBL files.

Introscope provides two versions of each default PBL—a full version which enables a larger set of Tracer Groups than the typical version which results in more detailed metric reporting, and a typical version that enables a smaller set of Tracer Groups. This results in less detailed metric reporting and reduced overhead. By default, introscope.autoprobe.directivesFile specifies the typical version of the default PBL file.



To change the tracing level between full and typical

1. Stop the managed application.


3. Specify the name of the PBL file you want to use in this property: introscope.autoprobe.directivesFile.

For example, to use the Full version of the standard PBL for WebLogic Server, set the property to:

introscope.autoprobe.directivesFile=weblogic-full.pbl


Dynamic ProbeBuilding

Introscope uses dynamic ProbeBuilding to implement new and changed PBDs without restarting managed applications or the agent itself. Dynamic ProbeBuilding is useful for making corrections to PBDs, or to change data collection levels during triage without interrupting application service.

Important! Dynamic ProbeBuilding is only available for use with Java 1.5 or higher. Dynamic ProbeBuilding is dependent on Java 1.5 capabilities, so previous versions of Java are not able to use this Introscope function. Dynamic ProbeBuilding is also dependent on the -javaagent command.

Note: The Introscope Workstation allows you to perform dynamic instrumentation through the Transaction Trace viewer. For more information, see the CA APM Workstation User Guide.

When dynamic ProbeBuilding is enabled, Introscope periodically checks for new and changed PBDs. To minimize overhead, Introscope selectively re-instruments classes affected by the modified PBDs. To improve performance, the scope of dynamic agent re-instrumentation is limited to reloading only those classes whose instrumentation has changed when PBDs were edited.

When a PBD is edited or added to the hotdeploy directory, only user directives (such as adding or removing directives for a class, or toggling tracer groups) are re-instrumented. System directives (such as adding a tracer or changing a new tracer mapping) are not re-instrumented. Arrays, interfaces, and classes specified in Skip directives are not re-instrumented, as well as any transformations. In addition, you can exclude all classes loaded by particular classloaders from the re-instrumentation process and limit the scope of the re-instrumentation process to specific class packages.

Note: Dynamic ProbeBuilding is not enabled by default.



If a class is re-instrumented so that it no longer reports data for a metric, the metric is still displayed in the Introscope Investigator. Existing metrics do not disappear from the Investigator window if their classes are re-instrumented.

Important! Due to a limitation in Java 1.5, access to some class bytes is not available, with the following effects:

Modifications to the j2ee.pbd file may not be picked up, and metrics may continue to be published under old names.

Some exceptions may appear in the agent log.

To avoid this issue, restart the application server after modifying the j2ee.pbd file.

When configuring dynamic ProbeBuilding, CA Technologies recommends that you base your changes on tracer groups. For example, if you want to control the level of instrumentation for the tracer group XYZ, you should create two tracer groups:

■ XYZTracing - regular tracing options

■ XYZTracingLite - fewer components are traced

Once these two tracer groups have been created, you can toggle between them, turning off XYZTracing and turning on XYZTracingLite. By toggling between the two tracer groups, you can view the impact that dynamic ProbeBuilding has on your environmental performance and adjust the tracing groups accordingly. This would affect all classes being traced as part of each tracer group.

Important! Changes to directives not using tracer groups are not supported. For example, changes in any directive like TraceAllMethods that does not have an IfFlagged switch are not supported. However, Introscope does not ship any out of the box directives without tracer groups or flags. Changes to skips or transformations are also not supported.

To configure dynamic ProbeBuilding


2. Verify that the property, introscope.autoprobe.enable, is set to true.

3. Uncomment and set the following properties:

■ introscope.autoprobe.dynamicinstrument.enabled=true

This property enables dynamic ProbeBuilding. You must restart the managed application before changes to this property take effect.

■ introscope.autoprobe.dynamicinstrument.pollIntervalMinutes=1

The polling interval in minutes to check for PBD changes. The default is set to one minute intervals. You must restart the managed application before changes to this property take effect.



■ introscope.autoprobe.dynamicinstrument.classFileSizeLimitInMegs=1

Some classloader implementations have been observed to return huge class files. This is to prevent memory errors. You must restart the managed application before changes to this property take effect.

■ introscope.autoprobe.dynamic.limitRedefinedClassesPerBatchTo=10

Re-defining too many classes at a time might be very CPU intensive. In cases where the changes in PBDs trigger a re-definition of a large number of classes, this property attempts to batch the process at a comfortable rate.

4. Save changes to the IntroscopeAgent.profile.

5. Restart the managed applications (if appropriate).

Dynamic ProbeBuilding vs. dynamic instrumentation

Dynamic ProbeBuilding and dynamic instrumentation are not the same thing.

As outlined in Dynamic ProbeBuilding (see page 73), dynamic ProbeBuilding is based on manual changes you make to PBD files and manual configurations you make in the IntroscopeAgent.profile. If you update or change a PBD file and save it in the correct location, dynamic ProbeBuilding picks up the changes and implements them. Dynamic ProbeBuilding requires you to make configuration changes in the IntroscopeAgent.profile, as well as all changes to the PBD files you want updated, and all changes are permanent (until you manually update or change the files again).

Dynamic instrumentation is performed from the Introscope Workstation transaction trace viewer. The changes to instrumentation you select using the interface are made automatically for you, and are often only temporary. Instrumenting a method dynamically means inserting the instrumentation during runtime. You can dynamically instrument one, more, or all of the methods during a transaction trace session, and subsequently view metrics returned by the newly instrumented methods. This allows you to do dynamic application performing tuning. Dynamic instrumentation does not require any changes to the IntroscopeAgent.profile, and if you decide to make instrumentation changes permanent, a new PBD is created and saved in the correct location for you.

Note: For more information on how to use dynamic instrumentation from the Introscope Workstation transaction trace viewer, see the CA APM Workstation User Guide.

Important! To use dynamic instrumentation, you (the user) must have write access to the <Agent_Home> directory as well as the <Agent_Home>/logs directory. Since you must sign in to the Workstation to perform dynamic instrumentation, your permissions must allow you write access to the directories above. If you do not have write access to these directories, dynamic instrumentation cannot create the <Agent_Home>/Dynamic directory or create the dynamic instrumentation cache which it needs to function.



Dynamic instrumentation requires class redefinition support. Use of class redefinition can significantly impact performance when running on IBM JDK version 5. IBM has provided technical information on this performance overhead in the Java Diagnostics Guide.

Introscope and IBM JDK version 5 customers that would like to take advantage of dynamic instrumentation should keep this performance overhead in mind, and when employing this configuration, CA Technologies recommends using the dynamic instrumentation feature only in a production environment. The Introscope documentation contains details about configuring Introscope on IBM JDK Version 5 with class redefinition both enabled and disabled. For more information see:

■ AutoProbe for WebSphere 6.1 (see page 47)

■ Socket metrics (see page 128)

Note: There is no performance overhead with the use of class redefinition when using Introscope with IBM JDK version 6.

ProbeBuilding class hierarchies (JVM 1.5)

In pre-1.5 JVMs, Introscope does not automatically instrument classes in the deeper levels of class hierarchy—only the classes that explicitly extend a probed class. For more information, see Instrumenting and inheritance (see page 111).

With a 1.5 JVM and higher, you can configure Introscope to instrument multiple levels of subclasses of a probed class—the tracer groups in the associated internal directive are updated appropriately and the classes are dynamically instrumented. Directive changes are written to a log file as well.

If you prefer to update your PBDs manually, you can disable directive updates and use the log file to determine appropriate updates.

Enable instrumentation of multiple levels of subclasses

Follow these steps to configure Introscope to dynamically update internal directives.

To enable instrumentation of multiple levels of subclasses

1. Verify that dynamic instrumentation is enabled as described in Dynamic ProbeBuilding (see page 73).

2. Open the IntroscopeAgent.profile.

3. To enable instrumentation of multiple levels of subclasses, uncomment this property setting:

introscope.autoprobe.hierarchysupport.enabled=true


http://publib.boulder.ibm.com/infocenter/javasdk/v5r0/index.jsp?topic=/com.ibm.java.doc.diagnostics.50/diag/tools/jvmti.html

http://publib.boulder.ibm.com/infocenter/javasdk/v5r0/index.jsp?topic=/com.ibm.java.doc.diagnostics.50/diag/tools/jvmti.html



Support for multiple inheritance, interfaces, and abstract methods

The Java agent supports instrumentation by interface and by inheritance. This ability has been extended to dynamic instrumentation in Introscope 9.0.

In addition, the 9.0 Java agent supports the instrumentation of methods through a call to subclasses by using the Introscope API getMethodCalls. When used, getMethodCalls allows you to better understand the consequences of instrumentation of inherited methods or interface methods by providing the following information:

■ if the class defining the method is an interface.

■ the number of classes affected by a possible instrumentation of the method. This is the number of subclasses or the number of implementing classes.

■ if the method is called within a specific stack trace.

In addition, Introscope 9.0 also supports the instrumentation of methods through a call to subclasses by using the Introscope API getMethodCalls. When used, getMethodCalls allows you to better understand the consequences of instrumentation of inherited methods or interface methods by providing the following information:

■ if the class defining the method is an interface.

■ the number of classes affected by a possible instrumentation of the method. This is the number of subclasses or the number of implementing classes.

■ if the method is called within a specific stack trace.

A new tracer to instrument interfaces and abstract methods is available with the 9.0 Java Agent, using the following syntax:

TraceOneMethodWithlabelIfInherits: <class> <method> <Label> <Tracer Group> <Tracer

Type> <Resource>

This tracer instruments a method of any class implementing interface or extending superclass if the method is either defined in the interface or is abstract in the superclass

Important! Using this tracer could have a heavy impact on the performance of your system. You should test the impact of this tracer during your system startup and instrumentation processes before deploying to a larger agent configuration.

For more information on tracers and creating customs tracers, see Creating custom tracers (see page 102).



Configure periodic polling for uninstrumented subclasses

When multi-level subclass instrumentation is enabled, Introscope will check for uninstrumented subclasses at application startup.

To configure Introscope to poll for uninstrumented subclasses


2. Uncomment this property setting:

introscope.autoprobe.hierarchysupport.runOnceOnly=false

3. To change the frequency with which Introscope polls for uninstrumented subclasses from its default value of 5, uncomment this property and set it to the desired polling frequency:

introscope.autoprobe.hierarchysupport.pollIntervalMinutes

4. Optionally, you can limit the number of times Introscope polls uninstrumented subclasses by uncommenting this property and setting it to the desired limit:

introscope.autoprobe.hierarchysupport.executionCount

The default of this property is 3 minutes.


Disable directive updates

If multi-level subclass instrumentation is enabled, when Introscope detects uninstrumented subclasses, by default, it updates internal directives appropriately to ensure the classes are instrumented. If you prefer to update PBDs manually, you can disable internal directive updates by uncommenting this property in the IntroscopeAgent.profile:

introscope.autoprobe.hierarchysupport.disableDirectivesChange=true



Controlling directive logging

When multi-level subclass instrumentation is enabled, you must uncomment the following properties in the IntroscopeAgent.profile to have multi-level subclass instrumentation logs created. When these properties are configured, a log file named pbdupdate.log is created in the <Agent_Home>/wily directory (by default), or in the custom location (if specified). The multi-level instrumentation details are written to the agent logs.

log4j.additivity.IntroscopeAgent.inheritance=false

log4j.logger.IntroscopeAgent.inheritance=INFO,pbdlog

log4j.appender.pbdlog.File=pbdupdate.log

log4j.appender.pbdlog=com.wily.introscope.agent.AutoNamingRollingFileAppender

log4j.appender.pbdlog.layout=com.wily.org.apache.log4j.PatternLayout

log4j.appender.pbdlog.layout.ConversionPattern=%d{M/dd/yy hh:mm:ss a z} [%-3p] [%c]

%m%n_

You must restart the managed application before changes to these properties take effect.

Removing line numbers in bytecode

When you instrument application bytecode, the bytecode line numbers are preserved by default. Preserving bytecode line number information is helpful when using debuggers or when obtaining stack trace information.

You can turn off this feature by adding a system property on the Java command line. Turning off this feature removes all line numbers when AutoProbe or ProbeBuilder instruments the application code.

To remove line numbers in bytecode when using AutoProbe or ProbeBuilder

■ Define this system property on the Java command line with the -D option:

com.wily.probebuilder.removeLineNumbers=true


Chapter 5: ProbeBuilder Directives

This section describes how to create and modify ProbeBuilder Directives.


ProbeBuilder Directives Overview (see page 81) Using the IntroscopeAgent.profile, PBLs, and PBDs together (see page 99) Applying ProbeBuilder Directives (see page 99) Creating custom tracers (see page 102) Using Blame Tracers to mark blame points (see page 113)

ProbeBuilder Directives Overview

ProbeBuilder Directive (PBD) files tell the Introscope ProbeBuilder how to add probes, such as timers and counters, to instrument an application. PBD files govern what metrics your agents report to the Introscope Enterprise Manager.

Note: All metrics are calculated using the time set by your system clock. If the system clock is reset during a transaction, the elapsed time reported for that transaction can be misleading.

Introscope includes a set of default PBD files. You can also create custom Introscope PBD files to track any classes or methods to obtain specific information about your applications.

Two kinds of files specify ProbeBuilder Directives:

■ ProbeBuilder Directive (PBD) files

A ProbeBuilder Directive (PBD) file contains directives used by ProbeBuilder to instrument your applications. This file determines which metrics the agents report to the Enterprise Manager.

■ ProbeBuilder List (PBL) files

A ProbeBuilder List (PBL) file contains a list of multiple PBD filenames. Different PBL files can refer to the same PBD files.

Important! PBDs and PBLs only support ASCII characters. Placing other characters (such as Unicode characters) in PBDs or PBLs could result in problems with AutoProbe.

When you install the Java agent and you are using Introscope AutoProbe, the relevant PBD and PBL files for your specific application server are included. These files are located in the <Agent_Home>\wily\core\config directory.



More information:

Default PBD Files (see page 83) Default PBL files (see page 86) Creating custom tracers (see page 102)

Components traced by the default PBDs

The default Introscope PBD files implement tracing of the following Java components:

■ Oracle JDBC

■ JSP Tag Libraries

■ JSP IO Tag Libraries

■ JSP DB Tag Libraries

■ Struts

■ Servlets

■ JavaServer Faces (JSF)

■ JavaServer Pages (JSP)

■ Enterprise JavaBeans (EJBs)

■ Java Database Connectivity (JDBC)

■ Network Sockets

■ Remote Method Invocation (RMI)

■ Extensible Markup Language (XML)

■ Java Transaction API (JTA)

■ Java Naming and Directory Interface (JNDI)

■ Java Message Service (JMS)

■ Common Object Request Broker Architecture (CORBA)

■ User Datagram Protocol (UDP)

■ File Systems

■ Threads

■ System Logs

■ Thrown and Caught Exceptions (off by default)



Occasionally, too many Java classes are selected for monitoring in a ProbeBuilder Directive (PBD) file, causing the Java agent to start incorrectly, or to experience a "hang". If this happens, use the AutoProbe.log file to identify the classes that caused the Java agent to hang and add a skip directive to the PBD file, skipping the classes that may have caused the problem. For more information about adding skip directives to your custom PBD files, see Skip directives (see page 73).

Default PBD Files

The Java agent includes the following default PBD files:

appmap.pbd

Provides tracer directives for application triage map instrumentation.

appmap-ejb.pbd

Provides tracer directives for application triage map EJB instrumentation.

appmap-soa.pbd

Provides tracer directives for application triage map SOA instrumentation for SPM supported Java SOAP stacks.

Note: For more information, see the CA APM for SOA Implementation Guide.

bizrecording.pbd

Provides tracer definitions and directives to setup agent business recording.

biz-trx-http.pbd

Provides tracer directives for business-centric HTTP instrumentation.

di.pbd

Provides directives ProbeBuilder to not instrument Apache Derby implementation classes.

errors.pbd

Configures Error Detector by specifying what code-level events constitute serious errors. By default, only front-end and back-end errors are considered serious. That is, only errors that manifest as a user-facing error page or that indicate a problem with a back-end system (ADO.NET, Messaging, and so on).

j2ee.pbd

Provides tracer groups for common Java Enterprise Edition components. Use either toggles-full.pbd or toggles-typical.pbd to TurnOn specific tracing.

java2.pbd

Provides tracer groups for common Java 2 components. Use either toggles-full.pbd or toggles-typical.pbd to TurnOn specific tracing.



jsf.pbd

Provides tracer groups for Java Server Face (JSF) components.

jsf-toggles-full.pbd

Provides on/off switches in the form of TurnOn directives for the tracing provided in the jsf.pbd. Most tracer groups are turned on.

jsf-toggles-typical.pbd

Provides on/off switches in the form of TurnOn directives for the tracing provided in the jsf.pbd.

jvm.pbd

Provides directives which implement support for various Java Virtual Machines. You use this property with the Introscope default files.

leakhunter.pbd

Provides instrumentation settings for CA APM LeakHunter, a leak detection utility. Typically, you do not modify the contents of this file.

oraclejdbc.pbd

Provides tracer groups for Oracle JDBC components. Comment or uncomment the TurnOn directives to alter the set of Oracle JDBC components that are traced.

ServletHeaderDecorator.pbd

Enables the servlet header decorator, which is part of the integration with CA CEM.

smwebagentext.pbd

Provides tracers for the SiteMinder Web Agent Introscope plug-in.

soaagent.pbd

Provides tracers for TransactionMinder Agent, part of the CA SOA Security Manager (SOA Agent for Web Server and Application Server).

spm-correlation.pbd

Provides directives that control the correlation of transaction traces across components. This file is required to enable cross-process transaction tracing when you use CA APM for SOA.



struts.pbd

Provides directives which monitor Apache struts. Use this property with the Introscope default files.

summary-metrics-6.1.pbd

Provides directives necessary for JSP tracing, Servlet tracing, and EJB tracing in pre-7.0 Introscope instances.

taglibs.pbd

Provides directives that monitor classes that should be traced as JSP tag libraries, Jakarta I/O libraries, and DGTags tag libraries.

TIBCO pbd

The Java agent is installed with several PBDs related to monitoring TIBCO through the SOA extension.


toggles-full.pbd

Provides on/off switches in the form of TurnOn directives for the tracing provided in other directives files. Most tracer groups are turned on.

toggles-typical.pbd

Provides on/off switches in the form of TurnOn directives for the tracing provided in other directives files. Only a small section of tracer groups is turned on.

webMethods pbds

The Java agent is installed with several PBDs related to monitoring webMethods through the CA APM for webMethods Broker.


WebSphere MQ pbds

The Java agent is installed with several PBDs related to monitoring WebSphere MQ connectors and messaging system through the CA APM for IBM WebSphere MQ.

Note: For more information, see the CA APM for IBM WebSphere MQ Guide.

The Java agent also installs application server-specific PBDs, which vary depending on the application server you are monitoring.

More information:

Default tracer groups and toggles files (see page 86) Turning tracer groups on or off (see page 95)



Default PBD Files From Previous Releases

The agent uses the PBD and PBL files from the current release by default. However, the product provides the PBD and PBL files from previous releases in the <Agent_Home>/wily/examples/legacy directory. Each of the file names in this directory have a -legacy suffix, for example, default-full-legacy.pbl.

Default PBL files

There are two PBL files available with each agent:

default-full.pbl (default)

References PBD files in which most tracer groups are turned on. Introscope uses this set by default to demonstrate full Introscope functionality.

default-typical.pbl

A subset of tracer groups in the referenced PBD files are turned on. The typical set includes common settings, and is the set you can customize for a particular environment.

The Java agent also installs application server-specific PBLs, which vary depending on the application server you are monitoring.

Default tracer groups and toggles files

Tracer groups are defined in PBD files. They cause the reporting of information about a set of classes. In PBD files, tracer group information is referred to by the term flag. For example, TraceOneMethodIfFlagged or SetFlag are defining tracer group information.

A tracer group consists of a set of tracers that is applied to a set of classes. For example, there are tracer groups which report the response times and rates for all RMI classes.

You can refine the gathering of metrics on your systems by turning on or off certain tracer groups. This affects overhead usage, either increasing or decreasing it, depending on how you configure the tracer groups.



Tracer groups are modified in the toggles-full.pbd and the toggles-typical.pbd files, which are referred to by the default-full.pbl and default-typical.pbl files. This table lists default tracer groups and their default configurations:

AgentInitialization

Agent Initialization Configuration

Full: on

Typical: on

ApacheStandardSessionTracing

HTTP Session Configuration

Full: on

Typical: on

AuthenticationTracing

Authentication Configuration

Full: on

Typical: on

CorbaTracing

CORBA method invocations

Full: on

Typical: on

DBCPTracing

DBCP Configuration

Full: on

Typical: on

DBCPv55Tracing

DBCP Configuration

Full: on

Typical: on

EJB2StubTracing

EJB 2.0 Configuration

Full: on

Typical: on



EJB3StubTracing

EJB 3.0 Configuration

Full: on

Typical: on

EntityBean3Tracing

Entity EJB 3.0 method invocations

Full: on

Typical: on

EntityBeanTracing

Entity EJB method invocations

Full: on

Typical: on

HTTPServletTracing

HTTP servlet service responses

Full: on

Typical: on

If you are using Application Server AutoProbe, turn on this tracer group: HTTPAppServerAutoProbeServletTracing.

InstanceCounts

Counts number of instances of object type identified with tracer group.

Full: on

Typical: on

Nothing will be traced until classes are identified with this tracer group.

J2eeConnectorTracing

J2EE connector information

Full: on

Typical: on

JavaMailTransportTracing

Mail sending times

Full: on

Typical: on



JDBCQueryTracing

JDBC queries

Full: on

Typical: on

JDBCUpdateTracing

JDBC updates

Full: on

Typical: on

JMSConsumerTracing

JMS message processing times

Full: on

Typical: on

JMSListenerTracing

JMS message processing times

Full: on

Typical: on

JMSPublisherTracing

JMS message broadcast times

Full: on

Typical: on

JMSSenderTracing

JMS message broadcast times

Full: on

Typical: on

JSPTracing

JSP service responses

Full: on

Typical: on

MessageDrivenBean3Tracing

Message-driven EJB 3.0 method invocations

Full: on

Typical: on



MessageDrivenBeanTracing

Message-driven EJB method invocations

Full: on

Typical: on

NIOSocketTracing

Generates a set of metrics for each socket connection under the NIO|Channels|Sockets node, with additional metrics under the Backends node.

Full: on

Typical: on

NIOSocketSummaryTracing

Generates a single set of metrics covering all NIO socket connections

Full: on

Typical: on

Included in these metrics are connections that were excluded from NIOSocketTracing metrics by agent properties, as well as some internal NIO sockets which are always excluded from NIOSocketTracing metrics.

NIODatagramTracing

Generates a set of metrics for each datagram "connection".

Full: on

Typical: on

See NIODatagramTracing metrics on page 269 for more information.

NIODatagramSummaryTracing

Generates a single set of metrics measuring all NIO datagram activity.

Full: on

Typical: on

Included in these metrics are connections that were excluded from NIODatagramTracing metrics by agent properties, as well as some internal NIO sockets which are always excluded from NIODatagramTracing metrics.

PersistentSessionTracing

HTTP session configuration

Full: on

Typical: on



RMIClientTracing

RMI client method invocations

Full: on

Typical: on

RMIServerTracing

RMI server method invocations

Full: on

Typical: on

ServerInfoTracing

Server info configuration

Full: on

Typical: on

SessionBean3Tracing

Session EJB 3.0 method invocations

Full: on

Typical: on

SessionBeanTracing

Session EJB method invocations

Full: on

Typical: on

SocketTracing

Network socket bandwidth as well as SSL tracking

Full: on

Typical: on

StrutsTracing

Execution times of actions in the Struts framework

Full: on

Typical: on

SuperpagesSessionTracing


Full: on

Typical: on



ThreadPoolTracing

Thread Pool Configuration

Full: on

Typical: on

UDPTracing

User Datagram Protocol (UDP) socket bandwidth

Full: on

Typical: on

UnformattedSessionTracing


Full: on

Typical: on

EJB3MethodLevelTracing

EJB 3.0 activity at method level

Full: on

Typical: off

EJBMethodLevelTracing

EJB activity at method level

Full: on

Typical: off

FileSystemTracing

File system bytes written and read

Full: on

Typical: off

JAXMListenerTracing

JAXM message sends

Full: on

Typical: off

JNDITracing

JNDI lookup times

Full: on

Typical: off



JSPDBTagsTagLibraryTracing

Jakarta DB Tags custom tag library for reading and writing from a SQL database

Full: on

Typical: off

JSPIOTagLibraryTracing

Jakarta IO custom tag library for a variety of input and output tasks

Full: on

Typical: off

JTACommitTracing

Commit times using JTA

Full: on

Typical: off

ThreadTracing

Number of active threads by class

Full: on

Typical: off

XMLSAXTracing

Time spent parsing XML document

Full: on

Typical: off

XSLTTracing

XML transformation time

Full: on

Typical: off

CatchException

Exception configuration

Full: off

Typical: off

FormattedSessionTracing

HTTP Session configuration

Full: off

Typical: off



HTTPAppServerAutoProbeServletTracing

HTTP Servlets configuration

Full: off

Typical: off

HTTPSessionTracing

HTTP Session configuration

Full: off

Typical: off

JSPTagLibraryTracing

Processing time of custom JSP tags

Full: off

Typical: off

ManagedSocketTracing

Network configuration

Full: off

Typical: off

ThrowException

Exception configuration

Full: off

Typical: off

Generally, the default toggles PBD files should not be edited. However, you can refine the gathering of metrics by turning on or off certain tracer groups. Tracer groups can be modified in the toggles files by:

■ Turning on/off tracer groups to save on system overhead

■ Adding classes to a tracer group

Tracer groups report information only when turned on (uncommented) and are activated with the keyword TurnOn.

Note: The Java Agent 9.0 supports EJB 2.0 and 3.0 instrumentation. Turn on or off the tracer groups associated with EJB 2.0 or 3.0 to tailor your metric collection. Support for EJBs in the application triage map extends only to Session and Entity beans. Message Driven beans are not supported yet.



Setting toggles to gather additional metric information

The following toggles, when turned on, cause the collection of additional metrics across all APIs for CA Technologies-provided tracer groups that are enabled. You must add these toggles to your full or typical toggle file to change the configuration.

DefaultStalledMethod Tracing

Stalled method tracing

Full: on

Typical: on

DefaultConcurrentInvocationTracing

Concurrent invocation information

Full: on

Typical: off

DefaultRateMetrics

Invocation rate metrics

Full: off

Typical: off

Turning tracer groups on or off

You can refine the gathering of metrics on your systems by turning on or off certain tracer groups.

To turn a tracer group on

1. Locate the toggles-full.pbd or toggles-typical.pbd file (depending on which file type (<appserver>-full.pbl or <appserver>-typical.pbl is in use by AutoProbe or the Java agent). These files are found within the <appserver home>wily/core/config directory or <Introscope_Home>/core/config/systempbd directory.

2. Locate the tracer group to turn on, and uncomment the line by removing the pound sign from the beginning of the line. The directive in the following example is turned on, and will cause the tracing of all HTTP Servlets.

TurnOn: HTTPServletTracing

Note: Any uncommented (turned on) directive for a tracer group causes the tracer group to be used.

To turn a tracer group off

■ Comment the tracer group by placing a pound sign at the beginning of the line, as in the following example:

#TurnOn: HTTPServletTracing



Adding classes to a tracer group

You can turn on tracing for a particular class by adding the class to an existing tracer group. To identify a class as being part of a tracer group, use one of the Identify keywords.

For example, to add the class, com.myCo.ejbentity.myEJB1, to the tracer group, EntityBeanTracing:

IdentifyClassAs: com.myCo.ejbentity.myEJB1 EntityBeanTracing

The identify keywords are:

■ IdentifyInheritedAs

■ IdentifyClassAs

■ IdentifyCorbaAs

EJB subclass tracing

By default, entity and session EJB-related directives add probes only for EJBs that directly and explicitly implement the entity, session, or message-driven EJB interfaces.

Often, an application’s EJBs are subclasses of classes which directly and explicitly implement the entity or session EJB interface. These are not tracked by default by Introscope.

For EJB subclasses to be tracked by Introscope, they must be added to the appropriate tracer group. To do this, add entries that refer to the direct ancestors of the EJB subclasses to be tracked.

From these models, replace <entity.bean.ancestor.class> or <session.bean.ancestor.class> with the fully-qualified class name of the immediate ancestor of the EJBs to be instrumented.

For entity EJBs:

IdentifyInheritedAs: <entity.bean.ancestor.class> EntityBeanTracing

For session EJBs:

IdentifyInheritedAs: <session.bean.ancestor.class> SessionBeanTracing



The examples below are based on this class hierarchy:

mySessionEJB implements javax.ejb.SessionBean

mySessionEJBsubclass1 extends mySessionEJB

mySessionEJBsubclass1a extends mySessionEJBsubclass1

mySessionEJBsubclass1b extends mySessionEJBsubclass1

mySessionEJBsubclass2 extends mySessionEJB

The tracer group, SessionBeanTracing, causes the tracking of mySessionEJB:

The following tracer traces mySessionEJBsubclass1 and mySessionEJBsubclass2.

IdentifyInheritedAs: mySessionEJB SessionBeanTracing

The following tracer traces mySessionEJBsubclass1a and mySessionEJBsubclass1b.

IdentifyInheritedAs: mySessionEJBsubclass1 SessionBeanTracing

Note: This example does not use packages. If your code is in a package, it needs to include the package name with the class name.

EJB 3.0 annotations

The following directive allows you to group any class containing the given class-level annotation into tracer groups. This directive supports EJB 3.0. EJBs conforming to the 3.0 specifications do not explicitly implement any well-known interface, but instead are entirely enabled via annotations. To easily identify EJB 3.0 classes, use this directive:

IdentifyAnnotatedClass <annotation-name> <flag-name>

To use this directive, create a directive class and directive parser class for the new directive. You must then add a matcher class to examine your bytecode to determine if a class contains a given annotation.

Note: This directive does not support method-level annotations.

EJB 2.0 and 3.0 support for the application triage map

Introscope 9.0 supports out-of-the-box tracing of EJB 2.0 and 3.0 session and entity beans, specifically for use in the Workstation application triage map. CA Technologies recommends using this out-of-the-box functionality in test environments only, as this configuration affects the start-up time of the agent.

If deploying this functionality to your production environments, it is best that you configure EJB 2.0 and 3.0 tracers for more specific things, as the out-of-the-box functionality may be too broad.



Use the following directive to instruct ProbeBuilder to flag a class that is inheriting from, or implementing, the superclass or interface:

IdentifyDeepInheritedAs

For EJB 2.0 application triage map support, the following directives have been added the j2ee.pbd file:

IdentifyDeepInheritedAs: javax.ejb.EJBObject EJB2StubTracing

IdentifyDeepInheritedAs: javax.ejb.SessionBean SessionBeanTracing

IdentifyDeepInheritedAs: javax.ejb.EntityBean EntityBeanTracing

IdentifyDeepInheritedAs: javax.ejb.MessageBean MessageBeanTracing

These directives allow the ProbeBuilder to identify the EJB stubs on the client side, and beans on the server side to be used in the application triage map.

For EJB 3.0 application triage map support, the following directive has been added to the j2ee.pbd file:

IdentifyInheritedAnnotatedClassAs

The directive matches all classes that implement interfaces directly, or through a super interface.

In the context of the application triage map, the following additional directive is set in j2ee.pbd:

IdentifyInheritedAnnotatedClassAs: javax.ejb.Remote EJB3StubTracing

EJB naming

You can name called backends, generic frontends, and monitored components that deal with EJBs. The name formatter lets you configure a suitable name for EJB 2.0 and 3.0 client stubs and bean implementations.

Use the EjbNameFormatter classes to define an EJB-related metric name, application triage map application name, or node name using following place holders:

■ For EJB client stubs: {classname}, {interface}, and {method}

■ For EJB beans: {classname}, {bean}, {interface}, and {method}

Using the IntroscopeAgent.profile, PBLs, and PBDs together


The following metric names are used by default:

■ EJB Bean frontend: EJB|{interface}

■ EJB Client stub backend: EJB|{interface}

■ Application triage map owner name for EJB bean: {interface}

■ Application triage map node name for EJB Client Stub: Client {interface}

■ Application triage map node name for EJB Bean: Server {interface}

These are default EJB name formatters. They are used in the j2ee.pbd and appmap-ejb.pbd files. You will use the same name formatters, but different metric names. For example, you could modify existing tracer directives to use a more appropriate name, but keep the same flags:

...

# Default commented out:

#TraceComplexMethodsIfFlagged: EJB2StubTracing EJB2BackendTracer "{interface}"

#Add the EJB application name to backend marker as well as called method

TraceComplexMethodsIfFlagged: EJB2StubTracing EJB2BackendTracer

"MyCustomerBeanApp-{interface}-{method}"

...

SetTracerClassMapping: EJB2BackendTracer

com.wily.introscope.agent.trace.BackendTracer

com.wily.introscope.probebuilder.validate.ResourceNameValidator

SetTracerParameter: EJB2BackendTracer nameformatter

com.wily.introscope.agent.trace.ejb.Ejb2StubNameFormatter

Note: The EJB context tracer is set on setContext() method of EJB 2.0 beans. This is an internal Introscope tracer for the EJB 2.0 bean name formatter, which allows the name formatter to function correctly.

Using the IntroscopeAgent.profile, PBLs, and PBDs together

When the Java agent is first installed, you set an instrumentation level, either full or typical. This setting refers to the ProbeBuilder List (PBL) files default-typical.pbl and default-full.pbl (see Default PBL files (see page 86) for more information).

Applying ProbeBuilder Directives

The way in which you apply PBDs depends on the method you choose to use. CA Technologies recommends you use JVM AutoProbe to implement your PBDs. You can also use the ProbeBuilder Wizard or the command line ProbeBuilder to implement your PBDs.



Using JVM AutoProbe

When you are ready to implement a PBD file, add it to the hotdeploy directory. AutoProbe looks for PBD files in the directory that contains the IntroscopeAgent.profile file (by default, this is the <Agent_Home>/wily/core/config directory), and the <Agent_Home>/wily/core/config/hotdeploy directory. AutoProbe resolves file names relative to these directories. If you have moved the location of your wily directory, be sure to map the file path to the correct directory.

To implement PBDs using AutoProbe

1. Save modified standard PBD or PBLs to the <Agent_Home>/wily/core/config directory.

2. Copy custom PBDs into the <Agent_Home>/wily/core/config/hotdeploy directory. Any PBDs added to this directory will be implemented without having to update or modify the introscope.autoprobe.directivesFile property in the IntroscopeAgent.profile.

Note: If you have enabled dynamic instrumentation, the PBDs in the hotdeploy directory are picked up live from the folder. No reboot is required. For more information about dynamic instrumentation, see Dynamic ProbeBuilding (see page 73).



Using the ProbeBuilder Wizard or command-line ProbeBuilder

When you are ready to implement a PBD file, add it to the hotdeploy directory. The Command-line ProbeBuilder looks for any custom directive files in the same directory where ProbeBuilder is run from, and the <Agent_Home>/wily/core/config/hotdeploy directory. The Command-line ProbeBuilder resolves file names relative to these directories.

The steps to implement ProbeBuilder Directives using the ProbeBuilder Wizard or command-line ProbeBuilder are the same as using JVM AutoProbe. See Using JVM AutoProbe (see page 100) for more information.

Instrumenting with new and changed PBDs

For new or changed directives to take effect, your applications must be instrumented using the latest PBDs. This process varies depending on the ProbeBuilding method you use.



JVM 1.5 systems using JVM AutoProbe via -javaagent

You can configure dynamic instrumentation, allowing changed PBDs to take effect without application or Java Agent restart. This enables you to perform PBD corrections, or perform triage-driven instrumentation without interrupting application service. For more information see Dynamic ProbeBuilding (see page 73).

Pre-JVM 1.5 systems or installations using –Xbootclasspath

New and changed ProbeBuilder Directive files or ProbeBuilder List files take effect the next time the application server loads the application classes.

If your managed applications are not running when you add or change directives, when you next start the applications, they will be instrumented using the updated directives.

If your managed applications are running, it is necessary to load, or reload, the managed application classes.

How you cause the classes to reload depends upon the application server you use. For instance, on SAP NetWeaver 6.40, a redeploy is sufficient. Most application servers require a restart.

Using the ProbeBuilder Wizard

To use the ProbeBuilder Wizard

1. The Custom Directives screen will list the PBD files you placed in the hotdeploy directory described in Using the ProbeBuilder Wizard or command-line ProbeBuilder (see page 100).

2. Select the custom directives files to use.

Using the command-line ProbeBuilder

Important! CA Technologies recommends using the command-line ProbeBuilder as your last option for Introscope-enabling your latest PBDs.

To use the command-line ProbeBuilder

1. Stop your managed application.

2. Run the command-line ProbeBuilder or the ProbeBuilder Wizard, supplying the custom PBD and PBL files in the command line.

3. Configure the application to use the new files.

4. Start the managed application.

5. If they are not already running, start the Enterprise Manager and the Workstation.

Creating custom tracers



You can further refine your metric collection by creating custom PBD files. Creating custom directives, by creating tracers to track application specific measurements, require the use of specific syntax and keywords. To write custom tracers, you must define:

■ The directive type (indicating generically how many class(es) or method(s) to trace)

■ The specific class(es) or method(s) to trace

■ The type of information to trace in the class(es) or method(s) (for example, a time, a rate, or a count)

■ The fully-qualified metric name (including the resource path) under which to present this information

Custom PBDs are stored in the <Agent_Home>/wily/core/config/hotdeploy directory. Any PBDs added to this directory will be implemented without having to update or modify the introscope.autoprobe.directivesFile property in the IntroscopeAgent.profile. If you have enabled dynamic instrumentation, the PBDs in the hotdeploy directory are picked up live from the folder. No reboot is required. For more information about dynamic instrumentation, see Dynamic ProbeBuilding (see page 73).

Once a custom PBD is created, Introscope treats it as if it is an out-of-the-box PBD. You can set alerts on the metrics created, save them to SmartStor, or use them in the creation of custom dashboards in the Introscope Workstation.

Note: Be sure to choose methods to trace carefully, as more methods traced means more overhead.

Using a custom BlamePointTracer tracer for common metrics

A BlamePointTracer is the most commonly used tracer. This tracer generates five separate metrics for associated methods or classes:

■ Average Response Time (ms)

■ Concurrent Invocations

■ Errors Per Interval

■ Responses Per Interval

■ Stall Count



The following is an example of a BlamePointTracer. A BlamePointTracer has been set for a method called search in class petshop.catalog.Catalog. PetShop|Catalog|search is the name of the node under which the BlamePoint metrics will be displayed in the Introscope Investigator.

TraceOneMethodOfClass: petshop.catalog.Catalog search BlamePointTracer

"PetShop|Catalog|search"

Directive names and arguments used in tracer syntax

In addition to simple keywords that associate tracers into groups or enable/disable groups, PBD files contain tracer definitions. For Introscope to recognize and process tracers, you must use a specific syntax when constructing custom tracers. A tracer is composed of a directive and information about the method or class to trace, in the following format:

<directive>: [arguments]

where [arguments] is a list, and is directive-specific.

Note: Depending on the directive used, only a subset of these parameters are required.

<directive>

The most common directives to use are the following trace directives:

TraceOneMethodOfClass

Traces a specified method in the specified class.

TraceAllMethodsOfClass

Traces all methods in the specified class.

TraceOneMethodIfInherits

Traces one method in all direct subclasses or direct interface implementations of the specified class or interface.

TraceAllMethodsIfInherits

Traces all methods in all direct subclasses or direct interface implementations of the specified class or interface.

TraceOneMethodIfFlagged

Traces one method if the specified class is included in a tracer group that has been enabled with the TurnOn keyword.



TraceAllMethodsIfFlagged

Traces all methods if the specified class is included in a tracer group that has been enabled with the TurnOn keyword.

Note: Only concrete, implemented methods can be traced and report metric data while running. An abstract method specified in a custom tracer results in no metric data being reported.

The expected syntax for trace directives usually consist of the following arguments:

<Tracer-Group>

The group to which the tracer is associated.

<class>

A fully qualified class or interface name to trace. Fully qualified classes include the full assembly name of the class as well as the name, for example:

[MyAssembly]com.mycompany.myassembly.MyClass

The assembly name must be enclosed in [] brackets.

<method>

The method name (for example, MyMethod)

OR

the full method signature with return type and parameters (for example, myMethod;[mscorlib]System.Void([mscorlib] System.Int32). For more information on method signatures, see Signature differentiation (see page 108).)

<Tracer-name>

Specifies the tracer type to be used. For example, BlamePointTracer. See the Tracer name table below for descriptions of tracer names.

<metric-name>

Controls how the collected data is displayed in the Introscope Workstation.

The following examples describe three ways to specify the name and location of a metric at different levels of the metrics tree.

metric-name—the metric appears immediately inside the agent node.

resource:metric-name—the metric appears inside one resource (folder) below the agent node.

resource|sub-resource|sub-sub-resource:metric-name—the metric appears more than one resource (folder) level deep below the agent node. Use pipe characters (|) to separate the resources.



Commonly used tracer names and examples

The following list describes the most commonly used tracer names and what they trace.

BlamePointTracer

Provides a standard set of metrics including average response time, per interval counts, concurrency, stalls, and errors for a blamed component.

ConcurrentInvocationCounter

Reports the number of times a method has started but not yet finished. The result is reported under the metric name specified in the tracer, <metric-name>, in the Investigator tree. An example use of this tracer would be counting the number of simultaneous database queries.

DumpStackTraceTracer

Dumps a stack trace to the instrumented application's standard error for methods to which it is applied. The exception stack trace thrown by the Dump Stack Tracer is not a true exception—it is a mechanism for printing the method stack trace.

This feature is useful for determining callpaths to a method.

Important! This feature imposes heavy system overhead. It is strongly recommended that this tracer only be used in a diagnostic context where a sharp increase in overhead is acceptable.

MethodCPUTimer

Average CPU time (in milliseconds) used during method execution and reports it under <metricname> in the metrics tree.

Note: This tracer requires a platform monitor on the supported platform.

MethodTimer

Average method execution time in milliseconds and reports it under the metric name specified in the tracer, <metric-name>, in the metrics tree.

PerIntervalCounter

Number of invocations per interval. This interval will change based on the view period of the consumer of the data (for example, the View pane in the Investigator). It is reported under the metric name specified in the tracer, <metric-name>, in the Investigator tree.



Using quotes in custom tracer definitions

Custom tracers can contain metric names with spaces in them. When using spaces in your custom metric names, CA Technologies recommends putting quotes ("") around all metric names.

Important! Do not place quotes around class names. This causes the custom tracers to malfunction. For example:

Correct

IdentifyClassAs: MyClass MyTracers

Incorrect

IdentifyClassAs: "MyClass" MyTracers

If you create a metric name that contains a class name, you must use quotes around the whole metric name. Metric names are allowed to have spaces, and all spaces in metric names must be contained within quotes. For example, the metric name "{classname}|Test One Node" should be represented as follows:

Correct

TraceOneMethodIfFlagged: MyTracers AMethod BlamePointTracer "{classname}|Test One

Node"

Incorrect

TraceOneMethodIfFlagged: MyTracers AMethod BlamePointTracer {classname}|Test One

Node

Important! Introscope will not monitor classes having invalid class file names. For example, in the class file name:

org/jboss/seam/example/seambay/AuctionImage$JaxbAccessorM_getData_setData_[B:

The _[B: causes the class file name to be invalid. You cannot use an open square brackets ([) as part of the Java class file name. When Introscope encounters such classes having invalid class names, it fails to instrument them and reports them as an error message in the agent logs.

The following sections are examples of method tracers. In the following example, quotes ("") are used around the metric names. CA Technologies highly recommends putting quotes around all metric names when you create custom metric names.

Average tracer example

This tracer tracks the average execution time of the given method in milliseconds.

TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search BlamedMethodTimer

"Petstore|Catalog|search:Average Method Invocation Time (ms)"



Rate tracer example

This tracer counts the number of times the method is called per second, and reports this rate under the specified metric name.

TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search

BlamedMethodRateTracer "Petstore|Catalog|search:Method Invocations Per Second"

Per interval counter tracer example

This method tracer counts the number of times the method is called per interval, and reports the per interval count under the specified metric name.

TraceOneMethodOfClass: com.sun.petstore.catalog.Catalog search PerIntervalCounter

"Petstore|Catalog|search:Method Invocations Per Interval"

The interval is determined by the monitoring logic in the Enterprise Manager, such as the Graph frequency.

The preview pane in the Investigator defaults to 15 second intervals.

Counter tracer example

This tracer counts the total number of times the method is called.

TraceOneMethodOfClass: com.sun.petstore.cart.ShoppingCart placeOrder

BlamedMethodTraceIncrementor "Petstore|ShoppingCart|placeOrder:Total Order Count"

Combined counter tracers example

These tracers combine incrementor and decrementor Tracers to keep a running count.

TraceOneMethodOfClass: com.sun.petstore.account.LoginEJB login

MethodTraceIncrementor "Petstore|Account:Logged In Users"

TraceOneMethodOfClass: com.sun.petstore.account.LogoutEJB logout

MethodTraceDecrementor "Petstore|Account:Logged In Users"

Advanced single-metric tracers

Directives and tracers track methods, classes, and sets of classes. A single-metric tracer reports a specific metric for a specific method, which is the smallest unit that Introscope can track. Single-metric tracers can be created in several ways: through the method signature, by substituting keywords, or by manipulating the metric name parameters.



Signature differentiation

Tracers can be applied to a method based on the method signature.

To trace a single instance of a method with a specific signature, append the signature to the method name (including return type) specified using the internal method descriptor format.

For example, myMethod(Ljava/lang/String;)V traces the instance of the method with a string argument and void return type.

For complete information about this format, see the Sun Java Virtual Machine Specification.

Metric name keyword-based substitution

Keyword-based substitution allows runtime substitution of values into the metric name.

The parameters in the metric name in the tracer are substituted at runtime for the actual values into the metric name. This feature can be used with any directive.

{method}

Name of the method being traced

{classname}

Runtime class name of the class being traced

{packagename}

Runtime package name of the class being traced

{packageandclassname}

Runtime package and class name of the class being traced

Note: If Introscope processes a class which does not have a package, it will replace {packagename} with the string "<Unnamed Package>".

Keyword-based substitution: Example 1

If the metric name for a tracer in the pbd file is:

"{packagename}|{classname}|{method}:Response Time (ms)"

and the tracer is applied to method myMethod with a runtime class of myClass that is in package myPackage, the resulting metric name would be:

"myPackage|myClass|myMethod:Response Time (ms)"

http://java.sun.com/docs/books/vmspec/2nd-edition/html/VMSpecTOC.doc.html





Keyword-based substitution: Example 2

If a tracer with a metric name in the .pbd file of

"{packageandclassname}|{method}:Response Time (ms)"

was applied to the same method, the resulting metric name would be

"myPackage.myClass|myMethod:Response Time(ms)"

Note: The . between the package and class instead of the | in the first example.

Metric-name-based parameters

You can create a single-method tracer that creates a metric name based on parameters passed to a method using the TraceOneMethodWithParametersOfClass keyword, using this format:

TraceOneMethodWithParametersOfClass: <class-name> <method> <tracer-name> <metric-name>

Parameters can be used in the metric name. This is accomplished by substituting the value of parameters for placeholder strings in the metric name. The placeholder strings to use are "{#}" where # is the index of the parameter to substitute. The indices start counting at zero. Any number of parameter substitutions can be used in any order. All parameters are converted to strings before substitution into the metric name. Object parameters other than strings should be used with caution because they are converted using the toString() method.

Important! If you are unclear about what string the parameter will be converted to, do not use it in the metric name.



Metric-name-based example

A Web site uses a class named order, with a method named process. The method has parameters for different kinds of orders, either book or music.

You can create a tracer like this:

TraceOneMethodWithParametersOfClass: order process(LJava/lang/string;)V

MethodTimer "Order|{0}Order:Average Response Time (ms)"

This tracer produces metrics like these:

Order

BookOrder


MusicOrder


You can also use the TraceOneMethodWithParametersIfInherits keyword.

Skip directives

Certain packages, classes, or methods can be skipped by AutoProbe or ProbeBuilder by using skip directives. By default, the Java agent and fundamental Java classes and packages are skipped by AutoProbe or ProbeBuilder.

Counting object instances

The InstanceCounts tracer group counts the number of instances of the particular object types associated with it (for information on associating object types with the InstanceCounts tracer group using the standard IdentifyClassAs and IdentifyInheritedAs directives, see Adding classes to a tracer group (see page 96)). Any instances explicitly allocated in your code will be counted. Subtypes will also be counted. Objects created through different mechanisms, such as deserialization or cloning, might not be counted. Tracing using this tracer group could potentially incur incremental performance (and memory) impact, depending entirely on the number of instances counted.

Note: CA Technologies testing has shown that in practice, the number of instances has to be quite large for an impact to be noticeable.



Turning on InstrumentPoint directives

There are two types of directives identified by the keyword, InstrumentPoint: those that trace exceptions, and one that causes agent initialization when the application starts up (instead of when the first Probe is run).

Exceptions

The following directives are used to turn on tracing of exceptions either where thrown or caught. They can cause performance degradation so they are not turned on by default. To turn either of these on, uncomment the appropriate line:

#InstrumentPoint: ThrowException

#InstrumentPoint: CatchException

Agent initialization

The agent initialization instrument point directive does not cause additional overhead and is turned on by default in both full and typical PBD sets.

InstrumentPoint: AgentInitialization

If multiple ProbeBuilder Directive files are used, any settings (such as tracer groups, Skips, InstrumentPoints, Custom Method Tracers) turned on in any file take effect.

Combining custom tracers

You can use multiple tracers that affect the same metric, in effect combining them. This is most commonly used with incrementors and decrementors.

This example creates a metric named Total Purchases. With a class cart and methods buyBook and buyCD, create the following tracers:

TraceOneMethodOfClass cart buyBook PerIntervalCounter "Total Purchases"

TraceOneMethodOfClass cart buyCD PerIntervalCounter "Total Purchases"

This increments the metric Total Purchases when someone buys a piece of merchandise.

Instrumenting and inheritance

Introscope does not automatically instrument classes in the deeper levels of a class hierarchy in pre-1.5 JVMs.

When subclasses of a probed class more than one level deep are loaded, the new and overridden methods are not automatically instrumented. Likewise, classes that do not explicitly name a probed interface as being implemented, even though they implement the interface indirectly, will not be instrumented either.



For example, assume a class hierarchy in which ClassB extends ClassA, and ClassC extends ClassB, like so:

Interface/ClassA

ClassB

ClassC

When you instrument ClassA, ClassB is also instrumented because it explicitly extends ClassA. However, Introscope does not instrument ClassC because ClassC does not explicitly extend ClassA. To instrument ClassC you must explicitly identify ClassC.

In pre-1.5 Java environments, to ensure that subclasses are instrumented, follow the instructions in EJB subclass tracing (see page 96).

If you run under JVM 1.5, you can configure Introscope to instrument multiple levels of subclasses of a probed class. For instructions, see ProbeBuilding class hierarchies (JVM 1.5). (see page 76)

Java 1.5 annotations

Introscope 8.0 and higher allows the use of Java 1.6 annotations introduced in Java 1.5 when creating custom metrics. For more information on Java annotations, see the following articles:

■ http://java.sun.com/docs/books/tutorial/java/javaOO/annotations.html

■ http://www.developer.com/java/other/article.php/3556176

Use IdentifyAnnotatedClassAs to place the class in a tracer group, then use TraceAllMethodsIfFlagged directives to instrument the methods in the class. For example:

SetFlag: AnnotationTracing TurnOn: AnnotationTracing

IdentifyAnnotatedClassAs: com.test.MyAnnotation AnnotationTracing

TraceAllMethodsIfFlagged: AnnotationTracing BlamePointTracer

"Target|MyTarget|{classname}"

In the example, com.test.MyAnnotation is the annotation name. When creating your own annotations, use a term in your code. Classes containing the annotation name are identified.

http://java.sun.com/docs/books/tutorial/java/javaOO/annotations.html

http://www.developer.com/java/other/article.php/3556176

http://www.developer.com/java/other/article.php/3556176

Using Blame Tracers to mark blame points



The Blame Technology for CA Introscope works in a managed Java application to enable you to view metrics at the application tiers: the frontends and backends of your application. This capability, referred to as boundary blame, allows users to triage problems to an application frontend or backend. This information is also used in the Workstation application triage map to mark the edges of your applications.

For information about how CA Introscope determines frontends and backends, and about options for configuring URL Groups to control how metrics for frontends are aggregated, see Configuring Boundary Blame (see page 153).

The following sections describe how you can use tracers to explicitly mark the frontends and backends in your application.

Blame Tracers

Introscope provides tracers for capturing front and backend metrics: FrontendMarker and BackendMarker. These tracers explicitly mark a frontend and backend, respectively.

You can use FrontendMarker and BackendMarker to instrument your own code, for instance code that accesses a backend, to cause Introscope to capture and present metrics for custom components in the Investigator tree.

If no components are instrumented with the FrontendMarker tracer (or its subclasses HttpServletTracer and PageInfoTracer), no frontend metrics are generated and no component will be marked as a frontend for transactions.

When more than one component within a transaction is instrumented with the FrontendMarker tracer (or its subclasses), only the first designated component will generate Frontend metrics.

Note: When using frontend tracers, the name of the application given in the frontend tracer must match the name given for the application triage map tracers, keeping in mind that both are case-sensitive. For example, if you name the frontend tracer AppOne and the application triage map tracer refers to this tracer as APPONE, information about AppOne will not be displayed correctly in the Workstation application triage map.

To prevent specific classes from being marked as frontend, the PBD parameter is.frontend.unless can be specified. For information on the PBD directive is.frontend.unless, see the Custom FrontendMarker directive (see page 114).

If no BackendMarker is configured, Introscope will infer a backend—any component that opens a client socket will be a default backend if none is explicitly marked.



It is useful to use BackendMarker:

■ to assign a desired name to an item that Introscope detects as a backend.

■ to mark custom Java sockets that Introscope does not instrument.

■ for native sockets that are called through the Java Native Interface (JNI), to identify a Java/JNI bridging method as the backend.

FrontendMarker and BackendMarker are instances of BlamePointTracer which provides metrics such as average response time, per interval counts, concurrency, stalls, and errors for a blamed component. A BlamePointTracer can be applied to middle components for a more granular Blame Stack.

High agent CPU overhead from deep nested frontend transactions

Servlets are configured by Introscope to be seen as frontends. A typical transaction starts with a servlet, which may call an EJB, which calls a back-end. It’s possible for servlets to call other servlets in a nested way, which Introscope sees as nested frontends. In most cases, this does not add to agent CPU overhead.

However, deep transactions having nested frontend levels (for example 40 levels deep) may result in high CPU overhead. For example, if a servlet repeatedly calls itself in a transaction (continuous recurring calls) or calls multiple other servlets, you may see an increase in agent CPU overhead. If the overhead is unacceptable, contact CA Support.

Custom FrontendMarker directive

Introscope 9.0 introduces the PBD parameter is.frontend.unless. This parameter enables some classes instrumented by the FrontendMarker (or its subclasses, such as HttpServletTracer) to not be marked as a frontend component. The parameter should be set as a comma-separated list of absolute class names. This may be useful when the initial component is a generic 'dispatcher' which forwards the request to a more specific component designed to handle the type of request received. The second component would therefore be a better frontend marker. The default is an empty list. PBD parameters are not dynamic, so a restart of the instrumented application server is required if the value of this parameter is changed.

Important! Class names should only be separated by a comma, not spaces. Use of spaces will invalidate the SetTracerParameter directive.

Any classes designated in the parameter list that are instrumented by the tracer to which this parameter is applied will not be designated as frontends and will not generate metrics under the Frontends node in the Introscope Investigator.



For example, to prevent the classes NotAFrontend and AnotherNonFrontend from being treated as frontends in the package com.ABCCorp, that are instrumented with a FrontendMarker named MyFrontendTracer, you would use the following PDB directive:

SetTracerParameter: MyFrontendTracer is.frontend.unless

com.ABCCorp.NotAFrontend,com.ABCCorp.AnotherNonFrontend

Blame Tracers in standard PBDs

Two of the standard PBDs provided with Introscope, j2ee.pbd and sqlagent.pbd, implement Boundary Blame Tracing.

■ HttpServletTracer in j2ee.pbd is an instance of FrontendMarker.

■ SQLBackendTracer in sqlagent.pbd is an instance of BackendMarker.

The following Blame Tracers used in previous versions of Introscope still exist, but are not typically used in Introscope PBDs:

■ BlamedMethodTimer

■ BlamedMethodRateTracer

■ BlamedMethodTraceIncrementor

■ BlamedMethodTraceDecrementor

Boundary Blame and Oracle backends

In the current version of Introscope, Oracle databases are not detected based on the socket connection—SQL Agent must be available for Introscope to automatically detect Oracle backends.

To enable Introscope to detect Oracle backends in the absence of SQL Agent, make the following modification to oraclejdbc.pbd:

In this portion of oraclejdbc.pbd:

#Socket data from the Oracle driver reports too many metrics SkipPackagePrefixForFlag: oracle.jdbc. SocketTracing SkipPackagePrefixForFlag: oracle.net. SocketTracing

comment out the skips, as shown below:

#Socket data from the Oracle driver reports too many metrics #SkipPackagePrefixForFlag: oracle.jdbc. SocketTracing #SkipPackagePrefixForFlag: oracle.net. SocketTracing

For more information, see Disabling Database Name Formatting in 7.1 (KB 1240).

http://support.wilytech.com/cgi-bin/wilytech.cfg/php/enduser/std_adp.php?p_faqid=1240

http://support.wilytech.com/cgi-bin/wilytech.cfg/php/enduser/std_adp.php?p_faqid=1240


Chapter 6: Java Agent Naming

This section has information about agent naming, related environmental and deployment considerations, and options for automatically naming your agents.


Understanding the Java Agent name (see page 117) Agent naming considerations for clustered applications (see page 120) Specifying an agent name using a Java system property (see page 120) Specifying an agent name using a system property key (see page 121) Obtaining an agent name from the application server (see page 121) Automatic agent naming (see page 122) Enabling cloned agent naming in clustered environments (see page 124) Application triage map and the agent name (see page 125)

Understanding the Java Agent name

Each Java Agent running in your Introscope environment has a name, whether you assigned one explicitly, configured a method of automatically assigning a name, or simply started an instrumented application that the Java Agent monitors. The Java Agent name is central to many views in the Introscope Workstation and Investigator, and it is key to the process of associating monitoring logic with target applications.



When an agent report metrics to an Enterprise Manager, a node is created for that agent in the Investigator tree. When you configure management logic in the Workstation—for instance, Dashboards, Alerts, and Actions—the agent name is a component in the regular expressions that identify the applications to which the management logic applies. The Investigator tree below shows agents named domain1//Adminserver, running on host qw32vtest01 under the WebLogic process.

How the agent determines its name

The Java Agent uses the following sequence to determine a name. If it determines a name using the first method, it accepts that name and connects to the Enterprise Manager. If it doesn’t determine a name using the first method, it tries the second method, and so on. If it doesn’t determine a name using any method, it calls itself "UnnamedAgent."

Method 1: Agent name specified in a Java system property

The agent name is defined using a Java system property on the command line. Using this method will override any other agent naming method. See Specifying an agent name using a Java system property (see page 120).

Method 2: Agent name specified in a system property key in the IntroscopeAgent.profile

The agent name is obtained from a Java system property specified in a property in the IntroscopeAgent.profile. See Specifying an agent name using a system property key (see page 121).



Method 3: Agent name obtained automatically from the application server

If you use certain versions of WebLogic or WebSphere, the agent name can be automatically obtained from the application server using automatic agent naming functionality. You can configure a time delay, to give the agent as much time as necessary to determine its name before connecting to the Enterprise Manager. See Obtaining an agent name from the application server (see page 121).

Method 4: Agent name specified explicitly in the agent profile

The agent name is defined in the IntroscopeAgent.profile, in the property introscope.agent.agentName. This was the standard method for naming agents in early Introscope versions. Use this option if you already have an agent profile for every application.

Method 5: Agent name determined to be "Unknown agent"

If the agent is unable to determine a name using one of the methods listed above, then the agent names itself "UnnamedAgent".

How Introscope resolves agent naming conflicts

The fully qualified agent name—comprised of host name, process name and agent name—is typically unique to each agent in an Introscope environment. Agents with the same agent name usually have a unique fully-qualified agent name because their host name and process names are likely to be different. Multiple agents will have the same fully-qualified agent name only if they reside on the same host, monitor the same process, and have the same agent name.

If an agent tries to connect to an Enterprise Manager to which an agent with the same fully-qualified agent name is already connected, the Enterprise Manager appends a unique identifier to the name of the newly connecting agent. The identifier consists of a percent (%) character and a digit. This mechanism ensures that multiple agents that connect using the same fully-qualified name can be uniquely identified for the duration of the connection. The Enterprise Manager renames the first duplicate agent to connect by appending "%1" to its agent name.

For instance, assume that two agents with the fully qualified agent name:

hostPA|processNIM|PodAgent

connect to the Enterprise Manager, one after the other. The Enterprise Manager renames the second agent:

PodAgent%1

If other agents with the same fully qualified name connect, they are renamed, in succession, PodAgent%2, PodAgent%3, PodAgent%4, and so on, where the digit following the percent character is the next number in sequence.

Agent naming considerations for clustered applications


When a renamed agent disconnects, the suffix it was assigned can be re-used. For example, if PodAgent%1 disconnects while PodAgent remains connected, the next agent with the fully qualified name hostPA|processNIM|PodAgent to connect will be renamed PodAgent%1.

Reuse of the suffix identifier makes it possible that the Enterprise Manager might assign the same suffix to a particular agent’s name from connection to connection. However, on subsequent connections, a given agent could just as well be renamed differently. Having an agent’s name vary from connection to connection is problematic when querying historical data—it is preferable to configure a naming strategy that avoids the Enterprise Manager renaming agents.

Agent naming considerations for clustered applications

If you run multiple instances of the same application, Introscope attempts to resolve identical agent names, including custom metric agents, by appending the agent name with a character and a random number. CA Technologies recommends, however, that you tell Introscope how to resolve the naming.

The options for resolving identical agent naming are:

■ Tell Introscope that the agents in question are cloned agents by enabling cloned agent naming (described in Enabling cloned agent naming in clustered environments (see page 124).)

■ Define unique agent names yourself and make separate agent profiles for each agent (described in Configuring unique names for application instances (see page 125).)

■ Let Introscope uniquely name each agent using its own naming scheme (described in How Introscope resolves agent naming conflicts (see page 119).)

Specifying an agent name using a Java system property

To specify an agent name using Java system property

■ On the Java command line, supply the desired name using this property:

-Dcom.wily.introscope.agent.agentName=

Specifying an agent name using a system property key


Specifying an agent name using a system property key

This method is the second the agent uses to look for its name. Use this method if you want the agent to be named from the value of an existing Java system property in your deployment.

To specify an agent name using the System Property Key


2. Under the Agent Name section, specify the Java system property that will provide the agent name in this property:

introscope.agent.agentNameSystemPropertyKey

Note: If the Java system property specified here does not exist, this property will be ignored.


Obtaining an agent name from the application server

You can configure the agent to extract the application server instance name automatically from the application server, and use that information to name itself. This eliminates the need to configure individual agent names in a separate agent profile file. The agent can also rename itself if there are changes in the application server environment. This enables you to deploy an agent profile across a large number of environments that might consist of a mix of application server platforms.

Application servers that support agent naming

Automatic agent Naming is supported when you use Introscope with these supported application server versions:

■ WebLogic 9.x

■ WebSphere 6.1.x distributed

■ WebLogic 10.0

■ WebSphere 7.0.x distributed

■ WebLogic 10.3

The name of the application server displayed in the Introscope Workstation is determined by a Java J2EE API. This sometimes causes the name of the application servers to display differently in the Workstation because all application servers implement the API differently. The names of multiple application servers may be formatted differently in the Workstation, and even the same application server name may be formatted differently from release to release.

Automatic agent naming



When automatic agent naming is enabled, the agent starts and looks for name information from the application server. The agent waits until an agent name is obtained before attempting to connect to the Enterprise Manager.

When the agent locates naming information, Introscope edits the information to make the agent name compliant with agent naming rules.

Agent names on supported application servers are comprised of several pieces of information, which differ according to application server.

■ For WebLogic, the agent name is comprised of:

Domain (data center) + cluster + instance (of WLS)

■ For WebSphere, the agent name is comprised of:

cell (domain) + process (instance of WAS)

When information is obtained, segments are separated by forward slashes—for example:

medrec/MyCluster/MedRecServer

Any forward slashes in the segment name are converted to underscores. For example, if a Domain is named Petstore/West, it will be converted to Petstore_West.

Note: When constructing the agent name that appears in Introscope, Introscope edits the information to make the agent name compliant with the following Introscope agent naming rules:

■ characters such as pipes, colons, or percentage signs are replaced by underscores

■ names that begin with any character other than a letter will have the letter "A" prepended to them

■ empty names are replaced by "UnnamedAgent" (so as to be distinguishable from the "UnknownAgent" condition)

To enable automatic agent naming

1. In the IntroscopeAgent.profile, set introscope.agent.agentAutoNamingEnabled to true.

2. Make these application server-specific changes:

■ For WebLogic, create an Introscope Startup Class. See Configuring a startup class for WebLogic (see page 42).

■ For WebSphere, create an Introscope Custom Service. See Configuring a custom service in WebSphere (see page 51).

■ For JBoss, create an XML file. See Configuring JBoss (see page 41).



Automatic agent naming and renamed agents

Using automatic agent naming, the agent always tries to obtain the most current application-server-specific agent name. The agent periodically checks for a new name.

If a change to the application server configuration results in an agent name change, the agent automatically renames itself. In the Investigator tree, the agent appears to disconnect. The disconnected agent remains in the Investigator tree, and unmounts automatically after the unmount time period has elapsed, or can be unmounted manually.

The renamed agent reconnects to the Enterprise Manager and appears in the Investigator tree. The agent logs these changes.

See Advanced automatic agent naming options (see page 123), for information on configuring automatic agent naming properties for Enterprise Manager connection delay, and rename checking interval time.

Advanced automatic agent naming options

There are several properties you can change to control automatic agent naming for your environment.

Initial Enterprise Manager Connection Delay

When using the automatic agent naming feature, the agent waits up to a configurable amount of time before connecting to the Enterprise Manager while trying to find agent name information. The default delay is 120 seconds.

To change the delay value


2. Under the Agent Name section, configure the desired delay in the property introscope.agent.agentAutoNamingMaximumConnectionDelayInSeconds.


Enabling cloned agent naming in clustered environments


Agent Rename Check Interval

When using the automatic agent naming feature, the agent periodically checks to see if the naming information from the application server has changed. The default interval is ten minutes.

To change this interval


2. Under the Agent Name section, configure the desired interval in the introscope.agent.agentAutoRenamingIntervalInMinutes property.


Turning Off Agent Log File Automatic Naming

By default, when the agent name is found automatically, either by information provided by a Java system property or application server, the log files associated with that agent are named automatically using that same information. However, you can turn off this automatic log naming, and continue to use the agent log name specified in the IntroscopeAgent.profile.

To turn off agent log file automatic naming


2. Set the property, introscope.agent.disableLogFileAutoNaming, to a value of true.



Enabling cloned agent naming in clustered environments

If two agents exist with the same name monitoring the same host and process and are not uniquely named by a user, the name is appended with a number. Cloned agent naming enables you to correlate an agent with a particular application instance in a clustered application.

You are running cloned agents if you:

■ are running agents that share a host, process, or Java Agent name with one or more other agents, or

■ are running two or more agents that are using the same agent profile.

Application triage map and the agent name


To enable cloned agent naming

1. Stop your managed application and the Java Agent.

2. Open the IntroscopeAgent.profile and set the following property to true:

introscope.agent.clonedAgent=true


4. Restart your managed application and the Java Agent.

Cloned agent naming scenario

With the Java Agent cloning property turned on, if you have four Java Agents, all named AgentX, the Enterprise Manager names the agents AgentX-1, AgentX-2, AgentX-3 and AgentX-4. If AgentX-1 disconnects and then reconnects, it will still use AgentX-1 as its name. With this naming, you will never have more Java Agent names in the database than the number of Java Agents originally cloned.

Configuring unique names for application instances

If you monitor multiple instances of an application on the same machine, you can configure unique agent names explicitly.

To configure unique agent names

1. Create a separate agent profile for each application.

2. Uniquely name each agent in the agent profile.

3. Specify which agent profile each application should use.

Application triage map and the agent name

The application triage map in the Workstation uses the agent name as part of several unique identifiers when defining the front- and backends of applications, as well as for storing information about application components in Introscope databases. If agent names change some aspects of the application triage map may also change. For example, during initial registration of an agent, the Enterprise Manager may assign a duplicated agent name a unique name by appending %<sequence number> to the agent name (e.g. MyAgent%1). If parts of the application triage map are relying on the duplicated agent name for information, aspects of the map could change.

While this does not affect the proper functioning of either the agent or the Enterprise Manager, it may reduce the overall capacity of your system; as such, CA Technologies recommends users be aware of the naming conventions employed in their Introscope environments. Users may want to designate specific agent names to sidestep this issue.


Chapter 7: Java Agent Monitoring and Logging

While Introscope monitors your applications, Introscope can also monitor the health and activity of the Java agent itself. This section contains information on monitoring the health of the agent, as well as logging options for the Java agent.


Configuring agent connection metrics (see page 127) Socket metrics (see page 128) Configuring logging options (see page 131) Managing ProbeBuilder Logs (see page 135)

Configuring agent connection metrics

By default, Introscope generates metrics on the connection status of agents connected to an Enterprise Manager, which you can monitor. You can monitor agent connection metrics to determine the current state of the connection between an agent and the Enterprise Manager.

Agent connection metrics appear in the Workstation Investigator under the Enterprise Manager process (the custom metric host):

Custom Metric Host (Virtual) \ Custom Metric Process(Virtual) \ Custom Metric Agent

(Virtual) (*SuperDomain*) \ Agents \ <HostName> \ <Agent Process Name> \ <Agent Name>

\ ConnectionStatus

Connection metrics have these values:

■ 0—No data about the agent is available

■ 1—agent is connected

■ 2—agent is slow to report

■ 3—agent is disconnected

An agent disconnecting also generates a "What’s Interesting" event. As with other events, users can query for agent disconnects using the historical query interface. Agent disconnect events are part of the data used in assessing application health in the Overview tab in the Workstation Console.

Socket metrics


Once an agent disconnects from the Enterprise Manager, Introscope continues to generate disconnected state metrics until the agent is timed out. When an agent times out, no additional connection metrics are generated or reported to the Enterprise Manager.

Follow these steps:

1. On the computer that the Enterprise Manager is installed on, open the IntroscopeEnterpriseManager.properties file located in the <Introscope_Home>/config directory.

2. Modify this property:

introscope.enterprisemanager.agentconnection.metrics.agentTimeoutInMinutes

The time increment is in minutes.

3. Save the IntroscopeEnterpriseManager.properties

Note: For information about Enterprise Manager properties, see the CA APM Configuration and Administration Guide.

Socket metrics

Socket and Secure Sockets Layer (SSL) metric collection is enabled by default in the agent.

Note: JVMs using the Agent.NoRedef.jar do not have socket metrics reported. See AutoProbe for WebSphere 6.1 (see page 47) for more information.

Restricting socket and SSL metric collection

Socket and SSL metric collection is enabled by default, but you can restrict some metric collection to target more relevant information or scale back on overhead.

To restrict socket and SSL metric collection


2. In the Agent I/O Socket Metrics section, edit the property values to contain the list of those hosts or ports for which metrics are required:

If an invalid host or port is included in the parameter values, a warning message is written to the agent log and that value is ignored. If, as a result, the list contains no entries, no restriction will apply.

■ introscope.agent.io.socket.client.hosts A comma separated list of hosts; only 'client' socket metrics for specified hosts will be generated. Hosts may be specified by name or textual representation of IP address (in either IPv4 or IPv6 forms).

Socket metrics


Note: Duplicate host names are discarded. In cases where multiple host names map to a single IP, only one of the names is retained. The property will, however, match client connections with any of the set of synonymous names.

■ introscope.agent.io.socket.client.ports A comma separated list of port numbers; only 'client' socket metrics for specified ports will be generated.

Note: Duplicate ports are discarded.

■ introscope.agent.io.socket.server.ports Comma separated list of port numbers, only 'server' socket metrics for specified ports will be generated.

The above properties are dynamic; you do not need to restart your applications for changes to take effect.


Fine-tuning socket and SSL metric collection

While you can restrict socket and SSL metric collection, as detailed in Restricting socket and SSL metric collection (see page 128), you can further refine your metric collection by turning on and off specific tracer groups. This helps to target the information you want as well as reduce overhead costs.

To fine-tune socket and SSL metric collection

1. Open the java2.pbd file, located in the <Agent_Home>/wily/core/config directory.

2. In the I/O Socket Tracer Group or Network Tracer Group sections of java2.pbd, determine the tracers you want to turn on or off, and comment or uncomment them. For example, to suppress the metrics for input bandwidth, you comment out the following:

#TraceOneMethodWithParametersIfFlagged: SocketTracing read

InputStreamBandwidthTracer "Input Bandwidth (Bytes Per Second)"

Note: Tracers with names ending with MappingTracer must not be commented out.

3. Save the java2.pbd file.

SSL, NIO, and socket tracing in the application triage map

The application triage map introduced in the Introscope 9.0 Workstation Investigator has the ability to display instrumented client socket connections. The agent records details about external systems being used in transactions, sends this information to the Enterprise Manager, and then this information is displayed graphically in the application triage map.

Socket metrics


The tracers contained in the appmap.pbd, located in the <Agent_Home>/wily/core/config directory, extend existing SSL, NIO, and socket instrumentation, allowing the agent to send more component information to the application triage map. The tracers are enabled by default, but can be disabled by commenting out specific tracers in the Trace Sockets and Trace NIO Sockets sections of the appmap.pbd.

Component names, when displayed in the application triage map, include destination host and port identities, in the same way socket client metric names are displayed. The host identity in a component name can be configured to either a host name or a host IP address. The default is the host name.

To change the displayed component name

1. Open appmap.pbd, located in the <Agent_Home>/wily/core/config directory.

2. In the Trace Sockets and Trace NIO Sockets sections, select the tracers you want to modify. Change {hostname} to {hostip} in the relevant tracers.

For example, the original tracers use the default {hostname}:

TraceOneMethodWithParametersIfFlagged: SocketTracing read AppMapSocketTracerBT

"System {hostname} on port {port}"

TraceOneMethodWithParametersIfFlagged: SocketTracing write

AppMapSocketTracerBT "System {hostname} on port {port}"

To display the host IP, use {hostip} instead:

TraceOneMethodWithParametersIfFlagged: SocketTracing read AppMapSocketTracerBT

"System {hostip} on port {port}"

TraceOneMethodWithParametersIfFlagged: SocketTracing write

AppMapSocketTracerBT "System {hostip} on port {port}"

3. Save the appmap.pbd file.

Turning off socket and SSL metric collection

If collecting socket and SSL metrics is not required, they can be turned off entirely.

To turn off socket and SSL metric collection

1. Open either toggles-full.pbd or toggles-typical.pbd files (open the file you are using in your deployment).

2. Comment out (place a pound or hash sign,#, at the beginning of the line) SocketTracing:


3. Save the file you modified.

For more information about turning on and off tracer groups, in the toggles-full.pbd and toggles-typical.pbd files, see Default tracer groups and toggles files (see page 86) and Turning tracer groups on or off (see page 95).

Configuring logging options


Backwards compatibility

In the 9.0 Java agent, new socket tracers have been introduced that are more configurable than pre-9.0 tracers. However, if for any reason, you want to revert to pre-9.0 tracers (and disable new 9.0 socket tracing features and configuration options), the required configuration changes are outlined below.

To collect socket metrics using pre-9.0 tracers

1. Open either toggles-full.pbd or toggles-typical.pbd files (open the file you are using in your deployment).

2. Comment out (place a pound or hash sign,#, at the beginning of the line) SocketTracing:


3. Uncomment ManagedSocketTracing:

TurnOn: ManagedSocketTracing

4. Save the file you modified.

If you are using pre-9.0 socket tracers and input and output bandwidth metrics are required, they are enabled as follows.

The following steps are optional.

To collect input and output bandwidth metrics


2. Locate the Agent Socket Rate Metrics section and change the following property to true:

introscope.agent.sockets.reportRateMetrics=true

Note: Only functions if ManagedSocketTracing is enabled and SocketTracing is disabled.



When the Java Agent is installed on an application server, after the server starts up a log directory is created here: <Agent_Home>/wily/logs. The application server process must have full read/write/execute permissions on the <Agent_Home> directory. To accomplish this, install the Java agent on the same operating system as the user who runs the application server process. Or, install the Java agent as a different user, then use the chmod command to grant the necessary permissions.



The Java agent has the option to run in verbose mode. Verbose mode records higher levels of details about actions and agent interactions with your environment. This information is useful in solving issues with your environment or agent functionality.

Introscope uses Log4J functionality for these functions. If you want to use other Log4J functionality, please consult the Log4J documentation.

Running the agent in verbose mode

Running the agent in verbose mode records higher levels of information to the agent log.

To run the agent in verbose mode


2. Modify this property, replacing the existing INFO with VERBOSE#com.wily.util.feedback.Log4JSeverityLevel:

log4j.logger.IntroscopeAgent=VERBOSE#com.wily.util.feedback.Log4JSeverityLeve

l, console, logfile


Note: Changes to this property should take effect within one minute and do not require the managed application to be restarted.

Redirecting agent output to a file

The property that controls the agent logging in verbose mode also controls where the agent log is output and the location of this log file (see Running the agent in verbose mode (see page 132) for more information).

To redirect agent output to a file


2. Find the property: log4j.logger.IntroscopeAgent

The options for this property are:

■ console: the information in the logfile is sent to the console

■ logfile: the information in the logfile is sent to a logfile. If this is selected, the location of the log file is configured using the log4j.appender.logfile.File property. See Changing the name or location of the agent logfile (see page 133).

http://jakarta.apache.org/log4j/docs/documentation.html



For example, if you wanted the agent to report in verbose mode to just a logfile, the property would be set to:


l,logfile

If you wanted the agent to report to both a logfile and console, you would include both logfile and console in the property.

Note: By default the agent log, IntroscopeAgent.log is written to the <Agent_Home>/wily/logs directory. If you configured agent autonaming options, the agent log files are also automatically named, as described in Agent log files and automatic agent naming (see page 133).


Changing the name or location of the agent logfile

You can also change the location and name of a log file by modifying a property.

To change the name or location of the logfile


2. Locate the log4j.appender.logfile.File property.

If logfile was specified in the log4j.logger.IntroscopeAgent property, the location of the log file is configured using the log4j.appender.logfile.File property. See step 2 in Redirecting agent output to a file (see page 132) for more information.

Note: System properties (Java command line -D options) can be included as part of the file name. For example, if a Java command starts with -Dmy.property=Server1, then log4j.appender.logfile.File=../../logs/Introscope-${my.property}.log is expanded to: log4j.appender.logfile.File=../../logs/Introscope-Server1.log.

3. Set the location and name of the log file, using a fully qualified path to the new location and file. For example:

log4j.appender.logfile.File=C:/Logs/AgentLog1.log


Agent log files and automatic agent naming

If you use the automatic agent naming functionality, by default the log files associated with an agent are named automatically using the same information used to name the agent.

Automatic agent naming affects the log file in the following way:

■ If the original name of the logfile does not end in .log, a period and log is added.



■ All characters that are not letters or digits will be replaced by underscores

■ If advanced Log4J functionality is used, the agent logfile automatic naming capability might not work.

The following examples show how an agent logfile is named. The examples use an agent name of DOM1//ACME42, where DOM1 is the WebLogic domain, and ACME42 is the instance of the agent.

When an agent log file is created (named AutoProbe.log by default), if the agent name is not yet available, a timestamp is included in the filename:

AutoProbe.20040416-175024.log

Once the agent name becomes available, the logfile is renamed using the agent’s automatic name:

AutoProbe.DOM1_ACME42.log

You can disable automatic log naming - see Advanced automatic agent naming options (see page 123) for more information.

Rolling up logs by date or size

You can roll up logs based on size or date, retaining a specified number of days of information and purging the rest.

To roll up log files

1. Open the IntroscopeAgent.profile, and locate the Logging Configuration section.

2. Modify the following properties:

log4j.logger.IntroscopeAgent

log4j.appender.logfile.File

log4j.appender.console.layout

log4j.appender.console.layout.ConversionPattern

log4j.appender.logfile

log4j.appender.logfile.MaxFileSize

log4j.appender.logfile.MaxBackupIndex

Note: You must restart the managed application before changes to this property take effect.

Managing ProbeBuilder Logs



For example, the following configuration will keep up to three backup/rolled logs, and each will be up to 2 kilobytes in size:


l, console, logfile

log4j.appender.logfile.File=logs/IntroscopeAgent.log

log4j.appender.console.layout=com.wily.org.apache.log4j.PatternLayout

log4j.appender.console.layout.ConversionPattern=%d{M/dd/yy hh:mm:ss a z} [%-3p]

[%c] %m%n

log4j.appender.logfile=com.wily.introscope.agent.AutoNamingRollingFileAppende

r

log4j.appender.logfile.MaxFileSize=2KB

log4j.appender.logfile.MaxBackupIndex=3


ProbeBuilder logs all the classes it sees, all the classes it instruments, as well as all the classes it does not add instrumentation for the probes it added during the instrumentation process and the PBDs it used. In addition, it logs the classes it did not instrument due to skips.

Command-line ProbeBuilder and ProbeBuilder Wizard log name and location

The command-line ProbeBuilder and ProbeBuilder Wizard log file location is determined by where you specify Java classes with the ProbeBuilder Wizard or with the Command-Line ProbeBuilder. For a directory, the log file is located inside the destination directory. For a file, the log file is located next to the destination file.

The ProbeBuilder log file is called:

<original-directory-or-original-file>.probebuilder.log

<original-directory> or <original-file> is the Java class location that you specify with the ProbeBuilder Wizard or with the Command-Line ProbeBuilder.

Only the most recent log is kept; all previous log files are overwritten.



AutoProbe log name and location

AutoProbe will always attempt to log the changes it makes. By default the AutoProbe log file is named AutoProbe.log.

To change the name or location of the AutoProbe log


2. Locate the introscope.autoprobe.logfile property and modify the log name and location, using a fully qualified file path. Non-absolute names are resolved relative to the location of the IntroscopeAgent.profile file.

Note: When loading the agent profile from a resource on a classpath, AutoProbe is unable to write to the AutoProbe log file, because the IntroscopeAgent.profile file is located within a resource.

You must restart the managed application before changes to this property take effect.



Chapter 8: Configuring LeakHunter and ErrorDetector

This section describes how to enable and configure LeakHunter and ErrorDetector.


LeakHunter (see page 137) Enabling and disabling LeakHunter (see page 140) Configuring LeakHunter properties (see page 140) Running LeakHunter (see page 143) Identifying potential leaks with collection IDs (see page 143) LeakHunter log file (see page 144) Using LeakHunter (see page 146) ErrorDetector (see page 147) Enabling ErrorDetector in the Java Agent (see page 148) Configuring ErrorDetector options (see page 149) Advanced error data capture (see page 150) Defining new error types (see page 150) Using ErrorDetector (see page 152)

LeakHunter

Introscope LeakHunter is an add-on component designed to help locate the source of potential memory leaks by watching for collection instances that appear to be increasing in size over time—that is, if the number of objects stored in the collection increases over time.

Memory leaks that occur in programs that run for short periods (minutes or hours) might not present a major issue. However for applications that are running 24 hours a day (like a web site), a small memory leak can soon escalate into a big problem.

Introscope LeakHunter is designed to track information about a noticed memory leak by being turned on, discovering collection classes, and then being turned off after information is gathered. This method of using LeakHunter creates only a small, temporary amount of overhead.

LeakHunter


How LeakHunter works

After you have enabled LeakHunter, you also define a timeout period during which LeakHunter looks for new potential leaks. If you use AutoProbe, you simply restart the managed application. If you use ProbeBuilder Wizard or Command-line ProbeBuilder, you must re-instrument the application using the leakhunter.pbd (in addition to any previously used PBD files).

If LeakHunter finds a collection that is growing in size over time, it:

■ identifies the collection with a unique ID

■ reports information about the collection to the Enterprise Manager as metric data

■ reports information about the collection to a log file on the agent machine

■ continues to track and report data for that collection

Should LeakHunter notice that a collection no longer appears to be leaking, it reports that fact to both the Enterprise Manager and the log file, but continues tracking and reporting data for that collection.

LeakHunter continues looking for potential leaks and monitors already identified potential leaks until the timeout period expires. After a timeout period expires, LeakHunter stops looking for potential leaks in newly allocated collections, and only continues checking collections that are already identified as potential leaks. This significantly reduces LeakHunter overhead and allows additional monitoring of the potential leaks. LeakHunter continues monitoring the identified potential leaks until the managed application is shut down.

To find the source of a memory leak, you can either browse the metric data in the Introscope Investigator, or view the log file.

What LeakHunter tracks in Java

Introscope LeakHunter tracks these collections in a Java implementation:

■ Implementations of java.util.Collection:

■ java.util.ArrayList

■ java.util.LinkedList

■ java.util.TreeSet

■ java.util.HashSet

■ java.util.LinkedHashSet

■ java.util.Vector

■ java.util.Stack

LeakHunter


■ Implementations of java.util.Map:

■ java.util.Map

■ java.util.SortedMap

■ java.util.HashMap

■ java.util.TreeMap

■ java.util.IdentityHashMap

■ java.util.Hashtable

■ java.util.Properties

■ java.util.LinkedHashMap

What LeakHunter does not track

Introscope LeakHunter does not track:

■ a leak not caused by a collection.

■ custom collection implementations or other data structures with an increasing number of references.

■ a leaking collection that is not instrumented.

In addition to the above, LeakHunter does not track the following in a Java implementation:

■ any subclasses created for collections described in What LeakHunter tracks in Java (see page 138). However, you can update the ProbeBuilder Directive (PBD) file to get this information. See the leakhunter.pbd file for more information.

Note: If you are using Application Server AutoProbe, LeakHunter does not automatically track collections allocated by the application server. To track these collections, you must statically instrument the application server, or use JVM AutoProbe.

System and version requirements

Introscope LeakHunter has the same system requirements as the Java Agent.

By default, LeakHunter is not enabled after installation. You must enable LeakHunter to use its functionality.

Enabling and disabling LeakHunter


Enabling and disabling LeakHunter

LeakHunter is run as an agent extension, so no class paths need to be updated. By default, LeakHunter is not enabled after installation. You must enable LeakHunter to use its functionality.

To enable LeakHunter

1. Open the agent profile, IntroscopeAgent.profile.

2. Under the LeakHunter Configuration heading, locate the property introscope.agent.leakhunter.enable, and enter a value of true.

3. Save the agent profile.

4. Open *.typical.pbl or *.full.pbl (open the file you have listed in the IntroscopeAgent.profile property introscope.autoprobe.directivesFile ) and uncomment leakhunter.pbd.

Note: When using ProbeBuilder Wizard, copy the leakhunter.pbd file to the <Agent_Home>\wily\core\config\hotdeploy directory.


Important! By default, agent extensions such as LeakHunter are located and referenced from the <Agent_Home>\wily\core\ext directory. However, you can change the location of the agent extension directory in the agent profile. If you change the location of the \ext directory, be sure to move the contents of the \ext directory as well.

To disable LeakHunter


2. Under the LeakHunter Configuration heading, locate the property introscope.agent.leakhunter.enable, and enter a value of false.



Configuring LeakHunter properties

LeakHunter configuration properties are located in the agent profile, IntroscopeAgent.profile.

To configure LeakHunter




2. Configure the following LeakHunter properties as desired:

introscope.agent.leakhunter.logfile.location

Specifies the location of LeakHunter.log file. The file name is relative to the <IntroscopeAgent.profile> directory. If the property is commented out or left blank, no log file is written.

The default value is logs/LeakHunter.log.

introscope.agent.leakhunter.logfile.append

Specifies whether to replace the log file (value of false) or append an existing log file (value of true) on application restart.

The default value is false.

introscope.agent.leakhunter.leakSensitivity

Specifies the sensitivity level for detecting memory leaks. A higher leak sensitivity setting will result in more potential leaks being reported and a lower sensitivity will result in fewer potential leaks reported.

The property value must be an integer from 1-10.

The default sensitivity level is 5.

introscope.agent.leakhunter.timeoutInMinutes

Specifies the period of time in minutes during which LeakHunter looks for new potential leaks. The property value must be a non-negative integer. A value of zero means no timeout.

The default value is 120 minutes.



introscope.agent.leakhunter.collectAllocationStackTraces

Specifies whether to collect allocation stack trace information. Turning on this option has the potential to create higher system CPU usage and memory. Changes to this property take effect immediately and do not require the managed application to be restarted.

The default value is false.

introscope.agent.leakhunter.ignore.n

Specifies the particular collections that you want LeakHunter to ignore.

For Generic collections, use a syntax that includes the generic type qualification, for example: System.Collections.Generic.List`1

Changes to this property take effect immediately and do not require the managed application to be restarted.

The default values for these properties (where n=0-4) are:

introscope.agent.leakhunter.ignore.0=org.apache.taglibs.standard.lang.jst

l.*

introscope.agent.leakhunter.ignore.1=com.bea.medrec.entities.RecordEJB_xw

cp6o__WebLogic_CMP_RDBMS

introscope.agent.leakhunter.ignore.2=net.sf.hibernate.collection.*

introscope.agent.leakhunter.ignore.3=org.jnp.interfaces.FastNamingPropert

ies

introscope.agent.leakhunter.ignore.4=java.util.SubList

3. Save changes to the agent profile.

Important! The IntroscopeAgent.profile contains properties that control which packages are ignored by LeakHunter. These properties are enabled by default. If you comment out these properties, exceptions are reported in the agent logs.

Ignoring collections that cause poor performance

Collections whose size() method executes in time proportional to the number of objects in the Collection will have poor performance. In other words, if the collection is such that the size() method takes longer and longer to execute (for example, a badly implemented LinkedList where to get the size of the list we traverse through each element of the list and count), this will have negative effects on application performance.

Such collections should be ignored, using the ignore property in IntroscopeAgent.profile, as shown in Configuring LeakHunter properties.

Running LeakHunter


Running LeakHunter

LeakHunter is run as an agent extension.

To run LeakHunter

■ Using JVM AutoProbe: after LeakHunter has been enabled, restart the application.

■ Using Command-Line ProbeBuilder: re-instrument the managed application using the leakhunter.pbd (and any previously used .pbds). Restart the managed application.

■ Using ProbeBuilder Wizard: run ProbeBuilder Wizard and select the leakhunter.pbd from the custom pbd list. Restart the managed application after implementing new .jars for use.

Identifying potential leaks with collection IDs

LeakHunter identifies each potential leak with a unique collection ID that can be used to correlate metric data from the Investigator tree with data in the log file, as well as provide stable names across applications.

The unique collection IDs have a specific syntax, based on one of these types:

<method>-<4 digit hash code>#<unique number>

<field>-<4 digit hash code>#<unique number>

■ <method>: the name of the method where the collection was allocated

■ <field>: name of field to which the collection was assigned

■ <4 digit hash code>: the hash code of the full name of the class containing the method or field name

■ #<unique number>: number appended to potential leaks with the same method and hash code to ensure unique collection IDs during the run of the agent.

The collection ID for a potential leak will be stable across runs of the application.

Examples of these collection IDs are:

theLookupTable-6314#1

getLoginID-1234#1

getLoginID-1234#2

getLoginID-1234#3

verifyCart-5678#1

verifyCart-0012#1

LeakHunter log file


LeakHunter log file

The LeakHunter log file contains information about potential leaks identified by Introscope LeakHunter in your managed application. Each entry contains information about a potential leak. The LeakHunter log file includes entries for four scenarios:

■ when a potential leak is first identified—see Potential leak first identified log entry (see page 144)

■ when an identified leak appears to have stopped leaking—see Identified potential leak stops leaking log entry (see page 145)

■ when a previously identified leak appears to have started leaking again—see Identified potential leak is leaking again log entry

■ when a LeakHunter timeout has occurred—see LeakHunter timeout log entry

Potential leak first identified log entry

This type of LeakHunter log entry contains information about a potential leak when it is first identified:

■ Current timestamp (when written to the log)

■ Collection ID

■ Class of the collection

■ Allocation Method of the collection

■ Allocation time of the collection

■ Allocation stack trace of the collection

■ Field name to which the collection was assigned

■ Current size of the collection

LeakHunter log file


Note: The current size of a leaked collection recorded in the LeakHunter log file is not dynamically updated. The log file identifies the size of a leak when the leak is first identified. To see up-to-date information about the size of a leaked connection, click the Leak tab in the Introscope Workstation.

Example: Log file entry if a potential leak is detected

The following example illustrates an entry in the Java log file when a potential leak is first identified:

5/2/09 9:55:06 AM PDT

Potential leak identified

Assigned ID: testInst-2604#1

Collection Class: java.util.Vector

Allocation Method: sonOfLH_test.testInst()

Allocation Timestamp: 5/2/09 9:54:21 AM PDT

Allocation Stack Trace:

Unknown

Field Name(s):

sonOfLH_test.v3

sonOfLH_test.v4

sonOfLH_test.v5

Current Size: 44

Identified potential leak stops leaking log entry

This type of LeakHunter log entry contains information about a potential leak that has stopped leaking:


■ Collection ID



Example: Log file entry if a potential leak stops leaking

The following example illustrates an entry in the Java log file when a potential leak appears to have stopped leaking:

4/27/10 1:18:12 PM PDT

Potential leak no longer appears to be leaking

Assigned ID: createNewInstance-2815#3

Collection Class: java.util.HashMap

Current Size: 70

Using LeakHunter


Identified potential leak is leaking again log entry

This type of entry contains information about a potential leak that has started leaking again:


■ Collection ID



Example: Log file entry if a potential leak resumes leaking

The following example illustrates an entry in the Java log file when a potential leak appears to have started leaking again:

4/27/10 1:21:42 PM PDT

Potential leak appears to be leaking again

Assigned ID: createNewInstance-2815#3

Collection Class: java.util.HashMap

Current Size: 79

LeakHunter timeout log entry

This type of LeakHunter log entry contains the number of potential leaks that will continue to be tracked.

Example: Log file entry when a timeout occurs

LeakHunter timeout occurred at 4/27/10 1:32:12 PM PDT

LeakHunter will only continue to track the 3 potential leaks

Using LeakHunter

For more information on how to use LeakHunter, see the CA APM Workstation User Guide.

ErrorDetector


ErrorDetector

Introscope ErrorDector allows application support personnel to detect and diagnose the cause of serious errors, which can prevent users from completing web transactions. These kinds of application availability issues typically result in error messages to the user such as "404 Not Found" and others, yet the error message lacks specific information that IT personnel need to isolate the root cause of the problem. Introscope ErrorDetector allows you to monitor these serious errors as they occur in live applications, determine the frequency and nature of the errors, and deliver specific information about the root cause to developers.

ErrorDetector is the only application management solution that helps ensure superior user experiences and improves transaction integrity by enabling root cause analysis of serious application errors.

Introscope ErrorDetector allows IT teams to:

■ Determine the frequency of anomalous transactions

■ Determine whether logged exceptions affect users

■ See exactly where errors occurred within the transaction path

■ Obtain the information needed to reproduce, diagnose and eliminate serious errors

Introscope ErrorDetector is integrated with Introscope, allowing you to monitor errors in the Introscope Workstation. When application errors do occur, you can also use the Live Error Viewer to examine detailed information about each error.

Types of errors

CA Technologies has defined a set of criteria to describe "serious" errors, based on information contained in the J2EE specifications. ErrorDetector considers both errors and exceptions to be errors. The most common type of error is a thrown exception.

Some examples of common errors are:

■ HTTP errors (404, 500, etc.)

Note: Occasionally, HTTP 404 errors originate in a web server instead of an application server. If this occurs, ErrorDetector cannot detect the web server error through the agent.

■ SQL statement errors

■ network connectivity errors (timeout errors)

■ backend errors (for example, can’t send a message through JMS, can't write a message to the message queue)

Enabling ErrorDetector in the Java Agent


What CA Technologies considers to be important errors might differ from what you consider important errors. If you do not consider some of the errors ErrorDetector tracks to be important, you can choose to ignore them. If there are additional errors you want to track, you can use error tracers to create new directives to trace them.

How ErrorDetector works

Introscope includes a ProbeBuilder Directive (PBD) file called errors.pbd with the agent installation. The tracers in this PBD capture serious errors.

ErrorDetector is installed automatically with your agent installation. Once ErrorDetector is installed, you configure Introscope to use the errors.pbd and enable ErrorDetector functionality. If you are using ProbeBuilder Wizard or Command-line ProbeBuilder, you must re-instrument the application using the errors.pbd (in addition to any previously used PBD files).

The agent collects error information as defined in the errors.pbd file.

From the Workstation, you can view:

■ error metric data in the Investigator

■ live errors in the Live Error Viewer

■ error details in the Error Snapshot, which shows component-level information on how the error occurred

ErrorDetector is integrated with Transaction Tracer, enabling you to see exactly why and how serious errors occurred within the context of the transaction path. Moreover, all errors and transactions are persisted to Transaction Event Database, enabling you to spot trends through analysis of historical data.

Introscope defines a transaction as the invocation and processing of a service. In the context of a web application, it is the invocation and processing of a URL sent from a web browser. In the context of a web service, it is the invocation and processing of a SOAP message.

Enabling ErrorDetector in the Java Agent

The property introscope.agent.errorsnapshots.enable should be set to true in the IntroscopeAgent.profile to enable the Java Agent to capture error data. By default, the Java Agent is enabled to capture error data.

To enable/disable ErrorDetector data capture in the Java Agent


Configuring ErrorDetector options


2. Confirm the introscope.agent.errorsnapshots.enable property to true.

Note: To disable ErrorDetector, set this property to false.

3. If you are using ProbeBuilder Wizard, copy the errors.pbd file to the <EM_Home>/config/custompbd directory, and re-instrument your applications.


Configuring ErrorDetector options

You can configure ErrorDetector to limit the maximum number of errors the agent sends to the Enterprise Manager, or specify the errors to ignore.

Enabling ErrorDetector captures error data without incurring much overhead. The out-of-the-box throttle is set at 10 errors per 15 seconds. If you want to capture more errors during this time period, you can increase the throttle, but be prepared to incur more overhead.

To change the ErrorDetector throttle (optional)


2. Enter a new value for the introscope.agent.errorsnapshots.throttle property.


Note: Changes to this property take effect immediately and do not require the managed application to be restarted.

You can configure the agent to ignore errors you don’t want to track. The information you specify to "tag" the error can be the exact error message, or any portion of the message, along with the "wildcard" asterisk character.

To ignore certain errors (optional)


2. For the introscope.agent.errorsnapshots.ignore property, define the value to be whatever information you think will identify that type of error.

For example, the following ignore property would ignore any errors with the term "IOException" found anywhere within it:

introscope.agent.errorsnapshots.ignore.0=*IOException*

Advanced error data capture


3. To ignore additional errors, add additional ignore properties sequentially. For example, to ignore two types of errors, the properties would look like this:

introscope.agent.errorsnapshots.ignore.0=*IOException*

introscope.agent.errorsnapshots.ignore.1=*HTTP Error Code *500*

4. Save changes to the agent profile.

Note: Changes to this property take effect immediately and do not require the managed application to be restarted.

Advanced error data capture

Although ErrorDetector captures many common error types by default, CA Technologies provides options for customizing the error detection mechanism to suit your needs.

You can use the following error-related tracers to create ProbeBuilder Directives (PBDs) to capture errors.

■ ExceptionErrorReporter reports standard exceptions.

■ ThisErrorReporter reports a current object as an error.

■ HTTPErrorCodeReporter captures HTTP error codes and associated error messages.

■ MethodCalledErrorReporter reports if specific methods get called.

You should place any new directives you create with these tracers in the errors.pbd file in the <Agent_Home>/wily directory.

Defining new error types

Errors are defined for ErrorDetector using PBDs. There are several special tracers that check for the presence of an error and capture (or construct) the error message. By placing these tracers at the right points, you can teach ErrorDetector about a new kind of error in your application or its infrastructure.

ExceptionErrorReporter

The ExceptionErrorReporter tracer can be used to check for exceptions being thrown from the instrumented method. If an exception is thrown, this tracer treats it as an error and gets the error message from the exception. This is the most common definition of an error.

Defining new error types


In order to capture error messages, the ExceptionErrorReporter tracer must be used with a "...WithParameters" directive. For example:

TraceOneMethodWithParametersOfClass: com.bank.CustomerAccount getBalance ExceptionErrorReporter "CustomerAccount:Errors Per Interval"

This directive specifies that any exception that is thrown from the getBalance() method on the CustomerAccount constitutes an error.

Note that you must use a "...WithParameters" directive to increment the errors per interval metric, but you only have to specify a "...WithParameters" directive once for any method to make the parameters available for all tracers on that method. For example, you can specify: TraceOneMethodWithParametersOfClass: com.myClass myMethod BlamePointTracer This directive makes the parameters for the com.myClass myMethod method available to other tracers, including the ExceptionErrorReporter tracer.

MethodCalledErrorReporter

The MethodCalledErrorReporter tracer is used on methods where the very act of the method being called means that an error has occurred. For example:

TraceOneMethodOfClass: com.bank.CheckingAccount cancelCheck MethodCalledErrorReporter "CustomerAccount:Canceled Checks Per Interval"

This directive specifies that whenever the cancelCheck() method is called (for any reason), this is an error. The error message simply states the class and method that was called.

If you do not know which methods may throw an exception or error, try using the ThisErrorReporter tracer.

ThisErrorReporter

The ThisErrorReporter tracer is similar to the MethodCalledErrorReporter, but it constructs the error message by calling toString() on the instrumented objects. This tracer is most useful to put on the constructor of an exception class. For example:

TraceOneMethodWithParametersOfClass: ezfids.util.exception.EasyFidsException [set

the init variable for your book] ThisErrorReporter

"Exceptions|{packageandclassname}:Errors Per Interval"

Note: To capture error messages, the ThisErrorReporter tracer must be used with a "...WithParameters" directive.

Using ErrorDetector


The directives specify that whenever the constructor ("init" or ".ctor") of an InvalidPINException is called, this constitutes an error. The error message is determined by calling toString() on the InvalidPINException which generally returns the error message that the application developer specified.

This tracer is good to use when you have a custom error management system based on your own exception types.

Note: Introscope cannot instrument any code in the java.* packages so placing this tracer on java.lang.Exception or java.sql.SQLException does not work.

HTTPErrorCodeReporter

The HTTPErrorCodeTracer tracer reports error codes from Servlets and JSPs. It is a per interval counter that counts incidents of:

■ HTTP response codes 400 or higher.

■ javax.servlet.http.HttpServletResponse subclass invocations of sendError or setStatus, for codes 400 or higher in Java environments.

See errors.pbd for example usage.

Using error tracer directives with caution

Be judicious in using the tracers described in the previous sections. The best practice is to incur the overhead associated with error tracing to report only the most serious problems, such as unrecoverable problems arising from backend systems.

The default errors.pbd is designed to report serious errors, while minimizing overhead as much as possible. Overuse of error tracing, for instance, applying ExceptionErrorReporter to every monitored method can result in a high volume of "false positives." For example, if a user enters "California" in a numeric field, this may cause a NumberFormatException, which you would not want to report as a serious problem.

Using ErrorDetector

For more information on how to use ErrorDetector, see the CA APM Workstation User Guide.


Chapter 9: Configuring Boundary Blame

This section describes default Java Agent blame reporting behaviors, and related configuration options.


Understanding Boundary Blame (see page 153) Using URL groups (see page 153) Using Blame tracers (see page 159)

Understanding Boundary Blame

Blame Technology lets you work in a managed Java Application to view metrics at the front and backends of your application. This capability, referred to as boundary blame, lets users triage problems.

How Introscope detects backends varies depending on the application. For database activity, Introscope uses the SQL Agent to detect backends. If the SQL Agent is unavailable, Introscope detect backend activity because backends such as client/server databases, JMS servers, and LDAP servers are accessed through a socket. If you have Oracle backends and do not use the Introscope SQL Agent, see Boundary Blame and Oracle backends (see page 115).

For information about how boundary blame is presented in the Introscope Investigator, see the CA APM Workstation User Guide.

Using URL groups

You can use URL Groups to monitor browser response time for sets of requests whose path prefix begins with a string you define. The path prefix is the portion of the URL that follows the hostname. For example, in this URL:

http://burger1.com/testWar/burgerServlet?ViewItem&category=

11776&item=5550662630&rd=1

the path prefix is:

/testWar

You can define a URL group for any useful category of requests that can be derived from a URL’s path prefix. For example, depending on the form of your application URLs, you could define URL groups for each customer your application supports, for each major application, or for sub-applications. This enables you to monitor performance in the context of committed service levels, or for mission-critical portions of your application.

Using URL groups


Example: How URL group properties are defined

The following example is an excerpt from a Java Agent profile, showing how URL Groups are defined:

introscope.agent.urlgroup.keys=alpha,beta,gamma

introscope.agent.urlgroup.group.alpha.pathprefix=/testWar

introscope.agent.urlgroup.group.alpha.format=foo {host} bar {protocol} baz {port}

quux {query_param:foo} red {path_substring:2:5} yellow {path_delimited:/:0:1} green

{path_delimited:/:1:4} blue {path_substring:0:0}

introscope.agent.urlgroup.group.beta.pathprefix=/nofilterWar

introscope.agent.urlgroup.group.beta.format=nofilter foo {host} bar {protocol} baz

{port} quux {query_param:foo} red {path_substring:2:5} yellow {path_delimited:/:0:1}

green {path_delimited:/:1:4} blue {path_substring:0:0}

introscope.agent.urlgroup.group.gamma.pathprefix=/examplesWebApp

introscope.agent.urlgroup.group.gamma.format=Examples Web App

Defining keys for URL groups

The property introscope.agent.urlgroup.keys defines a list of the IDs, or keys, of all of your URL Groups. The key for a URL Group is referenced in other property definitions that declare an attribute of the URL group. The following example defines the keys for three URL Groups:

introscope.agent.urlgroup.keys=alpha,beta,gamma

If you define URL Groups so some URLs fall into multiple groups, the order in which you list the keys for the URL Groups in the property is important. The URL Group with the narrower membership should precede the URL Group with broader membership. For example, if the IP Group with key alpha has the path prefix /examplesWebApp and the URL Group with key delta has the path prefix /examplesWebApp/cleverones, delta should precede alpha in the keys parameter

Defining membership of each URL group

The property introscope.agent.urlgroup.group.groupKey.pathprefix specifies a pattern against which the path prefix of a URL is matched, defining which requests fall within the URL Group.

Using URL groups


Example: Mapping a group key to a URL path prefix

This property definition assigns all requests in which the path portion of the URL starts with /testWar to the URL Group whose key is alpha:

introscope.agent.urlgroup.group.alpha.pathprefix=/testWar

Requests that match the specified pathprefix include:

http://burger1.com/testWar/burgerServlet?ViewItem&category=

11776&item=5550662630&rd=1

http://burger1.com/testWar/burgerServlet?Command=Order&item=5550662630

Example: Creating URL groups by application path

A company that provides call center services could monitor response time for functional areas by setting up a URL Group for each application function. If customers access services with this URL:

http://genesystems/us/application_function/

where application_function corresponds to applications such as OrderEntry, AcctService, and Support, the pathprefix property for each URL group would specify the appropriate application_function.

Note: You can use the asterisk symbol (*) as a wildcard in pathprefix properties.

Define the name for a URL group

The property introscope.agent.urlgroup.group.groupKey.format determines the names under which response time metrics for a URL group whose key is groupKey appear in the Introscope Workstation.

Typically, the format property is used to assign a text string as the name for a URL. The following example causes metrics for the URL Group with key alpha to appear in the Workstation under the name Alpha Group:

introscope.agent.urlgroup.group.alpha.format=Alpha Group

Advanced naming techniques for URL groups (optional)

You can derive a URL Group name from request elements such as the server port, the protocol, or from a substring of the request URL. This is useful if your application modules are easily differentiated by inspecting the request. This section describes advanced forms of the format property.

Using URL groups


Using host as a URL group name

To organize metrics for a URL group under names that reflect the hostname of the HTTP server associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={host}

When format={host}, statistics for these requests would appear under the metric names us.mybank.com and uk.mybank.com respectively:

https://us.mybank.com/mifi/loanApp......

https://uk.mybank.com/mifi/loanApp.....

Using protocol as a URL group name

To organize statistics for a URL group under names that reflect the protocol associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={protocol}

When format={protocol} statistics are grouped in the Investigator under metric names that correspond to the protocol portion of request URLs. For example, statistics for these requests would appear under the metric name https:

https://us.mybank.com/cgi-bin/mifi/scripts......

https://uk.mybank.com/cgi-bin/mifi/scripts......

Using port as a URL group name

To organize statistics for a URL group under names that reflect the port associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={port}

When format={port}, statistics are grouped under names that correspond to the port portion of request URLs. For example, statistics for these requests would appear under the name 9001.

https://us.mybank.com:9001/cgi-bin/mifi/scripts......

https://uk.mybank.com:9001/cgi-bin/mifi/scripts......

Using URL groups


Using parameter as a URL group name

To organize statistics for a URL group in Investigator under metric names that reflect the value of a parameter associated with requests, define the format parameter like this:

introscope.agent.urlgroup.group.alpha.format={query_param:param}

When format={query_param:param} statistics are grouped in Investigator under metric names that correspond to value of the parameter specified. Requests without parameters are listed under <empty>. For example, given this parameter definition:

introscope.agent.urlgroup.group.alpha.format=

{query_param:category}

Statistics for these requests would appear under the metric name "734"

http://ubuy.com/ws/shoppingServlet?ViewItem&category=734

&item=3772&tc=photo

http://ubuy.com/ws/shoppingServlet?ViewItem&category=734

&item=8574&tc=photo

Using a substring of the request path as a URL group name

To organize statistics for a URL group under names that reflect a substring of the path portion of request URLs, define the format parameter like this:


{path_substring:m:n}

where m is the index of the first character, and n is one greater than the index of the last character. String selection operates like the java.lang.String.substring() method. For example, given this setting:


{path_substring:0:3}

Statistics for the following request would appear under the metric node /ht:

http://research.com/htmldocu/WebL-12.html

Using URL groups


Using a delimited portion of the request path as a URL group name

To organize statistics for a URL group under names that reflect a character-delimited portion request URL path, define the format parameter like this:


{path_delimited:delim_char:m:n}

where delim_char is the character that delimits the segments in the path, m is the index of the first segment to select, and n is one greater than the index of the last segment to select. For example, given this setting:

introscope.agent.urlgroup.group.alpha.format={path_delimited:/:2:4}

statistics for the requests of this form:

http://www.buyitall.com/userid,sessionid/pageid

would appear under the metric name /pageid

Note that:

■ a delimiter character, in this example, the forward slash (/) counts as a segment

■ the segment count starts at 0

The segments as delimited by the slash character in the URL example are:

0=/, 1=userid,sessionid, 2=/, and 3=pageid

You can specify multiple delimiters as necessary. For example, given this setting:

introscope.agent.urlgroup.group.alpha.format={path_delimited:/,:3:4}

statistics for requests of the form shown above would appear under the metric name sessionid with the segments as delimited by the slash and the comma in the URL example are:

0=/, 1=userid, 2=, 3=sessionid, 4=/, and 5=pageid

Using multiple naming methods for URL groups

You can combine multiple naming methods in a single format string, as shown below:

introscope.agent.urlgroup.group.alpha.format=red {host} orange {protocol} yellow

{port} green {query_param:foo} blue {path_substring:2:5} indigo

{path_delimited:/:0:1} violet {path_delimited:/:1:4} ultraviolet

{path_substring:0:0} friend computer

Using Blame tracers


Running the URLGrouper

URLGrouper is a command-line utility that analyzes a web server log file in Common format. Using URLGrouper gives you a starting point for defining your own URL Groups.

Note: You can use the URLGRouper utility to analyze your Web server log file. URLGrouper outputs a set of property settings for potential URL Groups, based on the contents of the Web server log file. The asterisk symbol (*) can be used with URLGrouper as a wildcard.

To run URLGrouper

1. Open a command shell.

2. Enter this command

java -jar urlgrouper.jar logfile

where logfile is the full path to your web server log file.

3. Property definitions for a set of URL Groups are output to STDOUT.

4. To configure the proposed URL Groups, copy the property statements produced by URL Grouper into the IntroscopeAgent.profile.

Using Blame tracers

You can use tracers to explicitly mark the frontends and backends in your application. For more information, see Using Blame Tracers to mark blame points (see page 113).


Chapter 10: Configuring Transaction Trace Options

This section has information about default Transaction Tracing behaviors and related configuration options.


Controlling automatic Transaction Tracing behavior (see page 161) Configuring cross-process Transaction Tracing (see page 163) Extending transaction trace data collection (see page 163) Disabling the capture of stalls as Events (see page 166)

Controlling automatic Transaction Tracing behavior

Automatic Transaction Tracing enables historical analysis of potentially problematic transaction types without explicitly running Transaction Traces. Introscope offers two types of automatic Transaction Tracing:

■ Transaction Trace sampling that is enabled by default, based on your URL groupings

■ Configurable automatic trace sampling that gathers trace information regardless of URL groupings

Transaction Trace component clamp

Introscope sets a component clamp to limit the size of traces. The default is 5,000 components. When this limit is reached, a warning appears in the log, and the trace stops.

You can clamp a component transaction that exceeds expected component counts, for example, when a servlet executes hundreds of object interactions and backend SQL calls. Without the clamp, Transaction Tracer views it as one transaction, continuing infinitely. Without a clamp in place in certain extreme situations, the JVM can run out of memory before the trace completes.

The property for clamping transactions is located in the IntroscopeAgent.profile file:

introscope.agent.transactiontrace.componentCountClamp=5000

Controlling automatic Transaction Tracing behavior


Traces producing clamped components are marked with an asterisk. For more information about viewing these traces, see the CA APM Workstation User Guide.

Important! If the Transaction Trace component clamp size is increased, the memory required for Transaction Traces can increase. For more information, see the CA APM Configuration and Administration Guide.

Transaction trace sampling

Transaction trace sampling is enabled by default. As appropriate you can disable this behavior. For more information on Transaction Trace properties, see Transaction tracing (see page 289).

When you configure automatic trace sampling, you specify the number of transactions to trace, during a time interval you specify.

Note: These properties are located in the Enterprise Manager properties file. Before changing the defaults for the sampling.perinterval and sampling.interval properties, consider the potential for increased load in the Enterprise Manager with higher sampling rates. The Enterprise Manager will push this configuration to all agents connected to it. You can also configure these properties in an agent. Configuring these properties in the agent will overwrite the configuration set by the Enterprise Manager for an individual agent.

To configure automatic trace sampling, modify these properties

■ introscope.agent.transactiontracer.sampling.enabled

Set to false to disable Transaction Trace sampling. The default value is true.

■ introscope.agent.transactiontracer.sampling.perinterval.count

Specifies the number of transactions to trace, during the interval you specify.

The default number of transactions is 1.

■ introscope.agent.transactiontracer.sampling.interval.seconds

Specifies the length of time to trace the number of transactions you specify.

The default interval is every 2 minutes.

Agent heap sizing

The agent uses Java heap memory to store collected data. If your application’s heap is highly utilized, you may need to increase the heap allocation when you install the agent. For example, the 8.0 agent, on average, uses slightly more memory than the 7.x agent because of performance improvements to CPU and response times overhead. You may see an increase of up to 100 MB over your 7.x average runtime heap usage.

Configuring cross-process Transaction Tracing


If monitored applications are characterized by very deep or long lasting transactions, the agent’s Transaction Trace sampling may require more heap memory than previous Introscope versions. For more information, see Transaction Trace component clamp (see page 161) and Transaction trace sampling (see page 162).

You can view the agent GC heap usage in the GC Heap overview. See the CA APM Workstation User Guide.

If you are operating a high-performance Introscope environment, contact CA Professional Services for the appropriate agent JVM heap settings.

Configuring cross-process Transaction Tracing

The Introscope Java agent can trace transactions that cross JVM boundaries on WebLogic Server 9 or later, or WebSphere 6.1—if the environment is comprised of compatible versions of the same vendor’s application server.

Cross-process transaction tracing is supported for synchronous transactions, for instance servlets to EJBs, and asynchronous transactions. For more information about cross-process transaction tracing, see:

■ Cross-process Transaction Tracing in WebLogic (see page 43)

■ Cross-process Transaction Tracing in WebSphere (see page 54)

Important! Cross-process transaction tracing is available only with Introscope 9.0 agents. Cross-process transaction tracing does not function between 9.0 and pre-9.0 agents.

Extending transaction trace data collection

You can configure the Introscope Transaction Tracer to obtain additional information, including User ID data for Servlet and JSP invocations, and other transaction trace data such as HTTP request headers, request parameters, and session attributes. To capture this information, you must define the criteria in the IntroscopeAgent.profile.

About User ID data

To configure the Java Agent to identify User IDs for Servlet and JSP invocations, you must first obtain information on how your managed application specifies user IDs. The Application Architect who developed the managed application can probably provide this information.



Introscope Transaction Tracer can identify User IDs from managed applications that store User IDs in one of these ways:

■ HttpServletRequest.getRemoteUser()

■ HttpServletRequest.getHeader (String key)

■ HttpSession.getValue (String key), where returned object is either a String representing the UserID, or an Object whose toString() returns to the UserID

If your managed application stores User IDs using one of these methods, see Configuring Agent to collect additional transaction trace data (see page 164), to configure Java Agent settings to collect User ID data.

About servlet request data

Using Introscope, you can collect transaction trace data that matches user-configurable parameters. For example, you can specify the agent to collect transaction trace data for transactions that contain the User-Agent HTTP request header.

Introscope can record this servlet request information:

■ request headers

■ request parameters

■ session attributes

To record this servlet request information for your managed application, see Configuring Agent to collect additional transaction trace data (see page 164) to configure Java Agent settings to collect this data.

Configuring Agent to collect additional transaction trace data

You can configure the Java agent to collect additional transaction trace data such as User ID, HTTP request headers, HTTP request parameters, or HTTP session attributes.

Follow these steps:


2. Locate the Transaction Tracer properties under the Transaction Tracer Configuration heading.



Collecting user id data

To configure the Java agent to identify User IDs

1. Configure the properties that correspond to the method your managed application uses to store User IDs.

Note: Ensure that only one set of properties are not commented, or the wrong properties might be used.

2. For HttpServletRequest.getRemoteUser(), uncomment the property:

introscope.agent.transactiontracer.userid.method=HttpServletRequest.getRemote

User

3. For HttpServletRequest.getHeader (String key), uncomment the following pair of properties, and define a key string for the second property:

introscope.agent.transactiontracer.userid.method=HttpServletRequest.getHeader

introscope.agent.transactiontracer.userid.key=<application defined key string>

4. For HttpSession.getValue (String key), uncomment the following pair of properties, and define a key string for the second property:

introscope.agent.transactiontracer.userid.method=HttpServletRequest.getValue

introscope.agent.transactiontracer.userid.key=<application defined key string>

Collecting servlet request data

To record servlet request information such as HTTP request headers and parameters

1. To specify the HTTP request headers for which to collect transaction trace data, uncomment this property, and specify the HTTP request header(s) to track, in a comma-separated list:

#introscope.agent.transactiontracer.parameter.httprequest.headers=User-Agent

2. To specify the HTTP request parameters for which to collect transaction trace data, uncomment this property and specify the HTTP request parameter(s) to track, in a comma-separated list:

#introscope.agent.transactiontracer.parameter.httprequest.parameters=paramete

r1,parameter2

3. To specify the HTTP session attributes for which to trace data, uncomment this property and specify the HTTP session attribute(s) to track, in a comma-separated list, for example:

#introscope.agent.transactiontracer.parameter.httpsession.attributes=cartID,d

eptID


Disabling the capture of stalls as Events


Disabling the capture of stalls as Events

By default, Introscope captures transaction stalls as events in the Transaction Event database, and generates stall metrics from the detected events. Stall metrics are generated for the first and last method in the transaction. Users can view stall Events and associated metrics in the Workstation’s Historical Event Viewer.

Note: Generated stall metrics are always available, but stall events are only visible if Introscope Error Detector is installed. Stalls are stored as ordinary errors, and will be visible in the Errors TypeView, or in the historical query viewer by querying for "type:errorsnapshot".

You can disable the capture of stalls as events, change the stall threshold, or change the frequency with which the agent checks for stalls using these properties:

■ introscope.agent.stalls.thresholdseconds specifies the minimum threshold response time at which time a transaction is considered stalled.

■ introscope.agent.stalls.resolutionseconds specifies the frequency that the agent checks for stalls.

Note: By default, the introscope.agent.stalls.resolutionseconds property is set to 30 seconds. CA Technologies has had great success with this default value and recommends no change unless there is a good reason. Because of timing complications, it is generally not effective to set stall resolution below 15 seconds.


Chapter 11: Configuring the Introscope SQL Agent

This section has instructions for configuring Introscope SQL Agent.


The SQL Agent overview (see page 167) The SQL Agent files (see page 168) SQL statement normalization (see page 168) Turn Off Statement Metrics (see page 176) SQL metrics (see page 176)

The SQL Agent overview

The SQL agent reports detailed database performance data to the Enterprise Manager. The SQL agent provides visibility into the performance of individual SQL statements in your application by tracking the interaction between your managed application and your database.

In the same way that the Java agent monitors applications, the SQL agent monitors SQL statements. The SQL agent is non-intrusive, monitoring the application or database with very low overhead.

To provide meaningful performance measurements down to the individual SQL statement level, the SQL agent summarizes performance data by stripping out transaction-specific data and converting the original SQL statements into Introscope-specific normalized statements. Since normalized statements do not include sensitive information, such as credit card numbers, this process also protects the security of your data.

For example, the SQL agent converts this SQL query:

SELECT * FROM BOOKS WHERE AUTHOR = 'Atwood'

to this normalized statement:

SELECT * FROM BOOKS WHERE AUTHOR = ?

The SQL Agent files


Similarly, SQL agent converts this SQL update statement:

INSERT INTO BOOKS (AUTHOR, TITLE) VALUES ('Atwood', 'The Robber Bride')


INSERT INTO BOOKS (AUTHOR, TITLE) VALUES (?, ?)

Note: Only text within quotation marks ('xyz') is normalized.

Metrics for normalized statements are aggregated and can be viewed in the JDBC node of the Workstation Investigator.

The SQL Agent files

The SQL agent is included in all agent installations. The files providing SQL Agent functionality are:

■ wily/core/ext/SQLAgent.jar

■ wily/core/config/sqlagent.pbd

Note: By default, agent extensions like the SQLAgent.jar file are installed in the wily/core/ext directory. You can change the location of the agent extension directory with the introscope.agent.extensions.directory property in the agent profile. If you change the location of the /ext directory, be sure to move the contents of the /ext directory as well.

SQL statement normalization

Some applications may generate an extremely large number of unique SQL statements. If technologies like EJB 3.0 are in use, the likelihood of long unique SQL statements increases. Long SQL statements can contribute to a metric explosion in the agent, leading to poor performance as well as other system problems.

How poorly written SQL statements create metric explosions

In general, the number of SQL agent metrics should approximate the number of unique SQL statements. If your SQL agent is showing a large and increasing number of unique SQL metrics even though your application uses a small set of SQL statements, the problem could be in how the SQL statement is written.

There are several common situations in which SQL statements can cause metric explosions. For example, SQL statements that include embedded comments, create temporary tables, or insert lists of values can unintentionally create unique metric names each time they execute.



Example: Comments in SQL statements

One common reason for metric explosion is caused by how comments are used in SQL statements. For example, if you have a SQL statement like this:

"/* John Doe, user ID=?, txn=? */ select * from table..."

the SQL agent creates a metric with the comment as part of the metric name:

"/* John Doe, user ID=?, txn=? */ select * from table..."

The comment embedded in the SQL statement is useful for the database administrator to see who is executing what query and the database executing the query ignores them. The SQL agent, however, does not parse the comment string when it captures the SQL statement. Therefore, for each unique user ID, the SQL agent creates a unique metric, potentially causing a metric explosion.

This problem can be avoided by putting the SQL comment in single quotes. For example:

"/*' John Doe, user ID=?, txn=? '*/ select * from table..."

The SQL agent then creates the following metric where the comment no longer causes a unique metric name:

"/* ? */ select * from table..."

Example: Temporary tables or automatically generated table names

Another potential cause of metric explosion can be the result of applications that reference temporary tables or tables that have automatically generated names in SQL statements. For example, if you open the Investigator to view the metrics under Backends|{backendName}|SQL|{sqlType}|sql. you might see a metric similar to this:

SELECT * FROM TMP_123981398210381920912 WHERE ROW_ID = ?

This SQL statement is accessing a temporary table that has a unique identifier appended to the table name. The additional digits that are appended to the TMP_ table name create a unique metric name each time the statement is executed, causing a metric explosion.



Example: Statements that generate lists of values or insert values

Another common cause of metric explosion are SQL statements that generate lists of values or do mass modification of values. For example, assume you have been alerted to a potential metric explosion and your investigation brings you to a review of this SQL statement:

#1 INSERT INTO COMMENTS (COMMENT_ID, CARD_ID, CMMT_TYPE_ID,

CMMT_STATUS_ID,CMMT_CATEGORY_ID, LOCATION_ID, CMMT_LIST_ID, COMMENTS_DSC,

USER_ID,LAST_UPDATE_TS) VALUES (?, ?, ?, ?, ?, ?, ?, "CHANGE CITY FROM CARROLTON,TO

CAROLTON, _ ", ?, CURRENT)

In studying the code, you notice that "CHANGE CITY FROM CARROLTON, TO CAROLTON, _ " generates an array of cities.

Similarly, if you are investigating a potential metric explosion, you might review a SQL statement similar to this:

CHANGE COUNTRY FROM US TO CA _ CHANGE EMAIL ADDRESS FROM TO BRIGGIN @ COM _ "

In studying the code, you notice CHANGE COUNTRY results in a long list of countries. In addition, the placement of the quotes for countries results in people's e-mail addresses getting inserted into SQL statements, creating unique metrics that could be the source of the metric explosion.

SQL statement normalization options

To address long SQL statements, the SQL Agent includes the following normalizers for use:

■ Default SQL statement normalizer (see page 170)

■ Custom SQL statement normalizer (see page 171)

■ Regular expression SQL statement normalizer (see page 172)

■ Command-line SQL statement normalizer (see page 176)

Default SQL statement normalizer

The standard SQL statement normalizer is on by default in the SQL agent. It normalizes text within single quotation marks ('xyz'). For example, the SQL agent converts this SQL query:

SELECT * FROM BOOKS WHERE AUTHOR = 'Atwood'


SELECT * FROM BOOKS WHERE AUTHOR = ?

Metrics for normalized statements are aggregated and can be viewed in the Workstation Investigator.



Custom SQL statement normalizer

The SQL Agent lets you add extensions for performing custom normalization. To do so, you create a JAR file containing a normalization scheme that is implemented by the SQL agent.

To apply a SQL statement normalizer extension

1. Create an extension JAR file.

Note: The entry point class for the SQL normalizer extension file has to implement com.wily.introscope.agent.trace.ISqlNormalizer interface.

Making a JAR extension file involves creating a manifest file that contains specific keys for the SQL normalizer extension, which are detailed in step 2 below. However, for your extension to work, other general keys are required. These keys are the type you would use to construct any extension file. The extension file you create relates to database SQL statement text normalization, for example metrics under the Backends|{backendName}|SQL|{sqlType}|{actualSQLStatement} node. The {actualSQLStatement} is normalized by the SQL normalizer.

2. Place the following keys in the manifest of the created extension:

■ com-wily-Extension-Plugins-List:testNormalizer1

Note: The value of this key can be anything. In this instance, testNormalizer1 is used as an example. Whatever you specify as the value of this key, use it in the following keys as well.

■ com-wily-Extension-Plugin-testNormalizer1-Type: sqlnormalizer

■ com-wily-Extension-Plugin-testNormalizer1-Version: 1

■ com-wily-Extension-Plugin-testNormalizer1-Name: normalizer1

Should contain the unique name of your normalizer, for example normalizer1.

■ com-wily-Extension-Plugin-testNormalizer1-Entry-Point-Class: <Thefully-qualified classname of your implementation of ISQLNormalizer>

3. Place the extension file you created in the <Agent_Home>/wily/core/ext directory.

4. In the IntroscopeAgent.profile, locate and set the following property:

introscope.agent.sqlagent.normalizer.extension

Set the property to the com-wily-Extension-Plugin-{plugin}-Name from your created extension’s manifest file. The value of this property is not case sensitive. For example:

introscope.agent.sqlagent.normalizer.extension=normalizer1

Important! This is a hot property. Changes to the extension name will result in re-registration of the extension.



5. In the IntroscopeAgent.profile, you can optionally add the following property to set the error throttle count:

introscope.agent.sqlagent.normalizer.extension.errorCount

For more information about errors and exceptions, see Exceptions (see page 172).

Note: If the errors thrown by the custom normalizer extension exceed the error throttle count, the extension is disabled.


7. Restart your application.

Exceptions

If the extension you created throws an exception for one query, the default SQL statement normalizer uses the default normalization scheme for that query. When this happens, an ERROR message is logged, saying an exception was thrown by the extension, and a DEBUG message is logged with stack trace information. However, after five such exceptions are thrown, the default SQL statement normalizer disables your created extension and stops attempting to use the created extension for future queries until the normalizer is changed.

Null or empty strings

If the extension you created returns a null string or empty string for a query, the StatementNormalizer uses the default normalization scheme for that query and logs an INFO message saying the extension returned a null value. However, after five such null or empty strings have been returned, the StatementNormalizer stops logging messages, but will attempt to continue to use the extension.

Regular expression SQL statement normalizer

The SQL Agent ships with an extension that normalizes SQL statements based on configurable regular expressions (regex). This file, RegexNormalizerExtension.jar, is located in the <Agent_Home>/wily/core/ext directory.

For examples on how to use the regular expression SQL statement normalizer, see Regular expression SQL statement normalizer examples (see page 174).

To apply the regular expressions extension


2. Locate and set the following properties:

■ introscope.agent.sqlagent.normalizer.extension=RegexSqlNormalizer

Specifies the name of the SQL normalizer extension that will be used to override the preconfigured normalization scheme. When enabling the regular expressions extension, set this property to RegexSqlNormalizer.



■ introscope.agent.sqlagent.normalizer.regex.keys=key1

This property specifies the regex group keys, which are evaluated in the order they are listed. This property is required to enable the regular expressions extension. There is no default value.

■ introscope.agent.sqlagent.normalizer.regex.key1.pattern=A

This property specifies the regex pattern that is used to match against the SQL statements. All valid regular expressions allowed by the java.util.Regex package classes can be used here. This property is required to enable the regular expressions extension. There is no default value.

■ introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat=B

This property specifies the replacement string format. All valid regex allowed by the java.util.Regex package classes can be used here. This property is required to enable the regular expressions extension. There is no default value.

■ introscope.agent.sqlagent.normalizer.regex.matchFallThrough=false

If this property is set to true, SQL strings are evaluated against all the regex key groups. The implementation is chained. Hence, if the SQL strings match multiple key groups, the normalized SQL output from group1 is fed as input to group2, and so on.

If the property is set to false, as soon as a key group matches the SQL string, the normalized SQL output from that group is returned. The MatchFallThrough property does not enable or disable the extension.

For example, if you had a SQL string like: Select * from A where B, you would set the following properties:

introscope.agent.sqlagent.normalizer.regex.keys=key1,key2

introscope.agent.sqlagent.normalizer.regex.key1.pattern=A

introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat=X

introscope.agent.sqlagent.normalizer.regex.key2.pattern=B

introscope.agent.sqlagent.normalizer.regex.key2.replaceFormat=Y

If introscope.agent.sqlagent.normalizer.regex.matchFallThrough is false, then the SQL is normalized against key1 regex. Output from that regex will be Select * from X where B. This SQL will be returned.

If introscope.agent.sqlagent.normalizer.regex.matchFallThrough is true, then the SQL is normalized against key1 regex first. The output from that regex is Select * from X where B. This output is then fed to key2 regex. The output from key2 regex is Select * from X where Y. This will be the SQL returned.

Note: This property is not required to enable the regular expressions extension.

■ introscope.agent.sqlagent.normalizer.regex.key1.caseSensitive=false

This property specifies whether the pattern match is case sensitive. The default value is false. This property is not required to enable the regular expressions extension.



■ introscope.agent.sqlagent.normalizer.regex.key1.replaceAll=false

If this property is set to false, it will replace the first occurrence of the matching pattern in the SQL with the replacement string. If this property is set to true, it will replace all occurrences of the matching pattern in the SQL with the replacement string.

For example, if you have a SQL statement like Select * from A where A like Z, you would set the properties as follows:

introscope.agent.sqlagent.normalizer.regex.key1.pattern=A

introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat=X

If introscope.agent.sqlagent.normalizer.regex.key1.replaceAl is false, it will result in a normalized SQL statement: Select * from X where A like Z.

If introscope.agent.sqlagent.normalizer.regex.key1.replaceAl is true, it will result in a normalized SQL statement: Select * from X where X like Z.

The default value is false. This property is not required to enable the regular expressions extension.

Note: If none of the regular expression patterns match the input SQL, the RegexNormalizer will return a null string. The statement normalizer will then use the default normalization scheme.


Important! All properties listed above are hot, meaning changes to these properties take effect once you have saved the IntroscopeAgent.profile.

Regular expression SQL statement normalizer examples

The three examples below can help you understand how to implement the regular expression SQL statement normalizer.

Example 1

Here is a SQL query before regular expression SQL statement normalization:

INSERT INTO COMMENTS (COMMENT_ID, CARD_ID, CMMT_TYPE_ID,CMMT_STATUS_ID,

CMMT_CATEGORY_ID, LOCATION_ID, CMMT_LIST_ID,COMMENTS_DSC, USER_ID, LAST_UPDATE_TS)

VALUES(?, ?, ?, ?, ?, ?,?, ‘’CHANGE CITY FROM CARROLTON, TO CAROLTON, _ ", ?, CURRENT)

Here is the desired normalized SQL statement:

INSERT INTO COMMENTS (COMMENT_ID, ...) VALUES (?, ?, ?, ?, ?, ?,?, CHANGE CITY FROM

( )



Here is the configuration needed to the IntroscopeAgent.profile file to result in the normalized SQL statement shown above:

introscope.agent.sqlagent.normalizer.extension=RegexSqlNormalizer

introscope.agent.sqlagent.normalizer.regex.matchFallThrough=true

introscope.agent.sqlagent.normalizer.regex.keys=key1,key2

introscope.agent.sqlagent.normalizer.regex.key1.pattern=(INSERT INTO

COMMENTS \$COMMENT_ID,)(.*)(VALUES.*)''(CHANGE CITY FROM \\().*(\$)

introscope.agent.sqlagent.normalizer.regex.key1.replaceAll=false

introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat=$1 ...) $3$4 $5

introscope.agent.sqlagent.normalizer.regex.key1.caseSensitive=false

introscope.agent.sqlagent.normalizer.regex.key2.pattern='[a-zA-Z1-9]+'

introscope.agent.sqlagent.normalizer.regex.key2.replaceAll=true

introscope.agent.sqlagent.normalizer.regex.key2.replaceFormat=?


Example 2

Here is a SQL query before regular expression SQL statement normalization:

SELECT * FROM TMP_123981398210381920912 WHERE ROW_ID =

Here is the desired normalized SQL statement:

SELECT * FROM TMP_ WHERE ROW_ID =

Here is the configuration needed to the IntroscopeAgent.profile file to result in the normalized SQL statement shown above:



introscope.agent.sqlagent.normalizer.regex.keys=key1

introscope.agent.sqlagent.normalizer.regex.key1.pattern=(TMP_)[1-9]*


introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat=$1


Example 3

If you want to normalize a SQL statement like: Select .... ResID1, CustID1 where ResID1=.. OR ResID2=.. n times OR CustID1=.. OR n times, you could set the properties like this:


introscope.agent.sqlagent.normalizer.regex.keys=default,def

introscope.agent.sqlagent.normalizer.regex.default.pattern=(ResID)[1-9]

introscope.agent.sqlagent.normalizer.regex.default.replaceAll=true

introscope.agent.sqlagent.normalizer.regex.default.replaceFormat=$1

introscope.agent.sqlagent.normalizer.regex.default.caseSensitive=true

introscope.agent.sqlagent.normalizer.regex.def.pattern=(CustID)[1-9]

introscope.agent.sqlagent.normalizer.regex.def.replaceAll=true

introscope.agent.sqlagent.normalizer.regex.def.replaceFormat=$1

introscope.agent.sqlagent.normalizer.regex.def.caseSensitive=true

Turn Off Statement Metrics


Command-line SQL statement normalizer

If the regular expression SQL normalizer is not in use, and you have SQL statements that enclose values in the where clause with double quotes (" "), use the following command-line command to normalize your SQL statements:

-DSQLAgentNormalizeDoubleQuoteString=true

Important! You can use the regular expressions SQL normalizer instead of this command to normalize SQL statements in double quotes. See Regular expression SQL statement normalizer (see page 172) for more information.

Turn Off Statement Metrics

Some applications can generate large numbers of unique SQL statements, causing a metric explosion in the SQL agent. You can turn off SQL statement metrics in the SQL agent.

Note: When you turn off statement metrics, the backend or top-level JDBC metrics are not lost.

To turn off statement metrics

1. Open the sqlagent.pbd file and locate the SQL statements. For example:

TraceOneMethodWithParametersIfFlagged: SQLAgentStatements

executeQuery(Ljava/lang/String;)Ljava/sql/ResultSet; DbCommandTracer

"Backends|{database}|SQL|{commandtype}|Query|{sql}"

2. Remove {sql} from the trace directives you want to turn off. For example:

TraceOneMethodWithParametersIfFlagged: SQLAgentStatements executeQuery(Ljava/

lang/String;)Ljava/sql/ResultSet; DbCommandTracer

"Backends|{database}|SQL|{commandtype}|Query"

3. Save the sqlagent.pbd file.

The SQL statement metrics in the SQL agent are turned off.

SQL metrics

The SQL Agent metrics appear under the Backends node in the Introscope Workstation Investigator. SQL statement metrics can be found under the Backends|<backendName>|SQL node.

Note: Average Response Time (ms) will only display queries that return a data reader, i.e. queries executed via the ExecuteReader() method. This metric represents the average time spent in the data reader’s Close() method.

SQL metrics


Metric types specific to SQL data include:

■ Connection Count—The number of live connection objects in memory.

A connection is opened when a driver’s connect() method is invoked, and closed when the connection invocation is closed via the close() method. The SQL Agent maintains weak references to Connections in a Set. When the Connection objects are garbage collected, the counts reflect the changes.

■ Average Result Processing Time (ms)—The average processing time of a query.

This metric represents the average time spent processing a ResultSet from the end of the executeQuery() call to the invocation of the ResultSet's close() method.

Note: Instrumented XADataSources may not report commit or rollback metrics. Other instrumented DataSources may not report commit or rollback metrics unless those metrics contain data.


Chapter 12: Enabling JMX Reporting

This section contains information about enabling the Java agent to report JMX data.


Java Agent JMX support (see page 179) Default JMX metric conversion process (see page 180) Using primary key conversion to streamline JMX metrics (see page 181) Managing metric volume with JMX filters (see page 182) Configuring JMX reporting (see page 183) Enabling JSR-77 data for WAS 6.x (see page 184)

Java Agent JMX support

Introscope can collect management data that application servers or Java applications expose as JMX-compliant MBeans, and present the JMX data in the Investigator metric tree. Introscope supports the collection of JMX information on the following application servers:

■ Glassfish

■ JBoss

■ Tomcat

■ WebLogic

■ WebSphere

Introscope supports any MBean built to the Sun JMX specification. For more information on the Sun JMX specification, see Java Management Extensions.

Introscope converts the JMX data to the Introscope metric format and displays it in the Investigator under the following node:

<Domain>|<Host>|<Process>|AgentName|JMX|



Default JMX metric conversion process


Introscope support for WebLogic 9.0 JMX metrics

WebLogic versions prior to WebLogic 9.0 provided only a single MBeanServer as a source of JMX metrics. WebLogic 9.0 provides three:

■ RuntimeServiceMBean: per-server runtime metrics, including active effective configuration

■ DomainRuntimeServiceMBean: domain-wide runtime metrics

■ EditServiceMBean: allows user to edit persistent configuration

Introscope polls only the RuntimeServiceMBean, because it is the only one that supports local access (an efficiency issue), and because it contains most of the data expected to be relevant.

Default JMX metric conversion process

This section describes the process Introscope uses, by default, to convert JMX Metrics for display in the Investigator. This method is used to convert an MBean if:

■ You use WebLogic 9.0, or

■ You have not configured valid primary keys, as described in Using primary key conversion to streamline JMX metrics (see page 181).

Note: If you specify primary keys that no MBeans match, Introscope will use the default conversion method.

In the default conversion method, Introscope displays both the name and the value of the attribute, and lists the pairs alphabetically in the metric tree.

Domain>|<Host>|<Process>|AgentName|JMX|<domain name>| <key1>=<value1>|<key2>=<value2>:<metric>

For example, given a WebLogic MBean with these characteristics:

■ Domain name: WebLogic

■ Key/Value pairs: category=server, type=jdbc

■ Metric names: connections

If no primary keys are specified in the IntroscopeAgent.profile property introscope.agent.jmx.name.primarykeys, the MBean attributes above would be converted to the following Introscope metric:

<Domain>|<Host>|<Process>|AgentName|JMX|Weblogic|category=server|type=jdbc:connections

Note that the key/value pairs are displayed alphabetically in the Introscope metric.

Using primary key conversion to streamline JMX metrics


Using primary key conversion to streamline JMX metrics

You can optionally configure the order in which metrics appear under the JMX node by defining, in the agent profile, a Primary Key—those parts of an MBean’s ObjectName that uniquely identify it.

If you do not configure primary key conversion, Introscope converts the JMX data as described in Default JMX metric conversion process (see page 180). With the default conversion, metrics are listed alphabetically under the JMX node in Investigator.

This method of converting JMX data to Introscope metrics results in streamlined metric names, and allows you to control order of key/value pair information in the generated metrics.

The behavior is configured in the introscope.agent.jmx.name.primarykeys property in the agent profile. Values in the primarykeys property should specify the parts of an MBeans JMX ObjectName that uniquely identify an MBean. For example, a WebLogic MBean’s ObjectName contains a Type key that specifies the kind of MBean, and a Name key that specifies the name of the resource the MBean represents. The key/value pairs in an ObjectName can vary for different types of MBeans.

Introscope converts and presents the MBeans identified by value of the introscope.agent.jmx.name.primarykeys property according to these rules:

■ Only the key value information is displayed, not the key name.

■ Values are ordered in the sequence defined in the primarykeys property.

■ Values are case-sensitive.

For example, given a WebLogic MBean with these characteristics:

■ Domain name: WebLogic

■ MBean ObjectName key/value pairs: category=server, type=jdbc

■ Metric names: connections

If you configure:

introscope.agent.jmx.name.primarykeys=type,category the connections attribute appears in the Investigator tree in this structure:

<IntroscopeDomain>|<Host>|<Process>|AgentName|JMX|Weblogic|jdbc|server:connection

s

Note: WebLogic 9.0 does not have universally available primary keys, so for WebLogic 9.0 Introscope uses the key/value pair metric naming convention found in the Default Conversion Method described in Default JMX metric conversion process (see page 180). As a result, the JMX Metric tree for WebLogic 9.0 will have a different structure than the metric tree for other WebLogic versions.

Managing metric volume with JMX filters


Managing metric volume with JMX filters

Defining JMX filters determines what JMX MBean information is collected and displayed in Introscope. If no filters are set, all JMX MBean information is reported by the agent to the Enterprise Manager, increasing system overhead.

Filters are set in the introscope.agent.jmx.name.filter property in the agent profile, IntroscopeAgent.profile. Filters are keywords, entered as comma-separated strings in the property. Introscope 6.1 and higher supports filter strings that contain the asterisk (*) and question mark (?) wildcard characters.

Introscope matches the filter strings to JMX-generated Introscope metrics. If it finds a match, the metrics that match are reported to Introscope.

To limit the volume of metrics returned, define filter strings as narrowly as possible. For instance, if you define a filter string that matches an MBean attribute that exists on multiple MBeans, metrics from each of those MBeans are reported. If you are only interested in an attribute on selected MBeans, you can qualify the attribute name with the MBean name in your filter string.

For example, assume you want to capture the MessagesCurrentCount attribute value for the JMSDestinationRuntime MBean.

If the fully qualified metric name for MessagesCurrentCount is:

*SuperDomain*|host-name|Process|Agent-name|JMX|comp-1|

JMSDestinationRuntime|comp-2:MessagesCurrentCount

define introscope.agent.jmx.name.filter in the IntroscopeAgent.profile as:

JMX|comp-1|JMSDestinationRuntime|comp-2:MessagesCurrentCount

JMX filters for WebLogic

In the IntroscopeAgent.profile file for WebLogic, the following keywords are already defined:

■ ActiveConnectionsCurrentCount

■ WaitingForConnectionCurrentCount

■ PendingRequestCurrentCount

■ ExecuteThreadCurrentIdleCount

■ OpenSessionsCurrentCount

Configuring JMX reporting


Configuring JMX reporting

How you configure Introscope to support JMX depends upon the application server you use. This section describes how to configure Introscope to collect and present JMX data from WebLogic Server and WebSphere.

To configure JMX reporting, complete the following steps in this order:

1. Enable JMX support in the agent profile.

2. Define primary keys for JMX data conversion.

3. Define JMX filters.

4. Configure startup class or service.

Enable JMX support in the agent profile

1. Shut down the managed application if it is running.

2. For WebSphere agents only, in IntroscopeAgent.profile set introscope.agent.jmx.enable to true. (The default value is false in the WebSphere agent profile.)

Define primary keys for JMX data conversion

In IntroscopeAgent.profile, configure primary keys.

■ For WebLogic 9.0, comment out:

introscope.agent.jmx.name.primarykeys

■ For other WebLogic versions, uncomment:


1. If you modify the value of the property, the values must be case-sensitive, and multiple keys must be separated by commas.

2. Continue to the next step, Define JMX filters.

Define JMX filters

Note: Filters must include an MBean attribute, such as version number. For example, given the complete metric name:

*SuperDomain*|MyServer01|WebSphere|LODVMAPM032Node02Cell/server1|JMX|WebSpher

e|cell=LODVMAPM032Node02Cell|mbeanIdentifier=default_host/hello|name=hello-2_

1_1_2_war#hello-2.1.1.2.war|node=LODVMAPM032Node02|platform=dynamicproxy|proc

ess=server1|spec=1.0|type=SessionManager|version=6.1.0.0|StatsImpl|LiveCount:

HighWaterMark

Enabling JSR-77 data for WAS 6.x


A filtered version containing the version number attribute would be:

PlantsByWebSphere|name=PlantsByWebSphere#PlantsByWebSphere.war|node=LODVMAPM0

33Node01|platform=dynamicproxy|process=server1|spec=1.0|type=SessionManager|v

ersion=6.1.0.0

or even more simply:

Plant*re|*version*

Use the following steps to define JMX filters:

1. In IntroscopeAgent.profile, make sure the introscope.agent.jmx.name.filter property is uncommented.

2. Enter desired strings, separated by commas, in the property.

For Introscope to match filtered strings, the strings must be spelled exactly and case sensitive.

3. Save changes.


Configure startup class or service

■ To enable JMX data, configure an Introscope startup class in WebLogic Server, or a custom service in WebSphere. For instructions, see Configuring a startup class for WebLogic (see page 42), or Configuring a custom service in WebSphere (see page 51).


This section provides instructions for configuring Introscope to collect, retain, and report metrics for JSR-77 JMX MBean objects under WebSphere 6.1 and later.

JSR-77, the J2EE Management Specification, abstracts the manageable parts of the J2EE architecture and defines an interface for accessing management information.

JSR-77 support requires a JVM version 1.4 or later. If the JVM is 1.4, the application server must also support JSR-77.

To enable JSR-77 support

1. Shut down the managed application if it is running.

2. Configure a WebSphere Custom Service, as described in Configuring a custom service in WebSphere 6.1 (see page 51).

3. In the IntroscopeAgent.profile, verify that:

introscope.agent.jmx.enable=true

4. In the IntroscopeAgent.profile, enable JSR-77 by setting:

introscope.agent.jmx.name.jsr77.disable=false



5. Configure the primary keys method of metric conversion by uncommenting this property in the IntroscopeAgent.profile:

introscope.agent.jmx.name.primaryKeys=J2EEServer,Application,

j2eeType,JDBCProvider,name,mbeanIdentifier

Note: Only the IntroscopeAgent.profile provided with Introscope for WebSphere contains this property definition.

For more information see Using primary key conversion to streamline JMX metrics (see page 181).

6. To specify the JSR-77 Metrics to report, uncomment and set this property to identify desired metrics:

introscope.agent.jmx.name.filter

Although filtering is not required, it is highly recommended. For more information see Managing metric volume with JMX filters (see page 182).

7. To specify specific Mbean attributes to exclude in JSR-77 metrics, uncomment this property, and update as desired to exclude additional attributes:

introscope.agent.jmx.ignore.attributes=server


Chapter 13: Configuring Platform Monitoring

This section has instructions for configuring Introscope Platform Monitors.


Understanding platform monitors (see page 187) Enabling platform monitors on Windows Server 2003 (see page 189) Enabling platform monitors on AIX (see page 189) Disabling platform monitors (see page 190) Configure Permissions to Access Platform Monitors on HP-UX (see page 191) Troubleshooting platform monitoring (see page 191)

Understanding platform monitors

Platform monitors enable the Java agent to report system metrics, including CPU statistics, to the Enterprise Manager. Platform monitors are included with the Introscope agent installers.

Platform monitors on all operating systems except Windows Server 2003 and AIX are automatically enabled upon Java agent installation. Windows Server 2003 and AIX platform monitors require minimal configuration to work.

Note: Platform monitor binaries are independent of application server and operating system bit modes. Also, platform monitor binaries are purely dependent on JVM architecture.

Introscope can monitor CPU usage on these operating systems:

■ Solaris

Important! Platform monitoring is not supported on 64 bit Solaris using 64 bit BEA JRockit JVM.

■ Windows Server 2003

■ Windows 2000 Professional/Server/Advanced Server/Datacenter Server

■ Windows XP Professional

Important! Introscope does not monitor platform monitors on 64-bit Windows XP Professional.

Understanding platform monitors


■ AIX 4.0, 5.0, or 6.0

Important! AIX 5.3 64 bit Platform Monitor binaries work only on AIX machines which have Technology Level equal or greater than TL6 (5300-06). You should ensure that your AIX machine is no less than TL6.

■ RedHat 3.0, 4.0, or 5.0

■ SUSE Linux 9.0, or 10.0

■ HP-UX

Note: HP-UX Platform Monitor binaries (both PA-RISC and IA 64) work on 32 bit kernel with 32 bit JVM.

HP-UX Platform Monitor binaries (both PA-RISC and IA 64) work on 64 bit kernel with 32 bit JVM. (HP-UX binaries support 64 bit OS but not 64 bit JVM).

The Java Agent platform monitor for HP-UX reports the percentage use of CPUs when more than one process is present. For example, if you have 4 processes, the maximum use of the CPU would be 400% (4 processes using 100% of the CPU). If one process takes 110%, this means it is using 1.1 CPUs.

The following JVMs are supported by platform monitors:

■ Sun JVM

■ HP Hotspot JVM

■ IBM JVM (Both Classic and J9)

■ BEA JRocket JVM

Note: 32-bit Platform Monitor binaries can be installed on a 64-bit machine, provided a 32-bit JVM is installed.

The platform metrics generated are:

■ ProcessID

■ Processor Count - the number of CPUs

■ Utilization % (process) - for the Java Agent process, the percentage of total capacity of all processors this process is using. Regardless of how many processors there are, this metric generates only one number.

■ Utilization % (aggregate) - for this processor, its total utilization (as a percentage) by all processes in the system. Each processor is shown as a Resource in the Investigator tree.

Enabling platform monitors on Windows Server 2003


Enabling platform monitors on Windows Server 2003

To run platform monitors on Windows Server 2003, you must have admin privileges.

Note that system objects must be enabled for platform monitoring to work on Windows.

To determine whether system objects are enabled

1. Go to Start > Run.

2. Type perfmon and click Run.

3. In the dialog box click Add.

4. In the Add dialog, if "Process" and "Processor" performance objects are present in the drop down list box, then system objects are enabled.

To enable system objects

1. Go to Start > Accessories > Right click on command prompt > Run as… > Administrator.

2. Run the command: lodctr /r

"Process" and "Processor" objects will be enabled.

Enabling platform monitors on AIX

To enable platform monitors on AIX

1. After Java agent installation, verify that the following files are installed in the wily/core/ext directory:

■ introscopeAIXPSeries32Stats.jar


■ libIntroscopeAIXPSeries32Stats.so


2. Install the Perfstat Library.

■ AIX 4.3.3 and higher: A Perfstat Library has been created to work with AIX 4.3.3. Install the following packages from the IBM FTP site:

■ bos.perf.libperfstat

■ bos.perf.perfstat

■ AIX 4: Bring your system up to 4.3.3 and then install the above packages.

3. Restart your machine to ensure the patches have taken effect.

Disabling platform monitors


Disabling platform monitors

To disable platform monitors on any platform, move the .jar file from the /wily/core/ext directory to another directory.

This table shows the location of platform monitor files installed with a Java agent installer. All files listed below are located in the <Agent_Home>wily/core/ext/ directory.

Solaris

The platform monitor files are:

■ introscopeSolarisAmd32Stats.jar

■ introscopeSolarisAmd64Stats.jar

■ introscopeSolarisSparc32Stats.jar

■ introscopeSolarisSparc64Stats.jar

■ libIntroscopeSolarisAmd32Stats.so

■ libIntroscopeSolarisAmd64Stats.so

■ libIntroscopeSolarisSparc32Stats.so

■ libIntroscopeSolarisSparc64Stats.so

Windows Server 2003, Windows 2000 (all), and XP Professional


■ introscopeWindowsIntelAmd32Stats.dll

■ introscopeWindowsIntelAmd64Stats.dll

■ introscopeWindowsIntelAmd32Stats.jar

■ introscopeWindowsIntelAmd64Stats.jar

AIX






RedHat Enterprise Linux


■ introscopeRedHatStats.jar

■ libIntroscopeRedHatStats.so

Configure Permissions to Access Platform Monitors on HP-UX


Linux


■ introscopeLinuxIntelAmd32Stats.jar

■ introscopeLinuxIntelAmd64Stats.jar

■ libIntroscopeLinuxIntelAmd32Stats.so

■ libIntroscopeLinuxIntelAmd64Stats.so

HP-UX


■ introscopeHpuxItaniumStats.jar

■ introscopeHpuxParisc32Stats.jar

■ introscopeHpuxItanium64Stats.jar

■ libIntroscopeHpuxItaniumStats.so

■ libIntroscopeHpuxParisc32Stats.so

■ libIntroscopeHpuxItanium64Stats.so

Configure Permissions to Access Platform Monitors on HP-UX Installing the platform monitor on HP-UX using .zip or .tar installers requires administrators to change read-write permissions on the following files to 777:

wily/ext/introscopeHpuxItaniumStats.jar

wily/ext/introscopeHpuxItaniumStats.so

wily/ext/introscopeHpuxParisc32Stats.jar

wily/ext/introscopeHpuxParisc32Stats.so

Example (root user):

chmod 777 /<Agent_Home>/wily/ext/introscopeHpuxItaniumStats.jar

Make these changes after installing the agent and before starting it.

If you are using the first two files named above, you also must install gcc before starting the agent. You can download gcc, a compiler for C and C++, by searching the HP support website.

Troubleshooting platform monitoring

In most cases, the platform monitor successfully detects the operating system and runs if the operating system is supported. In rare cases where this does not occur, you can explicitly specify your operating system in the Java agent profile to ensure that the platform monitor runs.

Troubleshooting platform monitoring


To specify your operating system in the IntroscopeAgent.profile

1. Open IntroscopeAgent.profile.

2. Under the Platform Monitor Configuration heading, locate the exact matching value for your operating system and uncomment the property. The available values are:

#introscope.agent.platform.monitor.system=SolarisAmd32

#introscope.agent.platform.monitor.system=SolarisAmd64

#introscope.agent.platform.monitor.system=SolarisSparc32

#introscope.agent.platform.monitor.system=SolarisSparc64

#introscope.agent.platform.monitor.system=AIXPSeries32

#introscope.agent.platform.monitor.system=AIXPSeries64

#introscope.agent.platform.monitor.system=HP-UXItanium

#introscope.agent.platform.monitor.system=HP-UXParisc32

#introscope.agent.platform.monitor.system=WindowsIntelAmd32

#introscope.agent.platform.monitor.system=WindowsIntelAmd64

#introscope.agent.platform.monitor.system=LinuxIntelAmd32

#introscope.agent.platform.monitor.system=LinuxIntelAmd64


Troubleshooting platform monitoring on Windows

On Windows platforms, the following error can appear for the Java agent:

11/28/06 08:29:55 AM PST [ERROR] [IntroscopeAgent] An error occurred polling for

platform data

If the error is infrequent, it is likely caused by a transient error originating from Windows itself, and is harmless. On platforms other than Windows, or in the case that the error happens all the time, this error indicates something more serious and should be reported to CA Support for Introscope.

SAP Netweaver

Platform monitors may not function correctly on SAP Netweaver due to the lack of permissions for SAP user accounts.

By default, SAP user accounts are not registered under Performance Monitor Users, which is located by navigating to Computer Management > System Tools > Local Users and Groups > Groups > Performance Monitor Users . Users registered under Performance Monitor Users have access to Perfmon related data (HKEY_PERFORMANCE_DATA).

If you are experiencing problems with SAP Netweaver and Introscope performance monitoring, this issue can be resolved by adding SAP user account to the Performance Monitor Users group, then restarting the Windows machine.


Appendix A: Java Agent Properties


Configuring the IntroscopeAgent.profile location (see page 194) Command-line property overrides (see page 195) Agent failover (see page 196) Agent HTTP tunneling (see page 198) Agent memory overhead (see page 201) Agent metric aging (see page 202) Agent metric clamp (see page 206) Agent naming (see page 207) Agent recording (business recording) (see page 212) Agent thread priority (see page 213) Agent to Enterprise Manager connection (see page 214) Application triage map (see page 216) Application Triage Map and Catalyst Integration (see page 219) Application triage map business transaction POST parameters (see page 221) Application triage map managed socket configuration (see page 223) AutoProbe (see page 226) Bootstrap Classes Instrumentation Manager (see page 228) CA CEM Agent Profile Properties (see page 229) ChangeDetector configuration (see page 235) Cross-process tracing in WebLogic Server (see page 238) Cross-process transaction trace (see page 239) Dynamic instrumentation (see page 240) ErrorDetector (see page 243) Extensions (see page 244) GC Monitor (see page 245) Java NIO (see page 246) JMX (see page 252) LeakHunter (see page 256) Logging (see page 261) Metric count (see page 266) Multiple inheritance (see page 267) Platform monitoring (see page 270) Remote configuration (see page 271) Security (see page 272) Servlet header decorator (see page 273) Socket metrics (see page 273) SQL Agent (see page 275) SSL communication (see page 282) Stall metrics (see page 285) Thread dumps (see page 287) Transaction tracing (see page 289) URL grouping (see page 299) WebSphere PMI (see page 301)

Configuring the IntroscopeAgent.profile location


WLDF metrics (see page 312)

Configuring the IntroscopeAgent.profile location

The agent refers to properties in the IntroscopeAgent.profile for its basic connection and naming properties. When you install an agent, the agent profile is installed in the <Agent_Home>/wily/core/config directory.

Introscope looks for the agent profile in these locations, in this sequence:

■ location defined in the system property com.wily.introscope.agentProfile

■ location defined in com.wily.introscope.agentResource

■ <working directory>/wily/core/config directory

Note: When adding a path on a Windows computer, escape the backslash (\) with another backslash as follows: C:\\Introscope\\lib\\Agent.jar.

To change the location of the IntroscopeAgent.profile

1. Define the new location using one of these methods:

■ define a system property on the Java command line with the -D option to specify the full path to the location of the IntroscopeAgent.profile file:

■ -D com.wily.introscope.agentProfile.

■ Make the IntroscopeAgent.profile available in a resource on the classpath. Set com.wily.introscope.agentResource to specify the path to the resource containing the agent profile.

Note: If you change the location of the IntroscopeAgent.profile, the AutoProbe log location will also have to be changed. For more information, see Managing ProbeBuilder Logs (see page 135).

2. Move your ProbeBuilder directives (PBD and PBL files) to the same location as the agent profile—they are referenced relative to the profile location.

Command-line property overrides


If you use Sun ONE, then add the new location of the agent profile to the Sun ONE server.xml file

To change the location of the IntroscopeAgent.profile for Sun ONE

1. To add Introscope information to the startup scripts for Sun ONE 7.0, log in as Administrator or Root.

2. Open the server.xml file, in <SunOne_Home>/domains/domain1/server1/config/

3. Add a line to the jvm-options stanza in server.xml:

<jvm-options>

-Dcom.wily.introscope.agentProfile=SunOneHome/wily/core/config/IntroscopeAgen

t.profile

</jvm-options>

Command-line property overrides

In Introscope, you can override specific properties of the Enterprise Manager, agents, Workstation, and WebView using the command line. With regard to the Java agent, this is useful when you have a clustered environment with multiple copies of an agent being shared and you want to tailor some of the agent settings for each application being monitored.

These steps assume you have installed and configured an agent on the application server to be monitored, and that the agent successfully connects to the Enterprise Manager.

To override agent properties using the command line

1. Open the file where you modified the Java command to start the agent.

The location of this file varies depending on the application server you use in your environment.

2. Add a -D command to override a property. For example, you can add the following command to make the agent also use the weblogic-full.pbl file:

-Dintroscope.autoprobe.directivesFile=weblogic-full.pbl

Place this command next to other -D commands in the open file.

Note: When you use this command to override hot deployable properties, the property is no longer hot deployable. Also, if you modify the property at a later time in the configuration file, you will receive a warning message in the Workstation stating you modified an overridden property and your change will have no effect. To avoid this, remove the override command before modifying the property in a configuration file.

Agent failover


3. Save the file.

4. Restart the agent.

In the example used above, you would now see the additional WebLogic metrics in the agent node in the Workstation.

Important! System properties become part of the property space of Introscope properties, allowing things like java.io.tmpdir to be visible to anything using IndexedProperties.

Agent failover

If the Java agent loses connection with its primary Enterprise Manager, the Agent failover properties specify which Enterprise Manager the agent will failover to, and how often it will try to reconnect to its primary Enterprise Manager.

introscope.agent.enterprisemanager.connectionorder

Specifies the connection order of backup Enterprise Managers the agent uses if it is disconnected from its default Enterprise Manager.

Property settings

Names of other Enterprise Managers the agent can connect to.

Default

The Enterprise Manager defined by the DEFAULT host name, port number, and socket factory properties.

Example

introscope.agent.enterprisemanager.connectionorder=DEFAULT

Notes

■ Use a comma separated list.

■ You must restart the managed application before changes to this property take effect.

Agent failover


introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds

Specifies the number of seconds between attempts by a denied agent to reconnect to these Enterprise Managers:

■ Enterprise Managers based on order configured in the agent profile introscope.agent.enterprisemanager.connectionorder property value.

■ Any Enterprise Manager allowed based on loadbalancing.xml configuration.

If an agent cannot connect to an Enterprise Manager, it handles connections in these ways:

■ Tries to connect to the next allowed Enterprise Manager.

■ Does not connect to any Enterprise Manager on which it is disallowed.

Note: For information about configuring loadbalancing.xml and agent to Enterprise Manager connections, see the CA APM Configuration and Administration Guide.

Default

The default interval is 120 seconds.

Example

introscope.agent.enterprisemanager.failbackRetryIntervalInSeconds=120

Notes

Restart the managed application so that changes to this property take effect.

This property is commented out by default.

This property is useful for environments where an agent is allowed to connect across the following CA APM components:

■ Clusters.

■ Collectors and Standalone Enterprise Managers.

■ Any combination of clusters, Collectors, and Standalone Enterprise Managers.

If an agent can connect to Enterprise Managers in different clusters and this property is not set, then the following example shows what occurs:

1. An agent connected to an Enterprise Manager in Cluster disconnects.

2. The agent connects to an Enterprise Manager in Cluster 2 in disallowed mode.

Agent HTTP tunneling


3. The agent does not know when the allowed Enterprise Manager in Cluster 1 becomes available.

This property enforces the agent to keep trying to connect to its allowed Enterprise Managers until an Enterprise Manager is available for connection.


You can configure agents to send information using tunneling technology, enabling agents to connect to an Enterprise Manager remotely. To do this, the agent must be configured to connect to the Enterprise Manager’s embedded Web server, where the HTTP tunneling Web service is hosted.

To configure HTTP tunneled communication in IntroscopeAgent.profile as a new agent connection, specify:

■ The host name of machine running the Enterprise Manager. For more information, see introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT (see page 200).

■ The connection port to the Enterprise Manager Web server. This is the value for the introscope.enterprisemanager.webserver.port property specified in the IntroscopeEnterpriseManager.properties for the Enterprise Manager to which the agent will connect.

■ The HTTP tunneling socket factory. Specify this client socket factory:

com.wily.isengard.postofficehub.link.net.HttpTunnelingSocketFactory

Agent HTTP tunneling for proxy servers

The following properties only apply to agents configured to tunnel over HTTP and must connect to an Enterprise Manager using a proxy server:

■ introscope.agent.enterprisemanager.transport.http.proxy.host (see page 199)

■ introscope.agent.enterprisemanager.transport.http.proxy.port (see page 199)

■ introscope.agent.enterprisemanager.transport.http.proxy.username (see page 199)

■ introscope.agent.enterprisemanager.transport.http.proxy.password (see page 200)

For more information, see Configuring a proxy server for HTTP tunneling (see page 59).



introscope.agent.enterprisemanager.transport.http.proxy.host

Specifies the proxy server host name.

Default

Commented out; not specified.

Notes


introscope.agent.enterprisemanager.transport.http.proxy.port

Specifies the proxy server port number.

Default


Notes


introscope.agent.enterprisemanager.transport.http.proxy.username

If the proxy server requires the agent to authenticate it, specifies the user name for authentication.

Default


Notes




introscope.agent.enterprisemanager.transport.http.proxy.password

If the proxy server requires the agent to authenticate it, specifies the password for authentication.

Default


Notes


Agent HTTPS tunneling

You can configure agents to send information using HTTPS, enabling agents to connect to an Enterprise Manager remotely:

■ introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT (see page 200)

■ introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT (see page 201)

■ introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT (see page 201)

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT

Specifies the host name of the computer running the Enterprise Manager that the agent connects to by default.

Default

localhost

Example

introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT=localhost

Notes


Agent memory overhead


introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT

Specifies the port number on the computer that hosts the Enterprise Manager. If you are using HTTPS tunneling, the default port that listens for connections from the agent is 8444. This property is commented out by default.

Default

8444

Example


Notes


introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT

Specifies the client socket factory to use for connections from the agent to the Enterprise Manager when using HTTPS.

Default

com.wily.isengard.postofficehub.link.net.HttpsTunnelingSocketFactory

Example

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wily.i

sengard.postofficehub.link.net.HttpsTunnelingSocketFactory

Notes


Agent memory overhead

Significant agent memory overhead only occurs in certain extreme cases. The typical trade-off for lowering memory consumption is a potential increase in response time. However, each application is unique and the trade-off between memory usage and response time can vary depending on the application itself.

Agent metric aging


introscope.agent.reduceAgentMemoryOverhead

Specifies the agent configuration to use. Uncomment if you want to attempt to reduce the agent memory overhead by using architectural improvements to the 8.x agent. This property is not valid for older versions of the agent. This property is set to true and commented out by default.

Property settings

True or False

Default

True

Example

introscope.agent.reduceAgentMemoryOverhead=true

Notes


Agent metric aging

Agent metric aging periodically removes dead metrics from the agent memory cache. A dead metric is a metric that has no new data reported in a configured amount of time. Removing old metrics helps to improve agent performance and avoid potential metric explosions.

Note: A metric explosion happens when an agent is inadvertently set up to report more metrics than the system can handle. When too many metrics are reported, the agent can affect the performance of the application server, or in extreme cases, prevent the server from functioning at all.

Metrics that are in a group are removed only if all metrics in the group are considered candidates for removal. Currently, only BlamePointTracer and MetricRecordingAdministrator metrics are removed as a group. Other metrics are removed individually.

Agent metric aging


The MetricRecordingAdministrator has the following interfaces for creating, retrieving, or removing a metric group:

■ getAgent().IAgent_getMetricRecordingAdministrator.addMetricGroup

String component, collection metrics. The component name is the metric resource name of the metric group. The metrics must be under the same metric node to qualify as a group. The metrics are a collection of com.wily.introscope.spec.metric.AgentMetric data structures. You can only add AgentMetric data structures to this collection.

■ getAgent().IAgent_getMetricRecordingAdministrator.getMetricGroup

String component. Based on the component name which is the metric resource name, you can get the Collection of metrics.

■ getAgent().IAgent_getMetricRecordingAdministrator.removeMetricGroup

String component. The metric group is removed based on the component which is the metric resource name.

■ getAgent().IAgent_getDataAccumulatorFactory.isRemoved

Checks if the metric is removed. You use this interface if you keep an instance of an accumulator in your extension. If the accumulator is removed because of metric aging, you use this interface to prevent holding onto a dead reference.

Important! If you create an extension that uses a MetricRecordingAdministrator interface (for example, for use with other CA Technologies products), be sure to delete your own instance of an accumulator. If a metric is aged out because it has not been invoked, and data later become available for that metric, the old accumulator instance will not create new metric data points. To avoid this situation, do not delete your own instance of an accumulator and use instead the getDataAccumulatorFactory interface.

Configuring agent metric aging

Agent metric aging is on by default. You can choose to turn off this capability using the property introscope.agent.metricAging.turnOn (see page 204). If you remove this property from the IntroscopeAgent.profile, agent metric aging is turned off by default.

Agent metric aging runs on a heartbeat in the agent. The heartbeat is configured using the property introscope.agent.metricAging.heartbeatInterval (see page 205). Be sure to keep the frequency of the heartbeat low. A higher heartbeat will impact the performance of the agent and CA Introscope.

Agent metric aging


During each heartbeat, a certain set of metrics are checked. This is configurable using the property introscope.agent.metricAging.dataChunk (see page 205). It is also important to keep this value low, as a higher value will impact performance. The default value is 500 metrics to be checked per heartbeat. Each of the 500 metrics is checked to see if it is a candidate for removal. For example, if you set this property to check chunks of 500 metrics per heartbeat, and you have a total of 10,000 metrics in the agent memory, then it will take longer with lower impact on performance to check all 10,000 metrics. However, if you set this property to a higher number, you would check all 10,000 metrics faster, but with possibly high overhead.

A metric is a candidate for removal if the metric has not received new data after certain period of time. You can configure this period of time using the property introscope.agent.metricAging.numberTimeslices (see page 205). This property is set to 3000 by default. If a metric meets the condition for removal, then a check is performed to see if all the metrics in its group are candidates for metric removal. If this requirement has also been met then the metric is removed.

introscope.agent.metricAging.turnOn

Turns on or off agent metric aging.

Property settings

True or False

Default

True

Example

introscope.agent.metricAging.turnOn=true

Notes


Agent metric aging


introscope.agent.metricAging.heartbeatInterval

Specifies the time interval when metrics are checked for removal, in seconds.

Default

1800

Example

introscope.agent.metricAging.heartbeatInterval=1800

Notes


introscope.agent.metricAging.dataChunk

Specifies the number of metrics that are checked during each interval.

Default

500

Example

introscope.agent.metricAging.dataChunk=500

Notes


introscope.agent.metricAging.numberTimeslices

Specifies the number of intervals to check without any new data before making it a candidate for removal.

Default

3000

Example

introscope.agent.metricAging.numberTimeslices=3000

Notes


Agent metric clamp


introscope.agent.metricAging.metricExclude.ignore.0

Excludes specified metrics from being removed. To exclude one or more metrics from aging, add the metric name or a metric filter to the list.

Property settings

Comma separated list of metrics. You can use an asterisk (*) as a wildcard in metric names.

Default

The default is metric names beginning with Threads (Threads*).

Example

introscope.agent.metricAging.metricExclude.ignore.0=Threads*

Notes


Agent metric clamp

You can configure the agent to approximately clamp the number of metrics sent to the Enterprise Manager. If the number of metrics generated exceeds the value of the property, the agent stops collecting and sending new metrics.

introscope.agent.metricClamp

Configures the agent to approximately clamp the number of metrics sent to the Enterprise Manager.

Default

5000

Example

introscope.agent.metricClamp=5000

Notes

■ If the property is not set, then no metric clamping occurs. Old metrics will still report values.

Agent naming


■ Changes to this property take effect immediately and do not require the managed application to be restarted.

■ This clamp property works with the introscope.enterprisemanager.agent.metrics.limit property located in the apm-events-thresholds-config.xml file.

Note: For information about the introscope.enterprisemanager.agent.metrics.limit property, see the CA APM Configuration and Administration Guide.

If introscope.enterprisemanager.agent.metrics.limit clamp value is triggered before the introscope.agent.metricClamp value, then the Enterprise Manager reads agent metrics but does not report them in the Investigator metric browser tree.

If the introscope.agent.metricClamp clamp value is triggered before the introscope.enterprisemanager.agent.metrics.limit clamp value, the agent stops sending metrics to the Enterprise Manager

Agent naming

You can configure properties to obtain the Java agent name for application servers and much more.

More information:

Understanding the Java Agent name (see page 117)

Agent naming


introscope.agent.agentAutoNamingEnabled

Specifies whether agent autonaming is used to obtain the Java agent name for supported application servers.

Property settings

True or False

Default

Varies by application server, see Notes below.

Example

introscope.agent.agentAutoNamingEnabled=false

Notes

■ Agent autonaming is enabled in the following application servers: WebLogic, WebSphere, and JBoss

■ This property requires the Startup Class to be specified for WebLogic, and requires the Custom Service to be specified for WebSphere.

■ Agent auto naming is disabled in the following application servers: Oracle application server, Interstage, Sun ONE, and Tomcat.


Important! For WebLogic, WebSphere, and JBoss, the property introscope.agent.agentAutoNamingEnabled is set to TRUE by default.

introscope.agent.agentAutoNamingMaximumConnectionDelayInSeconds

Specifies the amount of time in seconds the agent waits for naming information before connecting to the Enterprise Manager.

Default

120

Example

introscope.agent.agentAutoNamingMaximumConnectionDelayInSeconds=120

Notes


Agent naming


introscope.agent.agentAutoRenamingIntervalInMinutes

Specifies the time interval in minutes during which the agent will check to see if it has been renamed.

Default

10

Example

introscope.agent.agentAutoRenamingIntervalInMinutes=10

Notes


introscope.agent.agentName

Uncomment this property to provide a default agent name if other agent naming methods fail.

Property settings

For any installation, if the value of this property is invalid or if this property is deleted from the profile, the agent name will be UnnamedAgent.

Example

#introscope.agent.agentName=AgentName

Notes


■ In the agent profile provided with application server-specific agent installers, the default reflects the application server, for instance WebLogic Agent.

■ In the agent profile provided with the default agent installer, the property value is AgentName, and the line is commented out.

Agent naming



Use this property if you want to specify the agent name using the value of a java system property.

Default

Not specified.

Example


Notes


introscope.agent.disableLogFileAutoNaming

Specifies whether to disable automatic naming of agent log files when using AutoNaming options.

Setting this property to true disables the auto-naming of log files for the agent, AutoProbe and LeakHunter with the agent name or a timestamp.

Property settings

True or False

Default

False

Example

introscope.agent.disableLogFileAutoNaming=false

Notes


■ Log file auto-naming only takes effect when the agent name can be determined using a Java system property or an application server custom service.

Agent naming


introscope.agent.clonedAgent

Enables you to run identical copies of an application on the same machine. Set this property to true if you have identical copies of an application running on the same machine.

Property settings

True or False

Default

False

Example

introscope.agent.clonedAgent=false

Notes


introscope.agent.customProcessName

Specify the process name as it should appear in the Introscope Enterprise Manager and Workstation.

Default

Varies by application server.

Example

introscope.agent.customProcessName=CustomProcessName

Notes


■ In the agent profile provided with application server-specific agent installers, the default reflects the application server, for instance "WebLogic."

■ In the agent profile provided with default agent installer, the property is commented out.

Agent recording (business recording)


introscope.agent.defaultProcessName

If no custom process name is provided and the agent is unable to determine the name of the main application class, this value is used for the process name.

Default

UnknownProcess

Example

introscope.agent.defaultProcessName=UnknownProcess

Notes


introscope.agent.display.hostName.as.fqdn

This property specifies whether the agent name is displayed as a fully qualified domain name (fqdn). To enable the fully-qualified domain name, set this property value to 'true.' By default, the agent displays the host name.

Note: For the Catalyst integration, set this property to 'true.'

Property settings

True or False

Default

False

Example

introscope.agent.display.hostName.as.fqdn=false

Notes


Agent recording (business recording)

You can control how the agent handles business transaction recording.

Note: For more information on agent business recording, see the CA APM Transaction Definition Guide.

Agent thread priority


introscope.agent.bizRecording.enabled

Enables or disables business transaction recording for the agent.

Property settings

True or False

Default

True

Example

introscope.agent.bizRecording.enabled=true

Notes


To further configure business transaction recording for the agent, see the additional properties for the application triage map.

More information:

Application triage map (see page 216) Application triage map business transaction POST parameters (see page 221) Application triage map managed socket configuration (see page 223)

Agent thread priority

The following property controls the priority of agent threads:

■ introscope.agent.thread.all.priority (see page 214)

Agent to Enterprise Manager connection


introscope.agent.thread.all.priority

Controls the priority of agent threads.

Property settings

You can set this from 1 (low) to 10 (high).

Default

Commented out;5.

Example

introscope.agent.thread.all.priority=5

Notes



You can control how the agent connects to the Enterprise Manager.



Default

localhost

Example


Notes





Specifies the port number on the computer that hosts the Enterprise Manager that listens for connections from the agent.

Default

The default port depends on the type of communication channel you want to configure. For direct communication between the agent and Enterprise Manager, the default port is 5001.

Example


Notes


■ If you want to connect to the Enterprise Manager using HTTPS (HTTP over SSL), the default port is 8444. If you want to connect to the Enterprise Manager using SSL, the default port is 5443. However, these default settings are commented out by default.


Specifies the default client socket factory to use for connections from the agent to the Enterprise Manager.

Default

The default socket factory depends on the type of communication channel you want to configure. For direct communication between the agent and Enterprise Manager, the default socket factory is as follows:

com.wily.isengard.postofficehub.link.net.DefaultSocketFactory

Example

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wily.i

sengard.postofficehub.link.net.DefaultSocketFactory

Notes


Application triage map


introscope.agent.enterprisemanager.transport.tcp.local.ipaddress.DEFAULT

Specifies the IP address of the computer running Enterprise Manager that the agent connects to by default.

Default

This property is not defined by default.

Example

introscope.agent.enterprisemanager.transport.tcp.local.ipaddress.DEFAULT=<address

>

Notes


introscope.agent.enterprisemanager.transport.tcp.local.port.DEFAULT

Specifies the local port of the computer running Enterprise Manager that the agent connects to by default.

Default

This property is not defined by default.

Example

introscope.agent.enterprisemanager.transport.tcp.local.port.DEFAULT=CA Portal

Notes



You can configure application triage map data.

Note: For information on how to use the application triage map, see the CA APM Workstation User Guide.



introscope.agent.appmap.enabled

Enables or disables the tracking of monitored code for the application triage map.

Property settings

True or False

Default

True

Example

introscope.agent.appmap.enabled=true

Notes

Enabled by default.

introscope.agent.appmap.metrics.enabled

Enables or disables the tracking of metrics for application triage map nodes.

Property settings

True or False

Default

False

Example

introscope.agent.appmap.metrics.enabled=false

Notes

This property is commented out by default.



introscope.agent.appmap.queue.size

Sets the buffer size for the application triage map.

Property settings

Positive integers.

Default

1000

Example

introscope.agent.appmap.queue.size=1000

Notes

■ The value must be a positive integer.

■ If the value is set to 0, the buffer is unbounded.

■ This property is commented out by default.

introscope.agent.appmap.queue.period

Sets the frequency in milliseconds for sending application triage map data to the Enterprise Manager.

Property settings

Positive integers.

Default

1000

Example

introscope.agent.appmap.queue.period=1000

Notes

■ Must be a positive integer.

■ If the value is set to 0, the default value is used.


Application Triage Map and Catalyst Integration


introscope.agent.appmap.intermediateNodes.enabled

Enables or disables the ability to include intermediate nodes between the application frontend and backend nodes.

Property settings

True or False

Default

False

Example

#introscope.agent.appmap.intermediateNodes.enabled=true

Notes

■ Changes to this property takes effect immediately and do not require the managed application to be restarted.

■ If you set this property to true, you may experience a slowdown of agent performance.



You can configure application triage map data for the Catalyst integration.

Note: For information on how to use the application triage map, see the CA APM Workstation User Guide.

Configure the Ability to Send Information

This property enables or disables the ability to send additional information from the agent for integration with Catalyst.

Follow these steps:

1. Open the default IntroscopeAgent.profile file in a text editor.



Locate the line: introscope.agent.appmap.catalystIntegration.enabled=<false|true>, and set a value as follows:

true

Enables the ability to send additional information from the agent for integration with Catalyst.

false

Disables the configuration.

The following example shows the format:

introscope.agent.appmap.catalystIntegration.enabled=false

Note: This property is commented out by default.


The agent is set up to use the configuration.

Configure a List of Available Networks

The introscope.agent.primary.net.interface.name property specifies the primary network interface name of the host computer used by the agent for the Catalyst integration. You can change the configuration of this property and the change is applied automatically.

Note: When the agent logging level is set to DEBUG, information about network interface names available for configuration appears in the log file. Alternatively, you can use the Network Interface utility to determine the primary network interface name for this property.

Follow these steps:

1. Open the default IntroscopeAgent.profile file in a text editor.

2. Locate the line: introscope.agent.primary.net.interface.name=<false|true>, and specify the name value.

The following example shows the name format:

introscope.agent.primary.net.interface.name=eth4

Note: The default value is undefined. When this property is not set, the agent assigns the first available network interface as the primary interface. You can use the Network Interface utility to determine the name value for this property.

Application triage map business transaction POST parameters


3. (Optional) Allow for multiple network addresses by specifying the subinterface number (starts at 0).

The following example shows the subinterface number format:

introscope.agent.primary.net.interface.name=eth4.1


The profile is set up to use the configuration.

More information:

Using the Network Interface Utility (see page 329)


You can configure your Local Product Shorts to perform more complex monitoring by matching POST parameters.

introscope.agent.bizdef.matchPost

This property determines when POST parameters are matched.

Property settings

The valid settings for this property are never, before, or after.

■ Set the property to never for full agent functionality and better performance. This setting allows your application to identify all business transactions using URLs, cookies or header parameters, but will fail to match any business transactions that are identified solely through POST parameters.

■ Set the property to before to get full agent performance. This setting allows applications to use POST parameters to identify some or all business transactions, but never access the servlet stream directly for HTTP form requests. New applications deployed must also conform to standard API when this property is set to before.

Important! Setting this property to before could have potentially hazardous repercussions for your applications. Review this property setting with a CA Technologies representative before implementation.



■ Set the property to after to safely match business transactions with POST parameters, but with limited agent functionality. When this property is set to after, the agent will not be able to map business transactions that are identified by POST parameters across processes, or produce full sets of metrics for them. This setting also consumes slightly more CPU time compared to the other options, but is considered the safest setting if one needs the POST parameter functionality. It allows applications to use POST parameters to identify some or all business transactions and cannot guarantee that the servlet stream is never accessed directly.

Example

introscope.agent.bizdef.matchPost=after

Notes

■ never - never attempt to match POST parameters. This is the fastest option but may result in inaccurate business transaction component matching.

■ before - matches POST parameters before the servlets execute.

■ after - matches POST parameter patterns after the servlet has executed. Cross process mapping and some metrics will not be available. Default setting for the parameter.

Known limitations

The metrics defined using agent recording are displayed in the application triage map in the Investigator. When configuring agent recording, there are some known limitations when using regular expressions. Most of the limitations have to do with POST parameters.

The known limitations are:

■ Line terminators (.) are not supported for POST parameter values.

■ If POST parameter definitions are dependent on the business transaction definition, only three metrics will be provided for the business transaction component. These metrics are:

■ Average Response Time

■ Responses Per Interval

■ Errors Per Interval

Application triage map managed socket configuration


■ If POST parameter definitions are dependent on the business transaction definition, then the business component name in the transaction trace component will have a generic name and not the specific name of the business service, business transaction, and business transaction component. This also applies to business transaction definitions that are dependent on POST parameter definitions that do not match.

■ Some versions of JBoss and Tomcat might save header keys as lower case values which makes the caseSensitiveName attribute not work properly for HEADER_TYPEs.

Note: For more information on agent recording, see the CA APM Transaction Definition Guide.


The following properties allow you to enable or disable the appearance of socket metrics in the application triage map:

■ introscope.agent.sockets.managed.reportToAppmap (see page 224)

■ introscope.agent.sockets.managed.reportClassAppEdge (see page 224)

■ introscope.agent.sockets.managed.reportMethodAppEdge (see page 225)

■ introscope.agent.sockets.managed.reportClassBTEdge (see page 225)

■ introscope.agent.sockets.managed.reportMethodBTEdge (see page 226)

See the CA APM Workstation User Guide for more information on how to use the application triage map.



introscope.agent.sockets.managed.reportToAppmap

Enables managed sockets to appear in application triage map.

Property settings

True or False

Default

True

Example

introscope.agent.sockets.managed.reportToAppmap=true

Notes


introscope.agent.sockets.managed.reportClassAppEdge

Enables managed sockets to report class level application edges to the application triage map.

Property settings

True or False

Default

False

Example

introscope.agent.sockets.managed.reportClassAppEdge=false

Notes




introscope.agent.sockets.managed.reportMethodAppEdge

Enables managed sockets to report method level application edges to the application triage map.

Property settings

True or False

Default

True

Example

introscope.agent.sockets.managed.reportMethodAppEdge=true

Notes


introscope.agent.sockets.managed.reportClassBTEdge

Enables managed sockets to report class level business transaction edges to the application triage map.

Property settings

True or False

Default

False

Example

introscope.agent.sockets.managed.reportClassBTEdge=false

Notes


AutoProbe


introscope.agent.sockets.managed.reportMethodBTEdge

Enables managed sockets to report method level business transaction edges to the application triage map.

Property settings

True or False

Default

True

Example

introscope.agent.sockets.managed.reportMethodBTEdge=true

Notes


AutoProbe

The following properties configure AutoProbe:

■ introscope.autoprobe.directivesFile (see page 226)

■ introscope.autoprobe.enable (see page 227)

introscope.autoprobe.directivesFile

Specifies ProbeBuilder directives files for AutoProbe.

Default

Varies by installer.

Notes


■ If this property includes one or more directories, and dynamic instrumentation is enabled, the agent will load directives files from the specified directories without an application restart.

AutoProbe


introscope.autoprobe.enable

Enables or disables inserting probes into bytecode automatically.

Property settings

True or False

Default

True

Example

introscope.autoprobe.enable=true

Notes

When this property is set to false, it turns off the automatic insertion of Probes into an application’s bytecode. It does not turn off the agent or agent reporting.

introscope.autoprobe.logfile

Introscope AutoProbe always attempts to log the changes it makes. Set this property to move the location of the log file to something other than the default.

Property settings

Absolute file paths, or non-absolute paths. Non-absolute names are resolved relative to the location of this properties file

Default

../../logs/AutoProbe.log

Example

introscope.autoprobe.logfile=../../logs/AutoProbe.log

Notes


■ To disable AutoProbe logging, modify the value as shown in the following example:

introscope.autoprobe.logfile=logs/AutoProbe.log

Bootstrap Classes Instrumentation Manager


Bootstrap Classes Instrumentation Manager

The following properties configure the Bootstrap Classes Instrumentation Manager:

■ introscope.bootstrapClassesManager.enabled (see page 228)

■ introscope.bootstrapClassesManager.waitAtStartup (see page 228)

The Bootstrap Classes Instrumentation Manager instruments a set of classes after the agent bootstrap, allowing easy implementation of tracers for Java NIO and Secure Sockets Layer (SSL), as well as improving agent performance. You can disable this property by commenting it out in the IntroscopeAgent.profile.

introscope.bootstrapClassesManager.enabled

Enables or disables the bootstrap manager.

Property settings

True or False

Default

True

Example

introscope.bootstrapClassesManager.enabled=true

Notes

■ This property only functions with JVMs running Java 1.5 or higher.

■ If set to false, no system classes will be instrumented.

■ If the property is not set, the default value is false.


introscope.bootstrapClassesManager.waitAtStartup

Sets the time, in seconds, for how long the agent waits after startup to instrument bootstrap classes.

Property settings

Time, in seconds

CA CEM Agent Profile Properties


Default

■ 240 seconds when used with HP-UX, Interstage, WebLogic, or WebSphere Application Server.

■ 5 seconds when used with JBoss, Oracle, Sun, or Tomcat.

Example

introscope.bootstrapClassesManager.waitAtStartup=5

Notes

■ This property only functions with JVMs running Java 1.5 or higher.

■ When this property is active, it can overrule classes that have been designated as skipped. If skipped classes are being instrumented, please contact your CA Technologies representative, or CA Support.


You can configure CA CEM-related IntroscopeAgent.profile properties. The CA Introscope agent profile file is found in the <Agent_Home>\wily directory.

All the CA CEM-related properties are preconfigured to the required options for the integration with CA CEM and CA Introscope to work.

introscope.autoprobe.directivesFile

You need to enable ServletHeaderDecorator / HTTPHeaderDecorator and CEMTracer through the directivesFile property configuration.

The directives file property specifies where to find the directive files (PBD) or directive lists (PBL) for AutoProbe.

AutoProbe uses directives to Introscope-enable your applications, and to determine which metrics the agents report to the Enterprise Manager.

Settings

Depends on the agent application server installed, with the format of <appserver>-full.pbl or <appserver>-typical.pbl.

Default

default-typical.pbl

Example

introscope.autoprobe.directivesFile=weblogic-typical.pbl



Notes

Although you can simply add "ServletHeaderDecorator.pbd" or "httpheaderdecorator.pbd" to the end of this property list, it is better practice to:

1. Locate the PBL file specified in the property (in the example above, weblogic-typical.pbl).

2. Open the PBL file in a text editor.

3. For a Java agent, uncomment to enable the ServletHeaderDecorator.pbd line.

4. For a .NET agent, uncomment to enable the httpheaderdecorator.pbd line.

5. Save your changes to the PBL file.

introscope.agent.remoteagentconfiguration.allowedFiles

Identifies the files that are allowed to be copied remotely from any machine to the agent directory.

The Enterprise Manager uses the file name in this property to identify the valid CA CEM domain configuration file to send to the agents. The domain configuration file contains the CA CEM business service and transactions definitions.

See Notes (below) for CA CEM release-specific information.

Settings

Can be any valid file name.

Default

domainconfig.xml

Example

introscope.agent.remoteagentconfiguration.allowedFiles=domainconfig.xml

Notes

This property can also be used by the Introscope Command-Line Workstation (CLW) Send Config File command.

For more information, see Using the Command-Line Workstation.

This property is valid for CA CEM releases. It is also valid with , CA CEM 4.2 / 4.5 only when you have selected the CEMTracer 4.0 / 4.1 Support option on the Introscope Settings page.



The CEMTracer 4.0 / 4.1 Support option allows you to stagger your agent migration from 4.0 or 4.1 to 4.2 / 4.5 over time; use only if required.

introscope.agent.remoteagentconfiguration.enabled

If this Boolean value is set to true, it allows remote file copies to the agent from another computer.

The Enterprise Manager requires this property to be set to true in order to send the CA CEM domain configuration file to the agents. The domain configuration file contains the CA CEM business service and transactions definitions.


Settings

true or false

Default

■ true for Java Agents

■ false for .NET Agents

Example

introscope.agent.remoteagentconfiguration.enabled=true

Notes

A remote user can also use the Introscope Command-Line Workstation (CLW) Send Config File command to copy the file or files specified in the introscope.agent.remoteagentconfiguration.allowedFiles property into the agent directory.

For more information, see Using the Command-Line Workstation.

This property is valid for CA CEM 4.0 and 4.1 releases. It is also valid with CA CEM 4.2 / 4.5 only when you have selected the CEMTracer 4.0 / 4.1 Support option on the Introscope Settings page.


For non-compatible agents (that is, a .NET agent prior to 7.2.2, an EPA agent, or other non-Java agent), set the introscope.agent.remoteagentconfiguration.enabled property to false.



introscope.agent.decorator.enabled

If this Boolean value is set to true, it configures the agent to add additional performance monitoring information to HTTP response headers. ServletHeaderDecorator / HTTPHeaderDecorator attaches the GUID to each transaction and inserts the GUID into an HTTP header, x-apm-info.

This enables the correlation of transactions between CA CEM and CA Introscope.

Settings

True or false

Default

■ false for Java agents

■ true for .NET agents

Example

introscope.agent.decorator.enabled=false

introscope.agent.decorator.security

Determines the format of decorated HTTP response headers, which are sent to CA CEM.

Settings

■ clear — clear text encoding

■ encrypted — header data is encrypted

Default

clear

Example

introscope.agent.decorator.security=clear

Notes

The default setting of clear is appropriate for initial testing, but might reveal information in the transaction header that you do not want known beyond the firewall. Set the property to encrypted for a more secure production environment.

You must be using JVM 1.4 or greater to set this property to encrypted.



introscope.agent.cemtracer.domainconfigfile

The name of the CA CEM domain configuration file that specifies the CA CEM business service and transaction hierarchy. CEMTracer looks for a file with this name in its installation directory.

Each time a CA CEM administrator clicks Synchronize All Monitors in CA CEM, the domain configuration file gets pushed to the Enterprise Manager, which in turn pushes the file to each connected agent.


Settings

Can be any valid file name.

Default

domainconfig.xml

Example

introscope.agent.cemtracer.domainconfigfile=domainconfig.xml

Notes



■ If an agent is not connected to the Enterprise Manager, the domain configuration file does not get sent.

■ If the agent directory is read-only, the domain configuration file cannot be written.

■ If CEMTracer 4.0 / 4.1 is not enabled on the agent, then no action will be taken on the domain configuration file once it is sent.

introscope.agent.cemtracer.domainconfigfile.reloadfrequencyinminutes

The frequency, in minutes, that the agent reloads the domain configuration file. (The 4.0 / 4.1 agent does not automatically reload the domain configuration file every time the Enterprise Manager sends it. If it has not changed, the agent will not reload it.)




Settings

numeric value

Default

1

Example

introscope.agent.cemtracer.domainconfigfile.reloadfrequencyinminutes=1

Notes


The CEMTracer 4.0 / 4.1 Support option allows you to stagger your agent migration from 4.0 or 4.1 to 4.2 over time; use only if required.

Configure the Session ID Collection

The introscope.agent.transactiontracer.parameter.capture.sessionid property enables or disables the collection of the Session ID in the TransactionTracer Data. By default, this property is enabled and recorded in the TransactionTracer Data. When you disable this property, data is not available for use in filters.

Follow these steps:

1. Open the IntroscopeAgent.profile file in a text editor.

Locate the following lines:

# Uncomment the following property to disable sessionid capture in

TransactionTracer data.

# By default, it is enabled and recorded in the TT Data.

# introscope.agent.transactiontracer.parameter.capture.sessionid=true

2. Follow the instructions to enable or disable the property by commenting or uncommenting the line:

# introscope.agent.transactiontracer.parameter.capture.sessionid=true

3. Save and close the file, and restart the agent.

The agent configuration is set to use the value you specified for collecting the session ID.

ChangeDetector configuration



You can control how the local agent works with ChangeDetector.

Note: For more information about using ChangeDetector, see the CA APM ChangeDetector User Guide.

introscope.changeDetector.enable

Specifies whether ChangeDetector is enabled or disabled. Set the property to true to enable ChangeDetector. It is commented out and set to false by default. If you enable ChangeDetector, you should also set the additional ChangeDetector-related properties.

Property settings

True or False

Default

False

Example

introscope.changeDetector.enable=false

Notes


introscope.changeDetector.agentID

Specifies the text string used by ChangeDetector to identify the local agent. This property is commented out by default. If you enable ChangeDetector, you should uncomment this property and set it to an appropriate value.

Default

The default value is SampleApplicationName.

Example

introscope.changeDetector.agentID=SampleApplicationName



introscope.changeDetector.rootDir

Specifies the root directory for ChangeDetector files. The root directory is the folder where ChangeDetector creates its local cache files.

Property settings

Full path to the root directory for ChangeDetector files as a text string.

Default

The default path is c://sw//AppServer//wily//change_detector.

Example

introscope.changeDetector.rootDir=c://sw//AppServer//wily//change_detector

Notes

Use a backslash to escape the backslash character, as in the example.

introscope.changeDetector.isengardStartupWaitTimeInSec

Specifies the number of seconds to wait after the agent starts before ChangeDetector tries to connect to the Enterprise Manager. This property is commented out by default.

Default

The default is 15 seconds.

Example

introscope.changeDetector.isengardStartupWaitTimeInSec=15

introscope.changeDetector.waitTimeBetweenReconnectInSec

Specifies the number of seconds ChangeDetector waits before retrying a connection to the Enterprise Manager. This property is commented out by default.

Default


Example

introscope.changeDetector.waitTimeBetweenReconnectInSec=10



introscope.changeDetector.profile

Specifies the absolute or relative path to the ChangeDetector datasources configuration file. This property is commented out by default.

Default

The default is ChangeDetector-config.xml.

Example

introscope.changeDetector.profile=CDConfig\\ChangeDetector-config.xml

Notes

Use a backslash to escape the backslash character, as in the example.

introscope.changeDetector.profileDir

Specifies the absolute or relative path to the directory that contains datasource configuration files. If this property is set, all of the datasource configuration files in this directory are used in addition to any file specified by the introscope.changeDetector.profile property. This property is commented out by default.

Default

The default is changeDetector_profiles.

Example

introscope.changeDetector.profileDir=c:\\CDconfig\\changeDetector_profiles

Notes

Use a backslash to escape the backslash character.

Cross-process tracing in WebLogic Server


introscope.changeDetector.compressEntries.enable

Specifies whether to allow compression on the ChangeDetector data buffer. You can set this property to true if you experience memory consumption at start-up to improve performance.

Property settings

True or False

Default

The default value is false if the property is not set in the agent profile or if commented out.

Example

introscope.changeDetector.compressEntries.enable=true

Notes


introscope.changeDetector.compressEntries.batchSize

This property defines the batch size for the compression job, set in introscope.changeDetector.compressEntries.enable above.

Default

1000

Example

introscope.changeDetector.compressEntries.batchSize=1000

Notes


Cross-process tracing in WebLogic Server

The following property configures cross-process tracing in WebLogic Server:

■ introscope.agent.weblogic.crossjvm (see page 239)

Cross-process transaction trace


introscope.agent.weblogic.crossjvm

Property settings

True or False

Default

Commented out; True

Example

introscope.agent.weblogic.crossjvm=true

Cross-process transaction trace

Uncomment the following property to enable automatic collection of downstream traces due to tail filter.

■ introscope.agent.transactiontracer.tailfilterPropagate.enable (see page 239)

Enabling this property and running long periods of Transaction Trace session with tail filters can cause large numbers of unwanted traces to be sent to the Enterprise Manager.

introscope.agent.transactiontracer.tailfilterPropagate.enable

Controls whether the presence of a tail filter triggers automatic collection of traces from downstream agents or not. This property does not affect collection of automatic downstream traces due to passing of head filters.

Property settings

True or False

Default

False; commented out.

Example

introscope.agent.transactiontracer.tailfilterPropagate.enable=false

Notes


Dynamic instrumentation



You can enable classes and methods to be instrumented dynamically without writing custom PBDs, restarting the application server, or restarting the agent.

Note: For more information about using Transaction Tracing and dynamic instrumentation from the Introscope Workstation, see the CA APM Workstation User Guide.

introscope.autoprobe.dynamicinstrument.enabled

Enables dynamic ProbeBuilding for agents that run under JDK 1.5, and use AutoProbe.

Property settings

True or False

Default

False

Example

introscope.autoprobe.dynamicinstrument.enabled=false

Notes

Note: For more information about dynamic ProbeBuilding, see Dynamic ProbeBuilding (see page 73).

autoprobe.dynamicinstrument.pollIntervalMinutes

For agents that run under JDK 1.5 using AutoProbe and dynamic ProbeBuilding, this property determines the frequency with which the agent polls for new and changed PBDs.

Default

1

Example

autoprobe.dynamicinstrument.pollIntervalMinutes=1



introscope.autoprobe.dynamicinstrument.classFileSizeLimitInMegs

Some classloader implementations have been observed to return huge class files.This is to prevent memory errors.

Default

1

Example

introscope.autoprobe.dynamicinstrument.classFileSizeLimitInMegs=1

Notes


introscope.autoprobe.dynamic.limitRedefinedClassesPerBatchTo

Re-defining too many classes at a time might be very CPU intensive. In cases where the changes in PBDs trigger a re-definition of a large number of classes, this batches the process at a comfortable rate.

Default

10

Example

introscope.autoprobe.dynamic.limitRedefinedClassesPerBatchTo=10



introscope.agent.remoteagentdynamicinstrumentation.enabled

Enables or disables remote management of dynamic instrumentation.

Property settings

True or False

Default

True

Example

introscope.agent.remoteagentdynamicinstrumentation.enabled=true

Notes

■ You must restart managed applications before changes to this property take effect.

■ Dynamic instrumentation is a CPU-intensive operation. CA Technologies highly recommends that you use configurations that minimize the classes that are being instrumented.

■ If you enabled dynamic instrumentation for applications using multiple CLRs within the same process, such as in-process side-by-side execution, you must specify the CLR version in the com.wily.introscope.nativeprofiler.monitor.inprocsxs.multiple.clrs property.The valid values are:

– V2 (CLR 2.0)

– V4 (CLR 4.0)

– V2 and V4 (CLR 2.0 and 4.0)

introscope.autoprobe.dynamicinstrument.pollIntervalMinutes

Defines the polling interval in minutes to poll for PBD changes.

Default

1

Example

introscope.autoprobe.dynamicinstrument.pollIntervalMinutes=1

Notes


ErrorDetector


ErrorDetector

You can control how the agent interacts with ErrorDetector.

introscope.agent.errorsnapshots.enable

Enables the agent to capture transaction details about serious errors. Introscope ErrorDetector is installed by default with the agent. This property must be set to true for error snapshots to be available for viewing.

Property settings

True or False

Default

True

Notes

This property is dynamic. You can change the configuration of this property during run time and the change will be picked up automatically.

introscope.agent.errorsnapshots.throttle

Specifies the maximum number of error snapshots that the agent can send in a 15-second period.

Default

10

Example

introscope.agent.errorsnapshots.throttle=10

Notes


Extensions


introscope.agent.errorsnapshots.ignore.<index>

Specifies one or more error message filters. You can specify as many filters as you need using the index identifier appended to the property name (for example, .0, .1, .2 ...). You can use wildcards (*) Error messages matching the criteria you specify are ignored. No error snapshots are generated for errors matching the filters you define and no error events are sent to the Enterprise Manager for them.

Default

Example definitions are provided, and commented out, as shown below.

Example

introscope.agent.errorsnapshots.ignore.0=*com.company.HarmlessException*

introscope.agent.errorsnapshots.ignore.1=*HTTP Error Code: 404*

Notes


Extensions

You can configure the location of agent extensions.

introscope.agent.extensions.directory

Specifies the location of all extensions to be loaded by the agent. You can specify an absolute or relative path to the directory. If you do not specify an absolute path, the value you specify is resolved relative to the location of the IntroscopeAgent.profiles file.

Default

The default location is ext directory in the <Agent_Home>/ext directory.

Example

introscope.agent.extensions.directory=../ext

Notes


GC Monitor


introscope.agent.common.directory

Configures the location of Agent Extension related files.

Default

The default location is common folder in the <Agent_Home>/common directory.

Example

introscope.agent.common.directory=../../common

GC Monitor

The metrics under the GC Monitor node report information on Garbage Collectors and Memory Pools. These metrics help you detect memory-related issues that are adversely affecting performance. You must manually enable the collection of these metrics in the agent profile.

introscope.agent.gcmonitor.enable

Enables or disables the metrics for Garbage Collectors and Memory Pools. This features requires the agent to be version 9.0 or later. It is not supported for older versions of the agents.

Property settings

True or False

Default

The default value is true.

Example

introscope.agent.gcmonitor.enable=true

Notes

This property is dynamic. You can change the configuration of this property during run time and the change is picked up automatically.

You can only report GC Monitor metrics for agents that monitor Sun or IBM JVMs and are Introscope version 9.x and above.

Java NIO


Java NIO

The Java Agent supports the Java New I/O (Java NIO, or NIO) capabilities introduced in Java 1.4. Java NIO is a collection of APIs designed to provide access to the low-level I/O operations of modern operating systems. Java NIO metrics capture information about how instrumented applications use Java NIO.

Note: Introscope Java NIO metrics are only available on Java 1.5 JVMs or higher. Metric collection of Java NIO information is not available for JVM versions prior to Java 1.5.

The Introscope Java Agent collects metrics for NIO channels. For more information, see Channels (see page 247).

You can restrict the generation of certain NIO metrics. For more information, see Restricting Java NIO metrics (see page 248).

Java NIO tracer groups have been enabled by default. You can turn off these tracer groups to further restrict metric generation. For more information on turning on and off Java NIO tracer groups, see Default tracer groups and toggles files (see page 86).

Buffers

Java NIO data transfer is based on buffers. Buffer classes represent memory in a continuous block, together with a small number of data transfer operations. There are buffer classes for all of Java's primitive types, except boolean, which can share memory with byte buffers and allow arbitrary interpretation of the underlying bytes.

Metrics for the following buffer types are collected separately and displayed in the Workstation Investigator:

■ Byte Buffer

■ Int Buffer

■ Double Buffer

■ Long Buffer

■ Float Buffer

■ Short Buffer

These groupings are further divided into different buffer types. Because buffers may overlap, the Total Capacity (bytes) metric may overestimate the amount of memory allocated to NIO buffers.

Java NIO


Direct and non-direct buffers are collected separately because of their differences in memory location and relative costs for creation. Direct buffers are allocated outside the JVM heap used for Java objects, so are not subject to Java garbage collection and relocation. This makes direct buffers optimal for I/O, but they may take more resources to create. Nondirect buffers, by contrast, are allocated as any normal Java object within the JVM heap.

Note: All buffer types may be either direct or non direct, except memory mapped files which are only direct buffers.

Channels

Java NIO channels provide bulk data transfers to and from NIO buffers, as well as external systems. This is a low-level data transfer mechanism that was specifically designed to address performance and scalability issues within standard Java I/O.

Channels provide mechanisms for moving bytes between buffers and external systems. Introscope channel metrics characterize data flow rates through channels. Collected NIO channel metrics correspond to metrics currently created for file and socket I/O using standard Java I/O techniques. Metrics for the following channel types are collected separately and displayed in the Workstation Investigator:

■ Datagram Channels

■ Socket Channels

NIODatagramTracing metrics

Although UDP is a connection-less protocol, the term "connection" is used in the description of NIODatagramTracing to describe how the Java Agent collects datagram metrics. The Java Agent collects metrics separately for each remote endpoint datagrams that are ‘sent to’ or ‘received from’. Connections are classified as client or server depending on the direction of the first datagram observed from each ‘to’ or ‘from’ endpoint.

For example, if the first datagram is ‘from’ the endpoint, the Java Agent classifies all further datagrams to or from that endpoint under NIO|Channels|Datagrams|Server|Port {PORT}, where {PORT} is the local port.

If the first datagram is ‘to’ the endpoint, all further datagrams to or from that endpoint are classified under NIO|Channels|Datagrams|Client|{HOST}|Port {PORT}, where {HOST} and {PORT} are the remote endpoint.

Java NIO


The Java Agent also generates backend metrics for client "connections", except in the case of datagrams read by the 'receive' method. Datagram channels created using DatagramChannel connect method are considered client connections irrespective of direction of first datagram observed.

Note: Datagram channels created using a UDP connection (with a connect method) are considered client connections.

Restricting Java NIO metrics

You can configure properties to control how Java NIO instrumentation works. You can also restrict generation of datagram and socket metrics. These properties only affect the detail metrics generated by NIOSocketTracing and NIODatagramTracing tracer groups.

Note: For more information about tracer groups, see Default tracer groups and toggles files (see page 86).

introscope.agent.nio.datagram.client.hosts

Restricts metric reporting to 'client' UDP "connections" with specified host(s).

Property settings

Comma separated list of hosts.

Default

undefined (no value)

Example

introscope.agent.nio.datagram.client.hosts=hostA,hostB

Notes

■ If the list is left empty, no host restriction will apply.

■ Hosts may be specified by name or textual representation of IP address (in either IPv4 or IPv6 forms).

■ Invalid host names will be reported in the agent log and ignored.

■ This property is dynamic. You can change the configuration of this property during run time and the change will be picked up automatically.

■ Duplicate host names are discarded. In cases where multiple host names map to a single IP, only one of the names is retained. The property will, however, match client connections with any of the set of synonymous names.

Java NIO


introscope.agent.nio.datagram.client.ports

Lists the ports that will report NIO metrics. Only 'client' datagram metrics for specified ports will be generated.

Property settings

Comma separated list of port numbers. Port is the remote port to/from which datagrams are sent/received.

Default


Example

introscope.agent.nio.datagram.client.ports=123,456,789

Notes

■ If the list is empty, no port restriction will apply.

■ Invalid port numbers will be reported in the agent log and ignored.


■ Duplicate ports are discarded. In cases where multiple ports map to a single IP, only one of the ports is retained.

Java NIO


introscope.agent.nio.datagram.server.ports

Lists the ports that will report NIO metrics. Only 'server' datagram metrics for specified ports will be generated.

Property settings

Comma separated list of port numbers. Port is the local port through which datagrams are sent/received.

Default


Example

introscope.agent.nio.datagram.server.ports=123,456,789

Notes

■ If the list is empty, no port restriction will apply.



introscope.agent.nio.socket.client.hosts

Restricts metric reporting to 'client' TCP "connections" with specified host(s).

Property settings

Comma separated list of hosts.

Default


Example

introscope.agent.nio.socket.client.hosts=hostA, hostB

Notes

■ If the list is empty, no host restriction will apply.

■ Hosts may be specified by name or textual representation of IP address (in either IPv4 or IPv6 forms).

■ Invalid host names will be reported in the agent log and ignored.


Java NIO


introscope.agent.nio.socket.client.ports

Lists the ports that will report NIO metrics. Only 'client' socket metrics for specified ports will be generated.

Property settings

Comma separated list of port numbers. Port is the remote port to/from which datagrams are sent/received.

Default


Example

introscope.agent.nio.socket.client.ports=123,456,789

Notes

■ If list is empty, no port restriction will apply.



introscope.agent.nio.socket.server.ports

Lists the ports that will report NIO metrics. Only 'server' socket metrics for specified ports will be generated.

Property settings

Comma separated list of port numbers. Port is the local port through which datagrams are sent/received.

Default


Example

introscope.agent.nio.socket.client.ports=123,456,789

Notes

■ If list is empty, no port restriction will apply.



JMX


Java NIO metrics appear under the NIO node under the agent's top level node in the Workstation Investigator. Additional NIO metrics for any 'client' connections will appear under the Backends node.

Individual NIO metrics may be suppressed by commenting out the TraceOneMethodIfFlagged or TraceOneMethodWithParametersIfFlagged directives for the metric(s) to be suppressed. However, no tracers whose name ends with BackendTracer or MappingTracer should be commented out.

For example, to suppress the Concurrent Readers metric for Datagrams, comment out:

TraceOneMethodWithParametersIfFlagged: NIODatagramTracing read

NIODatagramConcurrentInvocationCounter "Concurrent Readers"

JMX

The following properties configure JMX metrics:

■ introscope.agent.jmx.enable (see page 252)

■ introscope.agent.jmx.ignore.attributes (see page 253)

■ introscope.agent.jmx.name.filter (see page 253)

■ introscope.agent.jmx.name.jsr77.disable (see page 254)

■ introscope.agent.jmx.name.primarykeys (see page 255)

■ introscope.agent.jmx.excludeStringMetrics (see page 256)

introscope.agent.jmx.enable

Enables collection of JMX Metrics.

Property settings

True or False.

Default

Varies by agent version.

Example

introscope.agent.jmx.enable=false

Notes


JMX


introscope.agent.jmx.ignore.attributes

Controls which (if any) JMX MBean attributes are to be ignored.

Property settings

A comma-separated list of keywords.

Default

Commented out; server.

Example

introscope.agent.jmx.ignore.attributes=server

Notes

■ If an MBean attribute name matches one on the list, the attribute will be ignored.

■ Leave the list empty to include all MBean attributes.


introscope.agent.jmx.name.filter

Specifies a comma-separated list of filter strings to determine what JMX data Introscope collects and displays.

Introscope reports JMX-generated metrics that match a filter string. Filter strings can contain the asterisk (*) and question mark (?) wildcard characters:

■ * matches zero or more characters

■ ? matches a single character.

To match a literal * or ?, escape the character with \\.

Examples:

■ ab\\*c matches a metric name that contains ab*c

■ ab*c matches a metric name that contains abc, abxc, abxxc etc.

■ ab?c matches a metric name that contains abxc

■ ab\\?c matches a metric names that contains ab?c

JMX


Default

Commented out.

For WebLogic:

ActiveConnectionsCurrentCount,WaitingForConnectionCurrentCount,PendingRequestCurr

entCount,ExecuteThreadCurrentIdleCount,OpenSessionsCurrentCount,j2eeType

Example

#introscope.agent.jmx.name.filter=ActiveConnectionsCurrentCount,WaitingForConnect

ionCurrentCount,PendingRequestCurrentCount,ExecuteThreadCurrentIdleCount,OpenSess

ionsCurrentCount,j2eeType

Notes

■ Leave empty to include all MBean data available in the system.


introscope.agent.jmx.name.jsr77.disable

This property controls whether or not Introscope collects and reports full JSR77 data, including complex JMX data.

This property is only available for use in the WebLogic and WebSphere IntroscopeAgent.profile files.

Property settings

True or False

Default

True

Notes

■ JSR-77 Management support must be provided by the application server in order for this property to have any effect.


JMX



User-defined order of MBean information, and simplifies name conversion.

Property settings

A comma-separated, ordered list of keys which should uniquely identify a particular MBean.

Default

Commented out in default IntroscopeAgent.profile file.

Example

introscope.agent.jmx.name.primarykeys=J2EEServer

Notes

■ Property settings for WebLogic:

■ Type

■ Name

■ Comment out this property if using WebLogic Server 9.0.

■ Property settings for WebSphere:

■ J2EEServer

■ Application

■ j2eeType

■ JDBCProvider

■ name

■ mbeanIdentifier


LeakHunter


introscope.agent.jmx.excludeStringMetrics

Controls whether or not to include string-valued metrics. To enable string-valued metrics, set this property value to false.

Property settings

True or False

Default

True

Example

introscope.agent.jmx.excludeStringMetrics=true

Notes

■ Excluding string-valued metrics reduces the overall metric count, improving agent and EM performance.


LeakHunter

The following properties configure agent interaction with LeakHunter:

■ introscope.agent.leakhunter.collectAllocationStackTraces (see page 257)

■ introscope.agent.leakhunter.enable (see page 258)

■ introscope.agent.leakhunter.leakSensitivity (see page 258)

■ introscope.agent.leakhunter.logfile.append (see page 259)

■ introscope.agent.leakhunter.logfile.location (see page 259)

■ introscope.agent.leakhunter.timeoutInMinutes (see page 260)

LeakHunter


introscope.agent.leakhunter.collectAllocationStackTraces

Controls whether LeakHunter generates allocation stack traces for potential leaks. Setting this property to true gives you more precise data about the potential leak's allocation, but requires additional memory and CPU overhead. For this reason, the default is false.

Property settings

True or False

Default

False

Example

introscope.agent.leakhunter.collectAllocationStackTraces=

false

Notes

■ Setting this property to true has the potential to create higher system overhead, in CPU usage and memory.


LeakHunter


introscope.agent.leakhunter.enable

Controls whether the LeakHunter feature is enabled. Set the value to true to enable LeakHunter.

Property settings

True or False

Default

False

Example

introscope.agent.leakhunter.enable=false

Notes

■ Turning on this option has the potential to create higher CPU and memory usage. You should only enable this feature when other metrics suggest there is a memory leak.


introscope.agent.leakhunter.leakSensitivity

Controls the sensitivity level of the LeakHunter leak detection algorithm. A higher sensitivity setting will result in more potential leaks reported and a lower sensitivity will result in fewer potential leaks reported.

Property settings

The leak sensitivity range must be positive integer value from 1 (low) to10 (high).

Default

5

Example

introscope.agent.leakhunter.leakSensitivity=5

Notes


LeakHunter


introscope.agent.leakhunter.logfile.append

Specifies whether to replace the log file or add information to an existing log file on application restart.

Property settings

True or False

■ False replaces the log file.

■ True adds information to an existing log file.

Default

False

Example

introscope.agent.leakhunter.logfile.append=false

Notes


introscope.agent.leakhunter.logfile.location

Controls the location for the LeakHunter.log file. You can specify an absolute or relative path to the file name. Relative paths resolve relative to the <Agent_Home> directory. Leave the value blank or commented out if you do not want LeakHunter to record data in a log file.

Default

The default path is ../../logs/LeakHunter.log. This locates the log file in the <Agent_Home>logs directory.

Example

introscope.agent.leakhunter.logfile.location=../../logs/LeakHunter.log

Notes


LeakHunter


introscope.agent.leakhunter.timeoutInMinutes

Controls the length of time (in minutes) that LeakHunter spends looking for new potential leaks. After the time specified, LeakHunter stops looking for new potential leaks. It continues tracking the previously identified potential leaks.

Property settings

Must be a positive integer (no negative numbers).

Default

The default is 120 minutes.

Example

introscope.agent.leakhunter.timeoutInMinutes=120

Notes

■ Set the value to zero if you want LeakHunter to always look for new potential leaks.


introscope.agent.leakhunter.ignore.<number>

Use this to ignore any class matching any supplied patterns. Ten classes have been supplied by default - comment out any you want to use.

Property settings

A comma-separated list of class matching patterns.

Default

None

Logging


Example

introscope.agent.leakhunter.ignore.4=java.util.SubList

introscope.agent.leakhunter.ignore.5=com.sun.faces.context.BaseContextMap$EntrySe

t

introscope.agent.leakhunter.ignore.6=com.sun.faces.context.BaseContextMap$Key

Notes

■ Some collections cannot be used with LeakHunter. In order for a collection to be LeakHunter-safe, it must be safe to call size() at any time, from any thread.


■ You can used the "*" wildcard.

Logging

The following properties configure agent logging options:

■ log4j.logger.IntroscopeAgent (see page 262)

■ log4j.appender.logfile.File (see page 263)

■ log4j.logger.IntroscopeAgent.inheritance (see page 263)

■ log4j.appender.pbdlog.File (see page 264)

■ log4j.appender.pbdlog (see page 264)

■ log4j.appender.pbdlog.layout (see page 265)

■ log4j.appender.pbdlog.layout.ConversionPattern (see page 265)

■ log4j.additivity.IntroscopeAgent.inheritance (see page 266)

Logging


log4j.logger.IntroscopeAgent

This property controls both the logging level and the output location for log information.

Property settings

Level of detail value can be:

■ INFO

■ VERBOSE#com.wily.util.feedback.Log4JSeverityLevel

Destination value can be:

■ console

■ logfile

■ both console and logfile

Default

INFO, console, logfile

Example

log4j.logger.IntroscopeAgent=INFO,console,logfile

Notes


■ To disable agent logging, remove the value as shown in the following example:

log4j.logger.IntroscopeAgent=

Logging


log4j.appender.logfile.File

Specifies the name and location of IntroscopeAgent.log file if logfile is specified in log4j.logger.IntroscopeAgent. The filename is relative to the directory that contains the agent profile.

Default

IntroscopeAgent.log

Example

log4j.appender.logfile.File=../../logs/IntroscopeAgent.log

Notes

System properties (Java command line -D options) are expanded as part of the file name. For example, if Java is started with -Dmy.property=Server1, then log4j.appender.logfile.File=../../logs/Introscope-${my.property}.log is expanded to: log4j.appender.logfile.File=../../logs/Introscope-Server1.log.

log4j.logger.IntroscopeAgent.inheritance

Controls log level and destination for log messages about classes that require instrumentation.

Property settings

To configure logging of classes that have not been instrumented because they extend a supertype or interface, set this property to: INFO, pbdlog

For information about inheritance class logging see Controlling directive logging (see page 79).

Default

None

Example

log4j.logger.IntroscopeAgent.inheritance=INFO,pbdlog

Logging


log4j.appender.pbdlog.File

Identifies a log file for messages about classes that require instrumentation.

Property settings

To configure logging of classes that have not been instrumented because they extend a supertype or interface set to: pbdupdate.log

Default

None

Example

log4j.appender.pbdlog.File=../../pbdupdate.log

log4j.appender.pbdlog

Specifies a package for logging messages about classes that require instrumentation.

Property settings

To configure logging of classes that have not been instrumented because they extend a supertype or interface, set this property to: com.wily.introscope.agent.AutoNamingRollingFileAppender

Default

None

Example

log4j.appender.pbdlog=com.wily.introscope.agent.AutoNamingRollingFileAppender

Logging


log4j.appender.pbdlog.layout

Specifies rules for logging messages about classes that require instrumentation.

Property settings

To configure logging of classes that have not been instrumented because they extend a supertype or interface, set this property to: com.wily.org.apache.log4j.PatternLayout

Default

None

Example

log4j.appender.pbdlog.layout=com.wily.org.apache.log4j.PatternLayout

log4j.appender.pbdlog.layout.ConversionPattern

Specifies rules for logging messages about classes that require instrumentation.

Property settings

To configure logging of classes that have not been instrumented because they extend a supertype or interface, set this property to:

%d{M/dd/yy hh:mm:ss a z} [%-3p] [%c] %m%n

Default

None

Example

log4j.appender.pbdlog.layout.ConversionPattern=%d{M/dd/yy hh:mm:ss a z} [%-3p] [%c]

%m%n

Metric count


log4j.additivity.IntroscopeAgent.inheritance

Causes the directives for multiple level inheritance to be logged only in the pbdupdate.log file.

Property settings

True or False

Default

True

Example

log4j.additivity.IntroscopeAgent.inheritance=true

Notes

To configure the logging of multiple level inheritance directives in the pbdupdate.log only, add this property to the agent profile and set to false.

Metric count

The following property affects where you will see the Metric Count metric in the Investigator:

■ introscope.ext.agent.metric.count (see page 267)

Multiple inheritance


introscope.ext.agent.metric.count

Controls where you will see the Metric Count metric in the Investigator. By default, the Metric Count is displayed as under the Custom Metric Agent node. If you want to see the Metric Count metric under the Agent Stats node, add this property to the IntroscopeAgent.profile.

Property settings

True or False

Default

Not present in the IntroscopeAgent.profile; False

Example

introscope.ext.agent.metric.count=true

Notes

Add this property to the IntroscopeAgent.profile and set it to true to see the ‘Metric Count’ metric under the ‘Agent Stats’ node.


For directives based on interfaces or super classes, the agent is unable to detect multiple inheritance and hence those classes are not instrumented. Enable the following properties to determine those cases after the application server or the agent process starts up. These properties log classes which need to be instrumented but have not been and relies on dynamic instrumentation to affect the changes.

■ introscope.autoprobe.hierarchysupport.enabled (see page 268)

■ introscope.autoprobe.hierarchysupport.runOnceOnly (see page 268)

■ introscope.autoprobe.hierarchysupport.pollIntervalMinutes (see page 269)

■ introscope.autoprobe.hierarchysupport.executionCount (see page 269)

■ introscope.autoprobe.hierarchysupport.disableLogging (see page 270)

■ introscope.autoprobe.hierarchysupport.disableDirectivesChange (see page 270)



introscope.autoprobe.hierarchysupport.enabled

For agents that run under JDK 1.5 using AutoProbe and dynamic instrumentation, you can use this property to enable instrumentation of classes that extend a supertype or interface.

Property settings

True or False

Default

True

Example

introscope.autoprobe.hierarchysupport.enabled=true

Notes


introscope.autoprobe.hierarchysupport.runOnceOnly

If you have enabled instrumentation of classes that extend a supertype or interface, you can use this property to control whether the utility that enables the feature runs only once, or at a specified interval.

Change this property to true only if you need detection on a periodic basis.

Property settings

True or False

Default

False

Example

introscope.autoprobe.hierarchysupport.enabled=false

Notes

■ Logging properties related to dynamic instrumentation are defined in Logging.




introscope.autoprobe.hierarchysupport.pollIntervalMinutes

The polling interval to check for classes which could not be instrumented due to multiple inheritance. In most cases this happens only once; however, a conservative value is recommended to account for application server initialization.

Default

5

Example

introscope.autoprobe.hierarchysupport.pollIntervalMinutes=5

Notes


introscope.autoprobe.hierarchysupport.executionCount

If you need the polling interval to run a finite times instead of running it only once or running it periodically, use this property to specify the exact number of times the polling interval is run. Always use this property to specify the exact number of times it should run.

Using this property overrides the run once only setting.

Property settings

A positive integer.

Default

3

Example

introscope.autoprobe.hierarchysupport.executionCount=3

Notes


Platform monitoring


introscope.autoprobe.hierarchysupport.disableLogging

Uncomment this property if you do not need to log the classes being detected. Only uncomment this property if dynamic instrumentation is enabled.

Property settings

True or False

Default

True

Example

#introscope.autoprobe.hierarchysupport.disableLogging=true

Notes


introscope.autoprobe.hierarchysupport.disableDirectivesChange

Uncomment this property to only log the changes and disable the triggering of dynamic instrumentation.

Property settings

True or False

Default

True

Example

introscope.autoprobe.hierarchysupport.disableDirectivesChange=true

Notes


Platform monitoring

The following property configures platform monitoring metrics:

■ introscope.agent.platform.monitor.system (see page 271)

Remote configuration


introscope.agent.platform.monitor.system

Name of operating system to load a platform monitor for.

Property settings

See Troubleshooting platform monitoring (see page 191) for more information about the options for this property.

Default

Commented out; varies by platform.

Example

introscope.agent.platform.monitor.system=Solaris

Notes


Remote configuration

The following properties allow remote configuration of the Java Agent:

■ introscope.agent.remoteagentconfiguration.enabled (see page 271)

■ introscope.agent.remoteagentconfiguration.allowedFiles (see page 272)

introscope.agent.remoteagentconfiguration.enabled

This property enables or disables remote configuration of the agent.

Property settings

True or False

Default

True

Example

introscope.agent.remoteagentconfiguration.enabled=true

Notes


Security


introscope.agent.remoteagentconfiguration.allowedFiles

This property lists the exact list of files that are allowed to be remotely transferred to this agent.

Property settings

domainconfig.xml

Default

domainconfig.xml

Example

introscope.agent.remoteagentconfiguration.allowedFiles=domainconfig.xml

Notes


Security

The following property configures security of HTTP headers being sent to CA CEM:

■ introscope.agent.decorator.security (see page 272)

introscope.agent.decorator.security

This property determines the format of decorated HTTP response headers, which are sent to CA CEM.

Property settings

Clear: clear text encoding

Encrypted: header data is encrypted

Default

Clear

Example

introscope.agent.decorator.security=clear

Servlet header decorator


Servlet header decorator

The following property enables correlation of transactions between CA CEM and Introscope:

■ introscope.agent.decorator.enabled (see page 273)

introscope.agent.decorator.enabled

If this Boolean value is set to true, it configures the agent to add additional performance monitoring information to HTTP response headers. ServletHeaderDecorator attaches the GUID to each transaction and inserts the GUID into an HTTP header, for example: x-apm-info

Property settings

True or False

Default

False

Example

introscope.agent.decorator.enabled=false

Socket metrics

Generation of I/O Socket metrics may be restricted by the following parameters:

■ introscope.agent.sockets.reportRateMetrics (see page 274)

■ introscope.agent.io.socket.client.hosts (see page 274)

■ introscope.agent.io.socket.client.ports (see page 275)

■ introscope.agent.io.socket.server.ports (see page 275)

Socket metrics


introscope.agent.sockets.reportRateMetrics

Enables reporting of individual socket's input/output (I/O) bandwidth rate metrics.

Property settings

True or False

Default

False

Example

introscope.agent.sockets.reportRateMetrics=false

Notes

■ Only functions if ManagedSocketTracing is enabled and SocketTracing is disabled. See Backwards compatibility (see page 131) for more information.


introscope.agent.io.socket.client.hosts

Restrict socket client connections instrumented to those with specified remote hosts.

Property settings

A comma-separate list of values.

Example

introscope.agent.io.socket.client.hosts=

Notes

■ If any individual value is invalid, it will be ignored.

■ If any parameter is not defined, or after exclusion of any invalid values is an empty list, no restriction will apply to that parameter.


SQL Agent


introscope.agent.io.socket.client.ports

Restrict socket client connections instrumented to those with specified remote ports

Property settings


Example

introscope.agent.io.socket.client.ports=

Notes




introscope.agent.io.socket.server.ports

Restrict socket client connections instrumented to those using specified local ports.

Property settings


Example

introscope.agent.io.socket.server.ports=

Notes




SQL Agent

You can configure aspects of the SQL Agent.

SQL Agent


More information:

introscope.agent.sqlagent.normalizer.extension (see page 276) introscope.agent.sqlagent.normalizer.regex.matchFallThrough (see page 277) introscope.agent.sqlagent.normalizer.regex.keys (see page 278) introscope.agent.sqlagent.normalizer.regex.key1.pattern (see page 278) introscope.agent.sqlagent.normalizer.regex.key1.replaceAll (see page 279) introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat (see page 279) introscope.agent.sqlagent.normalizer.regex.key1.caseSensitive (see page 280) introscope.agent.sqlagent.sql.artonly (see page 280) introscope.agent.sqlagent.sql.rawsql (see page 280) introscope.agent.sqlagent.sql.turnoffmetrics (see page 281) introscope.agent.sqlagent.sql.turnofftrace (see page 281)

introscope.agent.sqlagent.normalizer.extension

Specifies the name of the SQL normalizer extension that will be used to override the preconfigured normalization scheme.

To make custom normalization extension work, the value of its manifest attribute com-wily-Extension-Plugin-{pluginName}-Name should match the value given in this property.

If you specify a comma separated list of names, only the first name will be used. For example:

introscope.agent.sqlagent.normalizer.extension=ext1, ext2

Only ext1 will be used for normalization. Limits how much of a SQL statement appears in the Investigator tree for SQL Agent metrics, in bytes.

Property settings

The name of the SQL normalizer extension that is used to override the preconfigured normalization scheme.

Default

RegexSqlNormalizer

Example


SQL Agent


Notes

If you use the default setting, you also must configure the regular expressions SQL statement normalizer properties:

■ introscope.agent.sqlagent.normalizer.regex.matchFallThrough (see page 277)

■ introscope.agent.sqlagent.normalizer.regex.keys (see page 278)

■ introscope.agent.sqlagent.normalizer.regex.key1.pattern (see page 278)

■ introscope.agent.sqlagent.normalizer.regex.key1.replaceAll (see page 279)

■ introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat (see page 279)

■ introscope.agent.sqlagent.normalizer.regex.key1.caseSensitive (see page 280)


introscope.agent.sqlagent.normalizer.regex.matchFallThrough

Use this property in conjunction with introscope.agent.sqlagent.normalizer.extension (see page 276) to set the regular expressions SQL statement normalizer. When this property is set to true, it will evaluate SQL strings against all regex key groups.

The implementation is chained. For example, if SQL matches multiple key groups, the normalized SQL output from group1 is fed as input to group2, and so on.

If the property is set to false, as soon as a key group matches, the normalized SQL output from that group is returned.

Property settings

True or False

Default

false

Example

introscope.agent.sqlagent.normalizer.regex.matchFallThrough=false

Notes


SQL Agent


introscope.agent.sqlagent.normalizer.regex.keys

Use this property in conjunction with introscope.agent.sqlagent.normalizer.extension (see page 276) to set the regular expressions SQL statement normalizer. This property specifies the regex group keys. They are evaluated in order.

Default

key1

Example

introscope.agent.sqlagent.normalizer.regex.keys=key1

Notes


introscope.agent.sqlagent.normalizer.regex.key1.pattern

Use this property in conjunction with introscope.agent.sqlagent.normalizer.extension (see page 276) to set the regular expressions SQL statement normalizer. This property specifies the regex pattern that will be used to match against the SQL.

Property settings

All valid regex entries allowed by java.util.Regex package can be used here.

Default

.*call(.*\)\.FOO(.*\)

Example

introscope.agent.sqlagent.normalizer.regex.key1.pattern=.*call(.*\)\.FOO(.*\)

Notes


SQL Agent


introscope.agent.sqlagent.normalizer.regex.key1.replaceAll

Use this property in conjunction with introscope.agent.sqlagent.normalizer.extension (see page 276) to set the regular expressions SQL statement normalizer. When this property is set to false, it will replace the first occurrence of the matching pattern in the SQL query with the replacement string. If set to true, it will replace all occurrences of the matching pattern in the SQL query with replacement the string.

Property settings

True or False

Default

false

Example


Notes


introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat

Use this property in conjunction with introscope.agent.sqlagent.normalizer.extension (see page 276) to set the regular expressions SQL statement normalizer. This property specifies the replacement string format.

Property settings

All valid regex entries allowed by the java.util.Regex package and java.util.regex.Matcher class can be used here.

Default

$1

Example

introscope.agent.sqlagent.normalizer.regex.key1.replaceFormat=$1

Notes


SQL Agent


introscope.agent.sqlagent.normalizer.regex.key1.caseSensitive

Use this property in conjunction with introscope.agent.sqlagent.normalizer.extension (see page 276) to set the regular expressions SQL statement normalizer. This property specifies whether the pattern match is sensitive to case.

Property settings

true or false

Default

false

Example


Notes


introscope.agent.sqlagent.sql.artonly

The introscope.agent.sqlagent.sql.artonly property=true configures the agent to create and send only the Average Response Time metric for individual SQL statements under back-ends. When the value for this property is true, performance of the agent for SQL metrics and transaction traces can improve.

Note: Setting introscope.agent.sqlagent.sql.turnoffmetrics (see page 281)=true overrides this property.

This property is turned off by default:

introscope.agent.sqlagent.sql.artonly=false

Changes to this property take effect after you restart the managed application.

introscope.agent.sqlagent.sql.rawsql

The introscope.agent.sqlagent.sql.rawsql property configures the agent to add unnormalized SQL as a parameter for SQL components in Transaction Trace. When the value for this property is true, performance of the agent for SQL metrics and transaction traces can improve.

SQL Agent



introscope.agent.sqlagent.sql.rawsql=false


Important! Enabling this property can result in passwords and sensitive information being presented in Transaction Trace.

introscope.agent.sqlagent.sql.turnoffmetrics

You can turn off individual SQL statement metrics to send fewer metrics from the agent to the Enterprise Manager by using the introscope.agent.sqlagent.sql.turnoffmetrics property. When the value for this property is true, performance of the agent for SQL metrics and transaction traces can improve.


introscope.agent.sqlagent.sql.turnoffmetrics=false


introscope.agent.sqlagent.sql.turnofftrace

The introscope.agent.sqlagent.sql.turnofftrace property controls whether the agent creates transaction trace components and sends them to the Enterprise Manager for individual SQL statements under back-ends. When the value for this property is true, performance of the agent for SQL metrics and transaction traces can improve.


introscope.agent.sqlagent.sql.turnofftrace=false


SSL communication


SSL communication

The agent can connect to the Enterprise Manager over SSL. Use the following properties to configure that communication:

■ introscope.agent.enterprisemanager.transport.tcp.host.DEFAULT (see page 200)

■ introscope.agent.enterprisemanager.transport.tcp.port.DEFAULT

■ introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT

■ introscope.agent.enterprisemanager.transport.tcp.truststore.DEFAULT

■ introscope.agent.enterprisemanager.transport.tcp.trustpassword.DEFAULT

■ introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT

■ introscope.agent.enterprisemanager.transport.tcp.keypassword.DEFAULT

■ introscope.agent.enterprisemanager.transport.tcp.ciphersuites.DEFAULT



Default

localhost

Example


Notes


SSL communication



Specifies the port number on the computer that hosts the Enterprise Manager that listens for connections from the agent. If you are using Secure Socket Layer (SSL) protocol, the default port that listens for connections from the agent is 5443.

Default

5443

Example


Notes



Specifies the client socket factory to use for connections from the agent to the Enterprise Manager when using SSL.

Default

com.wily.isengard.postofficehub.link.net.SSLSocketFactory

Example

ntroscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wily.is

engard.postofficehub.link.net.SSLSocketFactory

Notes


SSL communication


introscope.agent.enterprisemanager.transport.tcp.truststore.DEFAULT

Location of a truststore containing trusted Enterprise Manager certificates. If no truststore is specified, the agent trusts all certificates.

Property settings

Either an absolute path or a path relative to the agent's working directory.

Example

introscope.agent.enterprisemanager.transport.tcp.truststore.DEFAULT=/var/trustedc

erts

Notes

On Windows, backslashes must be escaped. For example: C:\\keystore

introscope.agent.enterprisemanager.transport.tcp.trustpassword.DEFAULT

The password for the truststore.

Example

introscope.agent.enterprisemanager.transport.tcp.trustpassword.DEFAULT=

introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT

Location of a keystore containing the agent's certificate. A keystore is needed if the Enterprise Manager requires client authentication.

Property settings

Either an absolute path or a path relative to the agent's working directory.

Example

introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT=c:\\keystore

Notes

On Windows, backslashes must be escaped. For example: C:\\keystore

Stall metrics


introscope.agent.enterprisemanager.transport.tcp.keypassword.DEFAULT

The password for the keystore.

Example

introscope.agent.enterprisemanager.transport.tcp.keypassword.DEFAULT=MyPassword76

8

introscope.agent.enterprisemanager.transport.tcp.ciphersuites.DEFAULT

Set the enabled cipher suites.

Property settings

A comma-separated list of cipher suites.

Example

introscope.agent.enterprisemanager.transport.tcp.

ciphersuites.DEFAULT=SSL_DH_anon_WITH_RC4_128_MD5

Notes

If not specified, use the default enabled cipher suites.

Stall metrics

The following properties are for stall metrics:

■ introscope.agent.stalls.thresholdseconds (see page 286)

■ introscope.agent.stalls.resolutionseconds (see page 286)

For more information on stall metric properties, see Disabling the capture of stalls as Events (see page 166).

Stall metrics


introscope.agent.stalls.thresholdseconds

Specifies the number of seconds that an executing process can take before it is considered a stalled process. To ensure an accurate Stall Count metric, you must set the stall resolution and stall threshold properties to 15 seconds or more. The stall threshold should not be set to less than 15 seconds to allow time for the Enterprise Manager to complete its harvest cycle. The stall resolution should always be set to a value equal to or greater than the Enterprise Manager harvest duration.

Default


Example

introscope.agent.stalls.thresholdseconds=30

Notes


introscope.agent.stalls.resolutionseconds

Specifies the frequency that the agent checks for stalls. To ensure an accurate Stall Count metric, you must set the stall resolution and stall threshold properties to 15 seconds or more. The stall threshold should not be set to less than 15 seconds to allow time for the Enterprise Manager to complete its harvest cycle. The stall resolution should always be set to a value equal to or greater than the Enterprise Manager harvest duration.

Default

The default is every 10 seconds.

Example

introscope.agent.stalls.resolutionseconds=10

Notes


Thread dumps


Thread dumps

These properties enable and configure agent aspects of CA Introscope thread dump functionality:

■ introscope.agent.threaddump.enable (see page 287)

■ introscope.agent.threaddump.deadlockpoller.enable (see page 288)

■ introscope.agent.threaddump.deadlockpollerinterval (see page 288)

■ introscope.agent.threaddump.MaxStackElements (see page 289)

Note: For more information about configuring thread dumps, see How to enable and configure thread dumps (see page 67).

introscope.agent.threaddump.enable

Enables thread dumps to be collected on an agent JVM and allows users to view the Thread Dumps tab.

Property settings

True or False

Default

True

Example

introscope.agent.threaddump.enable=true

Notes

■ Changes to this property take effect immediately and do not require you to restart the managed application.

■ This property works together with the IntroscopeEnterpriseManager.properties file introscope.enterprisemanager.threaddump.enable property, which enables the Enterprise Manager thread dump functionality if this property is set to true.

Thread dumps


introscope.agent.threaddump.deadlockpoller.enable

Enables the Deadlock Count metric in the metric browser tree to display the current number of deadlocks in the agent JVM.

Property settings

True or False

Default

False

Example

introscope.agent.threaddump.deadlockpoller.enable=true

Notes

■ Changes to this property take effect immediately and do not require you to restart the managed application.

introscope.agent.threaddump.deadlockpollerinterval

Frequency in milliseconds at which CA Introscope polls the agent JVM for deadlocked threads

Property settings

Integer greater than 0

Default

15000 (milliseconds)

Example

introscope.agent.threaddump.deadlockpollerinterval=15000

Notes

■ Restart the managed application so changes to this property can take effect.

Transaction tracing


introscope.agent.threaddump.MaxStackElements

The total number of lines in the thread stack trace determines the size of a CA Introscope thread dump. This property sets the number of lines allowed in the thread stack.

Property settings

Integer greater than 0 and no greater than 25,000

Default

12,000

Example

introscope.agent.threaddump.MaxStackElements=12000

Notes

Restart the managed application so that changes to this property can take effect.

Transaction tracing

The following properties are for transaction tracing and sampling:

■ introscope.agent.transactiontracer.parameter.httprequest.headers (see page 290)

■ introscope.agent.transactiontracer.parameter.httprequest.parameters (see page 290)

■ introscope.agent.transactiontracer.parameter.httpsession.attributes (see page 291)

■ introscope.agent.transactiontracer.userid.key (see page 291)

■ introscope.agent.transactiontracer.userid.method (see page 292)

■ introscope.agent.transactiontrace.componentCountClamp (see page 293)

■ introscope.agent.crossprocess.compression (see page 294)

■ introscope.agent.crossprocess.compression.minlimit (see page 295)

■ introscope.agent.crossprocess.correlationid.maxlimit (see page 296)

■ introscope.agent.transactiontracer.sampling.enabled (see page 296)

■ introscope.agent.transactiontracer.sampling.perinterval.count (see page 297)

■ introscope.agent.transactiontracer.sampling.interval.seconds (see page 297)

■ introscope.agent.transactiontrace.headFilterClamp (see page 298)

For more information, see Configuring Transaction Trace Options (see page 161).

Transaction tracing


introscope.agent.transactiontracer.parameter.httprequest.headers

Specifies (in comma-separated list) HTTP request header data to capture. Use a comma separated list.

Default

Commented out; User-Agent

Example

introscope.agent.transactiontracer.parameter.httprequest.headers=User-Agent

Notes

The IntroscopeAgent.profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the desired header names.

introscope.agent.transactiontracer.parameter.httprequest.parameters

Specifies (in comma-separated list) HTTP request parameter data to capture.

Default

Commented out; generic parameters.

Example

introscope.agent.transactiontracer.parameter.httprequest.parameters=parameter1,pa

rameter2

Notes

The IntroscopeAgent.profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the desired parameter names.

Transaction tracing


introscope.agent.transactiontracer.parameter.httpsession.attributes

Specifies (in comma-separated list) HTTP session attribute data to capture.

Default


Example

introscope.agent.transactiontracer.parameter.httpsession.attributes=attribute1,at

tribute2

Notes

The IntroscopeAgent.profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the desired parameter names.

introscope.agent.transactiontracer.userid.key

User-defined key string.

Default


Example

#introscope.agent.transactiontracer.parameter.httpsession.attributes=attribute1,a

ttribute2

Notes

The IntroscopeAgent.profile contains a commented out statement that sets the value of this property to a null value. The user may optionally uncomment the statement and supply the correct value if, in your environment, user IDs are accessed using HttpServletRequest.getHeader or HttpServletRequest.getValue.

For more information, see introscope.agent.transactiontracer.userid.method (see page 292).

Transaction tracing


introscope.agent.transactiontracer.userid.method

Specifies the method that returns User IDs. The Agent profile includes a commented out property definition for each of the three allowable values.

Uncomment the appropriate statement, based on whether user ID is accessed by getRemoteUser, getHeader, or getValue.

Property settings

Allowable values are:

■ HttpServletRequest.getRemoteUser

■ HttpServletRequest.getHeader

■ HttpServletRequest.getValue

Default

Commented out; see options above.

Example

The IntroscopeAgent.profile includes a commented out property definition for each of the three allowable values. You can uncomment the property you want to use.

introscope.agent.transactiontracer.userid.method=HttpServletRequest.getRemoteUser

#introscope.agent.transactiontracer.userid.method=HttpServletRequest.getHeader

#introscope.agent.transactiontracer.userid.method=HttpSession.getValue

Transaction tracing


introscope.agent.transactiontrace.componentCountClamp

Limits the number of components allowed in a transaction trace.

Default

5000

Warning! If the clamp size is increased, the requirements on memory are higher. In extreme cases, the maximum heap size for the JVM may need to be adjusted or the managed application could run out of memory.

Example

introscope.agent.transactiontrace.componentCountClamp=5000

Notes

■ Any Transaction Trace exceeding the clamp is discarded at the agent and a warning message is logged in the agent log file.


■ When the set limit is reached, warnings appear in the log, and the trace stops.

Transaction tracing


introscope.agent.crossprocess.compression

Use this property to reduce the size of cross process transaction tracing data.

Property settings

lzma, gzip, none

Default

lzma

Example

introscope.agent.crossprocess.compression=lzma

Notes

■ This option will increase agent CPU overhead, but reduce the size of interprocess headers.

■ lzma compression is more efficient than gzip, but may use more CPU.

■ .NET agents do not support the gzip option, so if interoperability is required, do not use gzip.


Transaction tracing


introscope.agent.crossprocess.compression.minlimit

Use this property to set the minimum length of cross process parameter data length for which to apply compression.

Property settings

Can be set from 0 to twice the total maximum limit, set in the introscope.agent.crossprocess.correlationid.maxlimit (see page 296).

If set below the default of 1500, the compression will run more frequently and consume more CPU overhead. The default setting of 1500 usually results in no impact to CPU overhead in normal conditions.

Default

1500

Example

introscope.agent.crossprocess.compression.minlimit=1500

Notes

■ Used with the introscope.agent.crossprocess.compression property above.


Transaction tracing


introscope.agent.crossprocess.correlationid.maxlimit

Maximum size of cross process parameter data allowed.

If the total size of cross process parameter data is more than this limit, even after applying compression, some data will be dropped and some cross process correlation functionality will not work properly.

However, this setting will protect user transactions from failing in network transmission due to too large header size.

Default

4096

Example

introscope.agent.crossprocess.correlationid.maxlimit=4096

Notes

■ Used with the introscope.agent.crossprocess.compression and introscope.agent.crossprocess.compression.minlimit properties above


introscope.agent.transactiontracer.sampling.enabled

Uncomment the following property to disable Transaction Tracer Sampling.

Property settings

True or False

Default

False

Example

introscope.agent.transactiontracer.sampling.enabled=false

Notes


Transaction tracing


introscope.agent.transactiontracer.sampling.perinterval.count

This property is normally configured in the Enterprise Manager. Configuring this property in the agent disables the configuration in the Enterprise Manager. See the CA APM Configuration and Administration Guide for more information.

Default

1

Example

introscope.agent.transactiontracer.sampling.perinterval.count=1

Notes


introscope.agent.transactiontracer.sampling.interval.seconds

This property is normally configured in the Enterprise Manager. Configuring this property in the agent disables the configuration in the Enterprise Manager.

Note: For more information, see the CA APM Configuration and Administration Guide.

Default

120

Example

introscope.agent.transactiontracer.sampling.interval.seconds=120

Notes


Transaction tracing


introscope.agent.transactiontrace.headFilterClamp

Specifies the maximum depth of components allowed in head filtering. Head filtering is the process of examining the start of a transaction for the purpose of potentially collecting the entire transaction. Head filtering checks each component until the first blamed component exits. For transaction with very deep call stacks, this can be a problem if no clamping is applied. The clamp value limits the memory and CPU utilization impact of this behavior by forcing the agent to only look up to a fixed depth.

Default

30

Warning! If the clamp size is increased, the requirement on memory is higher. Garbage collection behavior will be affected, which will have an application-wide performance impact.

Example

introscope.agent.transactiontrace.headFilterClamp=30

Notes


■ Any Transaction Trace whose depth exceeds the clamp will no longer be examined for possible collection unless some other mechanism, such as sampling or user-initiated transaction tracing, is active to select the transaction for collection.

URL grouping


introscope.agent.ttClamp

This property limits the number of transactions that are reported by the agent per reporting cycle.

Property settings

Integers.

Default

50

Example

introscope.agent.ttClamp=50

Notes


■ If the property is not set (left blank), the value defaults to 200.

URL grouping

The following properties are for configuring URL Groups for frontend metrics:

■ introscope.agent.urlgroup.keys (see page 300)

■ introscope.agent.urlgroup.group.default.pathprefix (see page 300)

■ introscope.agent.urlgroup.group.default.format (see page 300)

For more information, see Using URL groups (see page 153).

URL grouping


introscope.agent.urlgroup.keys

Configuration settings for Frontend naming.

Default

Default

Example

introscope.agent.urlgroup.keys=default

Notes

If a URL address belongs to two URL Groups, the order in which you list the keys for the URL Groups in this property is important. The URL Group defined by the narrower pattern should precede the URL Group specified by the broader pattern.

For example, if the URL Group with key alpha contains a single address, and the URL Group with key beta includes all addresses on the network segment that contains the address in the first URL Group, alpha should precede beta in the keys parameter.

introscope.agent.urlgroup.group.default.pathprefix

Configuration settings for frontend naming.

Default

*

Example

introscope.agent.urlgroup.group.default.pathprefix=*

introscope.agent.urlgroup.group.default.format

Configuration settings for Frontend naming.

Default

Default

Example

introscope.agent.urlgroup.group.default.format=default

WebSphere PMI


WebSphere PMI

The following properties configure WebSphere PMI metrics:

■ introscope.agent.pmi.enable (see page 302)

■ introscope.agent.pmi.enable.alarmManager (see page 302)

■ introscope.agent.pmi.enable.bean (see page 303)

■ introscope.agent.pmi.enable.cache (see page 303)

■ introscope.agent.pmi.enable.connectionPool (see page 304)

■ introscope.agent.pmi.enable.hamanager (see page 304)

■ introscope.agent.pmi.enable.j2c (see page 305)

■ introscope.agent.pmi.enable.jvmpi (see page 305)

■ introscope.agent.pmi.enable.jvmRuntime (see page 306)

■ introscope.agent.pmi.enable.objectPool (see page 306)

■ introscope.agent.pmi.enable.orbPerf (see page 307)

■ introscope.agent.pmi.enable.scheduler (see page 307)

■ introscope.agent.pmi.enable.servletSessions (see page 308)

■ introscope.agent.pmi.enable.system (see page 308)

■ introscope.agent.pmi.enable.threadPool (see page 309)

■ introscope.agent.pmi.enable.transaction (see page 309)

■ introscope.agent.pmi.enable.webApp (see page 310)

■ introscope.agent.pmi.enable.webServices (see page 310)

■ introscope.agent.pmi.enable.wlm (see page 311)

■ introscope.agent.pmi.enable.wsgw (see page 311)

■ introscope.agent.pmi.filter.objref (see page 312)

These properties are found in the IntroscopeAgent.websphere.profile file, or the default agent profile for a WebSphere installation.

WebSphere PMI


introscope.agent.pmi.enable

Enables collection of data from WebSphere PMI.

Property settings

True or False

Default

True

Example

introscope.agent.pmi.enable=true

Notes


introscope.agent.pmi.enable.alarmManager

Enables collection of PMI alarm manager data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.alarmManager=false

Notes

■ The alarm manager data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.bean

Enables collection of PMI bean data.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.bean=false

introscope.agent.pmi.enable.cache

Enables collection of PMI cache data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.cache=false

Notes

■ The cache data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.connectionPool

Enables collection of PMI connectionPool data.

Property settings

True or False

Default

True

Example

introscope.agent.pmi.enable.connectionPool=true

introscope.agent.pmi.enable.hamanager

Enables collection of PMI manager data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.hamanager=false

Notes

■ The manager data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.j2c

Enables collection of PMI J2C data when set to true.

Property settings

True or False

Default

True

Example

introscope.agent.pmi.enable.j2c=true

Notes

■ The J2C data category must be turned on in WebSphere to be visible as Introscope data.


introscope.agent.pmi.enable.jvmpi

Enables collection of PMI JVM PI data.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.jvmpi=false

Notes

For data to be provided to this module, JVMPI must be turned on in WebSphere.

WebSphere PMI


introscope.agent.pmi.enable.jvmRuntime

Enables collection of PMI JVM runtime data.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.jvmRuntime=false

Notes

■ For data to be provided to this module, JVMPI must be turned on in WebSphere.


introscope.agent.pmi.enable.objectPool

Enables collection of PMI object pool data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.objectPool=false

Notes

■ The object pool data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.orbPerf

Enables collection of PMI orbPerf data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.orbPerf=false

Notes

■ The orbPerf data category must be turned on in WebSphere to be visible as Introscope data.


introscope.agent.pmi.enable.scheduler

Enables collection of PMI scheduler data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.scheduler=false

Notes

■ The scheduler data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.servletSessions

Enables collection of PMI servletSessions data.

Property settings

True or False

Default

True

Example

introscope.agent.pmi.enable.servletSessions=true

Notes


introscope.agent.pmi.enable.system

Enables collection of PMI system data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.system=false

Notes

■ The system data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.threadPool

Enables collection of PMI thread pool data when set to true.

Property settings

True or False

Default

True

Example

introscope.agent.pmi.enable.threadPool=true

Notes

■ The thread pool data category must be turned on in WebSphere to be visible as Introscope data.


introscope.agent.pmi.enable.transaction

Enables collection of PMI transaction data.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.transaction=false

Notes


WebSphere PMI


introscope.agent.pmi.enable.webApp

Enables collection of PMI webApp data.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.webApp=false

Notes


introscope.agent.pmi.enable.webServices

Enables collection of PMI web services data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.webServices=false

Notes

■ The web services data category must be turned on in WebSphere to be visible as Introscope data.


WebSphere PMI


introscope.agent.pmi.enable.wlm

Enables collection of PMI WLM data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.wlm=false

Notes

■ The WLM data category must be turned on in WebSphere to be visible as Introscope data.


introscope.agent.pmi.enable.wsgw

Enables collection of PMI WSGW data when set to true.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.enable.wsgw=false

Notes

■ The WSGW data category must be turned on in WebSphere to be visible as Introscope data.


WLDF metrics


introscope.agent.pmi.filter.objref

Controls for hard-coded filters.

The objref filter filters out names ending with "@xxxxx" where "xxxxx" is a numeric string.

Property settings

True or False

Default

False

Example

introscope.agent.pmi.filter.objref=false

Notes

You must restart the managed application before changes to this property take effect

WLDF metrics

The following properties configure WLDF metrics:

■ introscope.agent.wldf.enable (see page 313)

WLDF metrics


introscope.agent.wldf.enable

Enables collection of WLDF metrics.

Property settings

True or False

Default

False

Example

introscope.agent.wldf.enable=false

Notes

For WebLogic 9.x and above only.


Appendix B: Alternative Methods for Instrumentation

This section describes alternate methods for instrumenting application when you cannot use JVM AutoProbe. CA Technologies recommends using JVM AutoProbe over the alternatives described in this section in all cases. However, if you cannot use JVM AutoProbe for a specific application server, you can use instructions in this section to instrument your applications.


Deploying the Java Agent on other application servers (see page 315) Configuring Sun ONE 7.0 (see page 316) Configuring Oracle 10g 10.0.3 (see page 317) Configuring WebLogic Server (see page 318) Configuring HTTP servlet tracing (see page 318) Creating an AutoProbe connector file (see page 319) About running ProbeBuilder manually (see page 322) AutoProbe for WebSphere 6.1 and 7.0 for z/OS (see page 322)

Deploying the Java Agent on other application servers

JVM AutoProbe (see Configuring JVM AutoProbe) is the most commonly used method for instrumenting applications. CA Technologies highly recommends using JVM AutoProbe to instrument your applications.

However, you can use Application Server AutoProbe if you are running JVMs 1.4 or lower on the following application servers:

■ Sun ONE 7.0

Application Server AutoProbe is only supported on Sun ONE version 7 application server. See Configuring Sun ONE 7.0 (see page 316).

■ Oracle 10g 10.0.3

Application Server AutoProbe is only supported on Oracle version 10g 10.0.3 application server. See Configuring Oracle 10g 10.0.3 (see page 317).

■ WebSphere or WebLogic

For information about configuring AutoProbe on WebSphere and WebLogic, see Deploying the Java Agent on WebSphere (see page 47) and Deploying the Java Agent on WebLogic (see page 42).

Configuring Sun ONE 7.0


Important! Application Server AutoProbe is not supported on:

■ any JVM 1.5 and above platforms

■ OS/400

Important! Use only one method to instrument your applications. If you have already started using JVM AutoProbe, do not use Application Server AutoProbe.

When starting the application server, avoid using the hyphen (-) character as an identifier for a classname. Introscope does not parse this character, and using it might lead to class loading errors in the agent logs.

Configuring Sun ONE 7.0

The following steps detail how to configure Sun ONE installations to use AutoProbe to instrument applications.

To configure Sun ONE 7.0 to use AutoProbe

Note: The use of "..." in the .xml examples below indicates that there is additional information in the .xml code (not relevant to the example) that is not shown.

1. In order to add Introscope information to startup scripts for Sun ONE 7.0, you must be logged in as Administrator or Root.

2. Open the server.xml file, located at:

<Sun ONE install dir>/domains/domain1/server1/config/

Note: The item separator is a colon (:).

3. Add the full path of wily/Agent.jar to the "server-classpath" property of the java-config element in the server.xml file. For example:

<java-config ... server-classpath="/sw/sun/sunone7/wily/Agent.jar:..." ...>

4. Add the following to the java-config element:

■ Add the bytecode-preprocessors property and set it to the value com.wily.introscope.api.sun.appserver.SunONEAutoProbe.

For example:

<java-config ...

bytecode-preprocessors="com.wily.introscope.api.sun.appserver.SunONEAutoP

robe">

Configuring Oracle 10g 10.0.3


■ Add a jvm-options element to define the location of the agent profile. Define either com.wily.introscope.agentProfile, or com.wily.introscope.agentResource.

The following is an example of com.wily.introscope.agentProfile:

<java-config ...>

...

<jvm-options>-Dcom.wily.introscope.agentProfile=/sw/sun/sunone7/wily/core

/config/IntroscopeAgent.profile </jvm-options>

</java-config>

The following is an example of com.wily.introscope.agentResource:

<java-config ...>

...

<jvm-options>-Dcom.wily.introscope.agentResource=<virtual path

to>/IntroscopeAgent.profile</jvm-options>

</java-config>

■ OPTIONAL: If you configured com.wily.introscope.agentResource, add the resource file to the server classpath.

5. Configure Tracer Groups to collect servlet data. For more information, see Configuring HTTP servlet tracing (see page 318).

Configuring Oracle 10g 10.0.3

The following steps detail how to configure Oracle 10g installations to use AutoProbe to instrument applications.

To configure Oracle 10g 10.0.3 to use AutoProbe

1. Add Agent.jar to the application server classpath.

2. Set the system property oracle.classpreprocessor.classes with the value of com.wily.introscope.api.oracle.OracleAutoProbe.

3. Set the system property oracle.j2ee.class.preprocessing with the value of true.

4. Run this command at the command line:

-Dcom.wily.introscope.probebuilder.oracle.enable=true

Configuring WebLogic Server


5. Restart the Oracle Application Server 10g, using this command:

java

-Doracle.classpreprocessor.classes=com.wily.introscope.api.oracle.OracleAutoP

robe -Doracle.j2ee.class.preprocessing=true

-Dcom.wily.introscope.probebuilder.oracle.enable=true -classpath

oc4j.jar:<path to wily install dir>/wily/Agent.jar

com.evermind.server.OC4JServer -config <path to oracle install

dir>/config/server.xml

Important! Users running Oracle 10g Release 2 using Sun JDK 1.42 must use the ^ (caret) character to escape a forward slash when issuing commands. For example:

-Xbootclasspath^/p:<IntroscopeAgent.jar path>


Configuring WebLogic Server

The following steps detail how to configure WebLogic Server 9.x or 10.x to use AutoProbe to instrument applications.

To configure WebLogic Server 9.x or 10.x to use AutoProbe

1. Edit the classpath in the application startup script (such as startMedRecServer.cmd) to include the wily/Agent.jar file.

2. Set the following property in the application startup script on the Java command line with the -D option to activate Introscope AutoProbe:

-Dweblogic.classloader.preprocessor=

com.wily.introscope.api.weblogic.PreProcessor


Configuring HTTP servlet tracing

Before you use AutoProbe with your application servers to instrument applications, you must configure Tracer Groups in the toggles-full.pbd and toggles-typical.pbd files. This will enable servlet data to be collected.

You will turn one Tracer Group off, and turn another Tracer Group on.

To configure HTTP servlet tracing

1. Navigate to the <your-application-server-home>/wily/core/config/toggles-full.pbd file and open it.

2. Go to the HTTP Servlets Configuration section of the PBD.

Creating an AutoProbe connector file


3. Turn off the HTTPServletTracing Tracer Group by placing a pound sign at the beginning of the line. For example:

#TurnOn: HTTPServletTracing

4. Turn on the HTTPAppServerAutoProbeServletTracing Tracer Group by removing the pound sign from the beginning of the line. For example:

TurnOn: HTTPAppServerAutoProbeServletTracing

5. Repeat steps 2-4 for <your-app-server-home>/wily/core/config/toggles-typical.pbd file.


The deprecated JVM AutoProbe method requires a connector .jar file to operate correctly. Follow this procedure to create the AutoProbe Connector. If your JVM is v1.5, follow the instructions in JVM AutoProbe.

To create an AutoProbe Connector

1. Change the working directory to wily/connectors under the installation directory.

2. Run the Create AutoProbe Connector tool using one of these commands:

■ to specify the JVM using the JVM that is running the tool: java -jar CreateAutoProbeConnector.jar -current

■ to specify the JVM by passing the JVM directory on the command line: java -jar CreateAutoProbeConnector.jar -jvm <directory>

The output is a file with the form: wily/connectors/AutoProbeConnector.jar

3. You may want to rename the created .jar file to be more manageable and universally accepted. For example:

■ wily/connectors/AutoProbeConnector131_02_Sun.jar

OR

■ wily/connectors/AutoProbeConnector130_IBM.jar

Running the AutoProbe Connector for a JVM

After you create the AutoProbe Connector for the Sun or IBM JVM, you run the created file to instrument your applications. The way you run the Connector depends on the application server you use. For more information, see the appropriate section for your application server.

To run the AutoProbe Connector for SAP J2EE 6.20

1. Open the file:

<drive>:\usr\sap\<J2EE_ENGINE_ID>\j2ee\j2ee_<INSTANCE>\cluster\

server\cmdline.properties



2. Append these commands to JavaParameters section:

-Xbootclasspath/p:PathToAutoProbeConnectorJar;PathToAgentJar

-Dcom.wily.introscope.agentProfile=<path-to-IntroscopeAgent.profile>

-Dcom.wily.introscope.agent.agentName=<yourAgentName>

For example:

Xbootclasspath/p:C:/usr/sap/P602/j2ee/j2ee_00/ccms/wily/connectors/AutoProbeC

onnector.jar;C:/usr/sap/P602/j2ee/j2ee_00/ccms/wily/Agent.jar

-Dcom.wily.introscope.agentProfile=C:/usr/sap/P602/j2ee/j2ee_00/ccms/wily/cor

e/config/IntroscopeAgent.profile


To run the AutoProbe Connector for NetWeaver 04/SAP J2EE 6.40

1. Run the SAP J2EE Configtool.

2. Select the server to modify.

3. Add these new java parameters in the Java Parameters field:

-Xbootclasspath/p:PathToAutoProbeConnectorJar;PathToAgentJar

-Dcom.wily.introscope.agentProfile=<path-to-IntroscopeAgent.profile>

For example:

Xbootclasspath/p:D:/usr/sap/ccms/wily/connectors/AutoProbeConnector.jar;D:/us

r/sap/ccms/wily/Agent.jar

-Dcom.wily.introscope.agentProfile=D:/usr/sap/ccms/wily/core/config/Introscop

eAgent.profile

Note: For NetWeaver 6.40 on Windows, the slashes for these java parameters must be forward slashes.

4. Click Disk to save.

5. Repeat steps 2 - 4 for each server.


7. To verify that Configtool changes were made, open the file: <drive>:\usr\sap\ccms\P66\JC00\j2ee\cluster\instance.properties

8. Look for a line beginning with ID<server_id>.JavaParameters, and confirm that it contains the lines you entered.

To run the AutoProbe Connector for Sun ONE

1. Log in as Administrator or Root.

You must be logged in with Administrator or Root permissions to add Introscope information to startup scripts for Sun ONE 7.0.

2. Open the server.xml file, located at: <SunONE install dir>/domains/domain1/server1/config/



3. Add this line to the server.xml file:

<jvm-options>

-Xbootclasspath/p:PathToAutoProbeConnectorJar:PathToAgentJar

</jvm-options>

The item separator is a colon (:). For example:

<jvm-options>

-Xbootclasspath/p:/sw/sun/sunone7/wily/connectors/AutoProbeConnector.jar:/sw/

sun/sunone7/wily/Agent.jar

</jvm-options>

To run the AutoProbe Connector for Oracle 10g

■ To run the AutoProbe Connector, modify the bootstrap classpath:

-Xbootclasspath/p:wily/connectors/AutoProbeConnector.jar:PathToAgentJar

Users running Oracle 10g Release 2 using Sun JDK 1.42 must use the ^ (caret) character to escape a forward slash when issuing commands. For example:

-Xbootclasspath^/p:<IntroscopeAgent.jar path>

Different versions of WebLogic use different versions of Java to run. If you use Java 1.4 or earlier, you will use the following steps to run the AutoProbe connector. If you use Java 1.5 or later, see JVM AutoProbe for more information.

To run the AutoProbe Connector for WebLogic

1. Edit the bootstrap classpath in the application startup script to include the AutoProbeConnector.jar you created (such as startMedRecServer.cmd) using this command:

-Xbootclasspath/p:PathToAutoProbeConnectorJar:PathToAgentJar

add the -X switch to the final start command at the end of the script, after the JAVA_VM and JAVA_OPTIONS. The excerpt below shows the correct place to insert the switch:

"$JAVA_HOME/bin/java" ${JAVA_VM} ${MEM_ARGS} ${JAVA_OPTIONS}

-Xbootclasspath/p:${WL_HOME}/wily/connectors/AutoProbeConnector.jar:${WL_HOME

}/wily/Agent.jar

-Dweblogic.Name=${SERVER_NAME}

-Dweblogic.management.username=${WLS_USER}

-Dweblogic.management.password=${WLS_PW}

-Dweblogic.ProductionModeEnabled=${PRODUCTION_MODE}

-Djava.security.policy="${WL_HOME}/server/lib/weblogic.policy"

weblogic.Server

2. If you are using something other than the default bootstrap classpath, add the Agent.jar and AutoProbeConnector.jar files to the beginning of your customized bootstrap classpath.

About running ProbeBuilder manually


To run the AutoProbe Connector for WebLogic with JRockit JVM

■ Add the following command-line options when starting the JVM:

-Xbootclasspath/a:<PathToAgentJar>

-Xmanagement:class=com.wily.introscope.api.jrockit.AutoProbeLoader

To run the AutoProbe Connector for other application servers

■ To run the AutoProbe Connector, add the Agent.jar and the AutoProbe Connector to the Application Server bootstrap classpath using this command:

-Xbootclasspath/p:wily/connectors/AutoProbeConnector.jar:PathToAgentJar

About running ProbeBuilder manually

Running ProbeBuilder manually is a non-dynamic method of instrumenting applications. When you run ProbeBuilder manually, it instruments classes on disk before the application server is run. You use manual ProbeBuilding when your environment does not support AutoProbe, or you prefer not to use AutoProbe.

Manual ProbeBuilding should not be used with other methods of instrumentation, and should be used as a last resort.

The instructions for manual ProbeBuilding assume you have performed the following installation and configuration tasks:

1. Installed the Java agent. See Installing the Java Agent for more information.

2. Configured Java agent connection properties. See Connecting to the Enterprise Manager (see page 57) for more information.

3. Configured the Java agent name. See Java Agent Naming (see page 117) for more information.

4. Configured options for ProbeBuilder. See AutoProbe and ProbeBuilding Options (see page 71) for more information.

AutoProbe for WebSphere 6.1 and 7.0 for z/OS

This section details how to configure WebSphere on z/OS installations to use AutoProbe to instrument applications. For more information about AutoProbe, see AutoProbe and ProbeBuilding Options (see page 71).

Note: Instrumenting WebSphere 7.0 for z/OS using the following procedure will not result in as detailed metrics as the JVM 1.5 AutoProbe method. For example, the thread metric levels will not be instrumented. For more information on how to instrument WebSphere 7.0 for z/OS using JVM 1.5 AutoProbe, see JVM AutoProbe for WAS 7 on z/OS.



Important! If you are using the Java Agent 9.0 or later to monitor WebSphere 7.0 on z/OS , you may see the application server process restart repeatedly. To avoid this problem, upgrade to WAS 7.0 build level 7.0.0.8 or above.

To configure JVM AutoProbe for WebSphere 6.1, and 7.0 for z/OS

1. In WebSphere, start the Administrator’s Console.

2. Select Application Servers > <your server> > Process Definition.

3. You should see two items, Control and Servant. Click Servant, then JavaVirtualMachine.

4. Set the Generic JVM Argument field to specify the classloader plug-in, and the location of the IntroscopeAgent.profile file. You will set one of the following:

com.wily.introscope.agentProfile

OR

com.wily.introscope.agentResource

The argument will then have the following value (there are several properties set in one argument):

-Dcom.ibm.websphere.classloader.plugin=com.wily.introscope.api

.websphere.WASAutoProbe

-Dcom.wily.introscope.agentProfile=<path to IntroscopeAgent.profile>

OR

-Dcom.ibm.websphere.classloader.plugin=com.wily.introscope.api

.websphere.WASAutoProbe

-Dcom.wily.introscope.agentResource=<path to Resource containing

IntroscopeAgent.profile>

5. Place the Agent.jar file in the <WebSphere Instance dir>/lib/ext directory.

Note: Do not place the Agent.jar file in the WebSphere installation directory.

The following shows examples of the wrong and right directory:

WRONG: /usr/lpp/zWebSphere/V5R0M0/lib/ext

RIGHT: /WebSphere/V5R0M0/AppServer/lib/ext

6. Confirm that all newly created Introscope files and directories within the ./wily directory are read-accessible by the WebSphere process.

7. Confirm that all *.log files (written by the Java Agent and ProbeBuilder) in the ./wily folder have write-access to the WebSphere process. These include:

■ all the Introscope files and directories

■ the Introscope files inside <WAS instance dir>/lib/ext

8. Restart WebSphere application server.



9. When WebSphere says "open for e-business," open the Administrator’s Console. Metrics should start reporting.

10. For AutoProbe to run correctly in WebSphere environments with Java2 Security enabled, it may be necessary to add permissions to your Java2 Security Policy. If Java2 Security is enabled, follow the instructions in Modifying Java2 Security Policy (see page 51).



Appendix C: Using the PBD Generator

You can use the PBD Generator tool to instrument custom Java class files for use by agents.


About the CA PBD Generator (see page 325) Configuring the PBD Generator (see page 326) Using the PBD Generator (see page 326)

About the CA PBD Generator

The PBD Generator utility can create a PBD file from Javadoc tags with which you have annotated your Java code, to facilitate the instrumentation of custom Java class files for use by the Java agent.

The Generator examines a set of Java source files, and instruments the methods in the classes that contain the Javadoc tag @instrument.

Using the PBD Generator tool, you can:

■ automate building of PBD files, to eliminate potential for errors that might be introduced by creating PBD files manually.

■ integrate PBD generation into your build systems to create and update PBD files automatically and incorporate any changes to the Java source.

You configure the PBD Generator by integrating it into an Apache Ant target using the PBDGenerator.jar file, then running it as an Ant Javadoc task.

Configuring the PBD Generator


Configuring the PBD Generator

This tool is intended to be incorporated into Ant-based build systems, as a Javadoc task in an Ant target.

This sample Javadoc task illustrates the use of this tool in Ant:

<javadoc sourcepath="/src/engineering/products/introscope/source"

destdir="/src/engineering/products/introscope/source/generatedpbd"

maxmemory="512m"

packagenames="com.wily.introscope.console.thornhill.ui.util"

verbose="false"

private="true">

<doclet name="com.wily.util.build.javadoc.PBDInstrumentDoclet"

path="/Wily/tools/WilyPBDGenerator.jar">

<param name="-d"

value="/src/engineering/products/introscope/source/generatedpbd"/>

</doclet>

</javadoc>

Required PBD Generator parameters

These key PBD Generator parameters are required:

sourcepath

the root directory of the Java source tree

destdir

the directory path of the PBD file that will be output from the tool

packagenames

a comma-separated list of the Java packages to be examined for instrumentation

doclet path

the path to find the PBD Generator jar file, which contains this tool

param name="-d"

this must contain the same value as destdir

Using the PBD Generator

Before you can use the PBD Generator, you insert special Javadoc tags into the Java source files to be instrumented.

Using the PBD Generator


The syntax for the JavaDoc tag is:

@instrument <valid metric prefix> <optional tracer name>

where:

<valid metric prefix> is any valid Introscope metric prefix—a string without a colon character (:). Pipe characters (|) are acceptable.

<optional tracer name> can be BlamePointTracer, FrontendMarker or BackendMarker. The default is BlamePointTracer if the tracer name is missing.

Appendix D: Using the Network Interface Utility 329

Appendix D: Using the Network Interface Utility

You use the Network Interface utility to determine the network interface name value of the host computer used by the agent for the Catalyst integration.


Determine Network Interface Names (see page 329)

Determine Network Interface Names

The Network Interface utility provides the name and subinterface values for the introscope.agent.primary.net.interface.name property. Run this utility on the same JVM and application server used by the agent.

Follow these steps:

1. From the command line, navigate to the following directory:

<Agent_Home>/wily/tools

2. Run this command to invoke the utility:

java -jar NetInterface.jar

Your browser displays the list of network interface names supported by Java on the Network Interfaces tab.

More information:

Configure a List of Available Networks (see page 220)

Index 331

Index

A

agent architectural overview • 25 depolyment process • 26, 28 directory structure • 38 manual installation • 37 removing from z/OS • 63 silent installation • 35 specifying the path to • 39 starting • 40 uninstalling • 62 upgrading • 61

application management introduction • 26

application server upgrading multiple agents • 61

E

Enterprise Manager architectural overview • 25 configuring the connection • 57 HTTP tunneling • 58 HTTPS connections • 59 proxy server • 59 using SSL • 60

I

installation directory structure • 38 GUI mode • 31 manual • 37 silent • 35

instrumentation full and typical tracing • 72

Introscope architectural overview • 25 discover functionality • 26

IntroscopeAgent.profile deployment process • 28

J

javaagent property • 71 JVM AutoProbe

configuring the agent location • 39

connector file • 319 creating a connector • 319 default metric collection • 71 JRockit • 46

M

manual ProbeBuilding • 71

P

ProbeBuilder Directive (PBD) files introduction • 25

ProbeBuilding customizing • 72 dynamic • 73 methods available • 71 support for older agents • 72

S

superset agent packages • 61

T

thread dump • 67 tracer groups

default configuration options • 72

W

Workstation architectural overview • 25

Date post:	16-Jul-2015
Category:	Technology
Upload:	miriam-soledad-bendezu-perea
View:	514 times
Download:	3 times