+ All Categories
Home > Documents > am41_pdg

am41_pdg

Date post: 03-Apr-2018
Category:
Upload: k4lonk
View: 218 times
Download: 0 times
Share this document with a friend
116
7/28/2019 am41_pdg http://slidepdf.com/reader/full/am41pdg 1/116 IBM Tivoli Access Manager Problem Determination Guide Version 4.1 GC32-1106-01
Transcript
Page 1: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 1/116

IBM Tivoli Access Manager

Problem Determination Guide

Version 4.1

GC32-1106-01

Page 2: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 2/116

Page 3: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 3/116

IBM Tivoli Access Manager

Problem Determination Guide

Version 4.1

GC32-1106-01

Page 4: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 4/116

NoteBefore using this information and the product it supports, read the information in “Notices”, on page 93.

First Edition (August 2003)

This edition applies to version 4.1 of IBM Tivoli Access Manager (product number 5724-C08) and to all subsequentreleases and modifications until otherwise indicated in new editions.

© Copyright International Business Machines Corporation 2002, 2003. All rights reserved.US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contractwith IBM Corp.

Page 5: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 5/116

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiWho should read this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

What this book contains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiPublications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Release information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiBase information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiWebSEAL information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiWeb security information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiDeveloper references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixTechnical supplements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixRelated publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xAccessing publications online . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiOrdering publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiContacting software support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiConventions used in this book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Typeface conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiOperating system differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Chapter 1. Introduction to problem determination. . . . . . . . . . . . . . . . . . 1Problem determination overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Preventing and preparing for problems . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 2. Diagnostic tools and scripts . . . . . . . . . . . . . . . . . . . . . . 3Obtaining version information for installed components (pdversion). . . . . . . . . . . . . . . . . 3Reporting system information (pdinfo) . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Running the pdinfo utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Contents of the tar file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Using the pdinfo.cfg configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Validating and maintaining the policy database (pdacld_dump) . . . . . . . . . . . . . . . . . . 6Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Understanding the database summary report . . . . . . . . . . . . . . . . . . . . . . . 8Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Chapter 3. Using trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Mechanisms for controlling tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Using the trace utility to capture actions . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Basic trace command syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Understanding the trace entry format. . . . . . . . . . . . . . . . . . . . . . . . . . 12Example Base trace output (pd.acl) . . . . . . . . . . . . . . . . . . . . . . . . . . 13Example WebSEAL trace output (pdweb.debug) . . . . . . . . . . . . . . . . . . . . . . 14Location and name of the trace log file . . . . . . . . . . . . . . . . . . . . . . . . . 15

Using routing files to control trace. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Example routing file contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Chapter 4. Using AutoTrace . . . . . . . . . . . . . . . . . . . . . . . . . . . 21AutoTrace overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Building the Tivoli Access Manager product with AutoTrace . . . . . . . . . . . . . . . . . . . 22Description of AutoTrace files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Description of the AutoTrace product file . . . . . . . . . . . . . . . . . . . . . . . . 25Description of the AutoTrace config file . . . . . . . . . . . . . . . . . . . . . . . . . 25Description of the AutoTrace exclude file . . . . . . . . . . . . . . . . . . . . . . . . 27

Verifying that AutoTrace is functioning properly . . . . . . . . . . . . . . . . . . . . . . . 27Initializing atserv.exe (Windows only) . . . . . . . . . . . . . . . . . . . . . . . . . 27

© Copyright IBM Corp. 2002, 2003 iii

Page 6: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 6/116

Initializing atctl.(exe) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Using AutoTrace to collect trace data . . . . . . . . . . . . . . . . . . . . . . . . . . . 29AutoTrace output format and example . . . . . . . . . . . . . . . . . . . . . . . . . . 30Useful AutoTrace commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31How to create a trace ID file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Technique 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Technique 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Known problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Chapter 5. Message logging . . . . . . . . . . . . . . . . . . . . . . . . . . 37Message log file overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Installation log files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Configuration log files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Tivoli Access Manager component log files . . . . . . . . . . . . . . . . . . . . . . . . 38General runtime log files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Logging Base serviceability messages . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Message log entry format and example . . . . . . . . . . . . . . . . . . . . . . . . . 42

Logging WebSEAL serviceability messages . . . . . . . . . . . . . . . . . . . . . . . . . 43Configuring default HTTP logging . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Enabling and disabling HTTP logging . . . . . . . . . . . . . . . . . . . . . . . . . 46Specifying the timestamp type . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Specifying log file rollover thresholds. . . . . . . . . . . . . . . . . . . . . . . . . . 46Specifying the frequency for flushing log file buffers . . . . . . . . . . . . . . . . . . . . 47HTTP common log format (for request.log) . . . . . . . . . . . . . . . . . . . . . . . . 47Displaying the request.log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Displaying the agent.log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Displaying referer.log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Configuring HTTP logging using event logging . . . . . . . . . . . . . . . . . . . . . . . 48

Chapter 6. Tracing and logging for Tivoli Access Manager Java runtime components 51

Chapter 7. Using WebSEAL statistics . . . . . . . . . . . . . . . . . . . . . . 59pdadmin stats command syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Basic pdadmin stats command . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Enable statistics dynamically . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Disable statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Show enabled statistics components . . . . . . . . . . . . . . . . . . . . . . . . . . 61Get current statistic values dynamically . . . . . . . . . . . . . . . . . . . . . . . . . 61Reset statistics values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61List all available statistics components . . . . . . . . . . . . . . . . . . . . . . . . . 62

Statistics components and activity types . . . . . . . . . . . . . . . . . . . . . . . . . . 62pdweb.authn component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62pdweb.authz component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62pdweb.http component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63pdweb.https component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63pdweb.threads component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64pdweb.jmt component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64pdweb.sescache component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64pdweb.doccache component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65pdweb.jct.# component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Enabling statistics statically using event logging . . . . . . . . . . . . . . . . . . . . . . . 67

Chapter 8. Basic troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . 69Verifying software installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Verifying the functionality of individual components . . . . . . . . . . . . . . . . . . . . . 70

IBM Global Security Toolkit (GSKit) . . . . . . . . . . . . . . . . . . . . . . . . . . 71LDAP server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71IBM SecureWay Directory client . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Active Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Domino Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

iv IBM Tivoli Access Manager: Problem Determination Guide

Page 7: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 7/116

Tivoli Access Manager servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Tivoli Access Manager policy server . . . . . . . . . . . . . . . . . . . . . . . . . . 74Tivoli Access Manager runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Tivoli Access Manager WebSEAL . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

 Junctioned third-party Web server . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Tivoli Access Manager Plug-in for Web Servers . . . . . . . . . . . . . . . . . . . . . . 75

Chapter 9. Solutions to common problems . . . . . . . . . . . . . . . . . . . . 77Tivoli Access Manager Base common problems . . . . . . . . . . . . . . . . . . . . . . . 77Unable to configure the policy server . . . . . . . . . . . . . . . . . . . . . . . . . . 77Unable to create new user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78Invalid account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Insufficient disk space causes problems . . . . . . . . . . . . . . . . . . . . . . . . . 79″Not found″ error during user/group create . . . . . . . . . . . . . . . . . . . . . . . 80Unexpected permit/deny access to resources . . . . . . . . . . . . . . . . . . . . . . . 80

LDAP common problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81LDAP does not start after secAuthority=Default suffix is created . . . . . . . . . . . . . . . . 81Insufficient LDAP access privileges to perform operation . . . . . . . . . . . . . . . . . . . 82IBM Directory Server error log warnings . . . . . . . . . . . . . . . . . . . . . . . . 82

Tivoli Access Manager WebSEAL common problems. . . . . . . . . . . . . . . . . . . . . . 83WebSEAL does not respond on ports 80 or 443 . . . . . . . . . . . . . . . . . . . . . . 83

Multiple logins required during e-community . . . . . . . . . . . . . . . . . . . . . . . 83Tivoli Access Manager for WebSphere Application Server common problems . . . . . . . . . . . . . 83WebSphere administrative server does not start . . . . . . . . . . . . . . . . . . . . . . 83WebSphere server does not start after configuration . . . . . . . . . . . . . . . . . . . . . 84WebSphere server does not start after unconfiguration . . . . . . . . . . . . . . . . . . . . 84Migration fails on Windows files with short name . . . . . . . . . . . . . . . . . . . . . 85Client authentication lost due to session expiration . . . . . . . . . . . . . . . . . . . . . 85Unable to access WebSphere Application Server sample applications . . . . . . . . . . . . . . . 85

Tivoli Access Manager for WebLogic Server common problems . . . . . . . . . . . . . . . . . . 86Authorization service won’t start on HP-UX . . . . . . . . . . . . . . . . . . . . . . . 86WebLogic Server throws memory exception . . . . . . . . . . . . . . . . . . . . . . . 86

Migration common problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86Cannot create users and groups after migration . . . . . . . . . . . . . . . . . . . . . . 87

Chapter 10. Contacting customer support . . . . . . . . . . . . . . . . . . . . 89General Data to Collect for Customer Support . . . . . . . . . . . . . . . . . . . . . . . . 89Collecting defect information for pdadmin and C APIs . . . . . . . . . . . . . . . . . . . . . 90

Basic procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Getting the pdmgrd trace file . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Getting the pdacld trace file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Getting the pdadmin and C API trace file . . . . . . . . . . . . . . . . . . . . . . . . 90Getting the message files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Collecting defect information for Java APIs . . . . . . . . . . . . . . . . . . . . . . . . . 91Basic procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Enabling tracing and messaging for Java runtime components . . . . . . . . . . . . . . . . . 91Getting the pdmgrd trace file . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92Getting the pdacld trace file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Appendix. Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Contents v

Page 8: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 8/116

vi IBM Tivoli Access Manager: Problem Determination Guide

Page 9: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 9/116

Preface

Welcome to the IBM ® Tivoli®  Access Manager Problem Determination Guide.

IBM® Tivoli® Access Manager (Tivoli Access Manager) is the base software that isrequired to run applications in the IBM Tivoli Access Manager product suite. Itenables the integration of IBM Tivoli Access Manager applications that provide awide range of authorization and management solutions. Sold as an integratedsolution, these products provide an access control management solution thatcentralizes network and application security policy for e-business applications.

Note: IBM Tivoli Access Manager is the new name of the previously releasedsoftware entitled Tivoli SecureWay® Policy Director. Also, for users familiarwith the Tivoli SecureWay Policy Director software and documentation, themanagement server is now referred to as the policy server.

This problem determination guide provides a comprehensive set of procedures andreference information for troubleshooting Tivoli Access Manager.

Who should read this book 

This guide is for system administrators and field support personnel responsible fortroubleshooting a Tivoli Access Manager environment.

Readers should be familiar with the following:

v PC and UNIX® operating systems

v Database architecture and concepts

v Security management

v

Internet protocols, including HTTP, TCP/IP, File Transfer Protocol (FTP), andTelnet

v Lightweight Directory Access Protocol (LDAP) and directory services

v A supported user registry

v Authentication and authorization

If you are enabling Secure Sockets Layer (SSL) communication, you also should befamiliar with SSL protocol, key exchange (public and private), digital signatures,cryptographic algorithms, and certificate authorities.

What this book contains

This book contains the following sections:

v Chapter 1: Introduction to problem determination

v Chapter 2: Diagnostic tools and scripts

v Chapter 3: Using trace

v Chapter 4: Using AutoTrace

v Chapter 5: Message logging

v Chapter 6: Tracing and logging for Tivoli Access Manager Java Runtimecomponents

v Chapter 7: Using WebSEAL statistics

© Copyright IBM Corp. 2002, 2003 vii

Page 10: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 10/116

v Chapter 8: Basic troubleshooting

v Chapter 9: Solutions to common problems

v Chapter 10: Contacting customer support

Publications

The Tivoli Access Manager library is organized into the following categories:v “Release information”

v “Base information”

v “WebSEAL information”

v “Web security information”

v “Developer references” on page ix

v “Technical supplements” on page ix

Release informationv IBM Tivoli Access Manager Read Me First Card

GI11-4198-00 (am41_readme.pdf)

Provides information for installing and getting started using Tivoli AccessManager.

v IBM Tivoli Access Manager Release NotesSC32-1130-00 (am41_relnotes.pdf)

Provides late-breaking information, such as software limitations, workarounds,and documentation updates.

Base informationv IBM Tivoli Access Manager Base Installation Guide

SC32-1131-01 (am41_install.pdf)

Explains how to install, configure, and upgrade Tivoli Access Manager software,

including the Web Portal Manager interface.v IBM Tivoli Access Manager Base Administrator’s Guide

SC32-1132-01 (am41_admin.pdf)

Describes the concepts and procedures for using Tivoli Access Manager services.Provides instructions for performing tasks from the Web Portal Managerinterface and by using the pdadmin command.

WebSEAL informationv IBM Tivoli Access Manager WebSEAL Installation Guide

SC32-1133-01 (amweb41_install.pdf)

Provides installation, configuration, and removal instructions for the WebSEALserver and the WebSEAL application development kit.

v IBM Tivoli Access Manager WebSEAL Administrator’s GuideSC32-1134-01 (amweb41_admin.pdf)

Provides background material, administrative procedures, and technicalreference information for using WebSEAL to manage the resources of yoursecure Web domain.

Web security informationv IBM Tivoli Access Manager for WebSphere Application Server User’s Guide

SC32-1136-01 (amwas41_user.pdf)

viii IBM Tivoli Access Manager: Problem Determination Guide

Page 11: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 11/116

Provides installation, removal, and administration instructions for Tivoli AccessManager for IBM WebSphere® Application Server.

v IBM Tivoli Access Manager for WebLogic Server User’s GuideSC32-1137-01 (amwls41_user.pdf)

Provides installation, removal, and administration instructions for Tivoli AccessManager for BEA WebLogic Server.

v

IBM Tivoli Access Manager Plug-in for Edge Server User’s GuideSC32-1138-01 (amedge41_user.pdf)

Describes how to install, configure, and administer the plug-in for IBMWebSphere Edge Server application.

v IBM Tivoli Access Manager Plug-in for Web Servers User’s GuideSC32-1139-01 (amws41_user.pdf)

Provides installation instructions, administration procedures, and technicalreference information for securing your Web domain using the plug-in for Webservers.

Developer referencesv IBM Tivoli Access Manager Authorization C API Developer’s Reference

SC32-1140-01 (am41_authC_devref.pdf)

Provides reference material that describes how to use the Tivoli Access Managerauthorization C API and the Access Manager service plug-in interface to addTivoli Access Manager security to applications.

v IBM Tivoli Access Manager Authorization Java Classes Developer’s ReferenceSC32-1141-01 (am41_authJ_devref.pdf)

Provides reference information for using the Java™ language implementation of the authorization API to enable an application to use Tivoli Access Managersecurity.

v IBM Tivoli Access Manager Administration C API Developer’s ReferenceSC32-1142-01 (am41_adminC_devref.pdf)

Provides reference information about using the administration API to enable anapplication to perform Tivoli Access Manager administration tasks. Thisdocument describes the C implementation of the administration API.

v IBM Tivoli Access Manager Administration Java Classes Developer’s ReferenceSC32-1143-01 (am41_adminJ_devref.pdf)

Provides reference information for using the Java language implementation of the administration API to enable an application to perform Tivoli AccessManager administration tasks.

v IBM Tivoli Access Manager WebSEAL Developer’s ReferenceSC32-1135-01 (amweb41_devref.pdf)

Provides administration and programming information for the Cross-domainAuthentication Service (CDAS), the Cross-domain Mapping Framework (CDMF),

and the Password Strength Module.

Technical supplementsv IBM Tivoli Access Manager Command Reference

GC32-1107-01 (am41_cmdref.pdf)

Provides information about the command line utilities and scripts provided withTivoli Access Manager.

v IBM Tivoli Access Manager Error Message ReferenceSC32-1144-01 (am41_error_ref.pdf)

Preface ix

Page 12: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 12/116

Provides explanations and recommended actions for the messages produced byTivoli Access Manager.

v IBM Tivoli Access Manager Problem Determination GuideGC32-1106-01 (am41_pdg.pdf)

Provides problem determination information for Tivoli Access Manager.

v IBM Tivoli Access Manager Performance Tuning Guide

SC32-1145-01 (am41_perftune.pdf)Provides performance tuning information for an environment consisting of TivoliAccess Manager with the IBM Directory server defined as the user registry.

Related publicationsThis section lists publications related to the Tivoli Access Manager library.

The Tivoli Software Library provides a variety of Tivoli publications such as whitepapers, datasheets, demonstrations, redbooks, and announcement letters. The TivoliSoftware Library is available on the Web at:http://www.ibm.com/software/tivoli/library/

The Tivoli Software Glossary includes definitions for many of the technical termsrelated to Tivoli software. The Tivoli Software Glossary is available, in English only,from the Glossary link on the left side of the Tivoli Software Library Web pagehttp://www.ibm.com/software/tivoli/library/

IBM Global Security ToolkitTivoli Access Manager provides data encryption through the use of the IBM GlobalSecurity Toolkit (GSKit). GSKit is included on the IBM Tivoli Access Manager BaseCD for your particular platform.

The GSKit package installs the iKeyman key management utility, gsk5ikm, whichenables you to create key databases, public-private key pairs, and certificaterequests. The following document is available on the Tivoli Information Center

Web site in the same section as the IBM Tivoli Access Manager productdocumentation:

v Secure Sockets Layer Introduction and iKeyman User’s Guide(gskikm5c.pdf)

Provides information for network or system security administrators who plan toenable SSL communication in their Tivoli Access Manager environment.

IBM DB2 Universal DatabaseIBM DB2® Universal Database™ is required when installing IBM Directory Server,z/OS™, and OS/390® LDAP servers. DB2 is provided on the product CDs for thefollowing operating system platforms:

v IBM AIX®

v

Microsoft™

Windows™

v Sun Solaris Operating Environment

DB2 information is available at:

http://www.ibm.com/software/data/db2/

IBM Directory ServerIBM Directory Server, Version 4.1, is included on the IBM Tivoli Access ManagerBase CD for all platforms except Linux for zSeries™. You can obtain the IBMDirectory Server software for Linux for S/390 at:

x IBM Tivoli Access Manager: Problem Determination Guide

Page 13: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 13/116

http://www.ibm.com/software/network/directory/server/download/

If you plan to use IBM Directory Server as your user registry, see the informationprovided at:

http://www.ibm.com/software/network/directory/library/

IBM WebSphere Application ServerIBM WebSphere Application Server, Advanced Single Server Edition 4.0.3, isincluded on the Web Portal Manager CDs and installed with the Web PortalManager interface. For information about IBM WebSphere Application Server, see:

http://www.ibm.com/software/webservers/appserv/infocenter.html

IBM Tivoli Access Manager for Business IntegrationIBM Tivoli Access Manager for Business Integration, available as a separatelyorderable product, provides a security solution for IBM MQSeries®, Version 5.2,and IBM WebSphere® MQ for Version 5.3 messages. IBM Tivoli Access Manager forBusiness Integration allows WebSphere MQSeries applications to send data withprivacy and integrity by using keys associated with sending and receiving

applications. Like WebSEAL and IBM Tivoli Access Manager for OperatingSystems, IBM Tivoli Access Manager for Business Integration, is one of theresource managers that use the authorization services of IBM Tivoli AccessManager for e-business.

The following documents associated with IBM Tivoli Access Manager for BusinessIntegration Version 4.1 are available on the Tivoli Information Center Web site:

v IBM Tivoli Access Manager for Business Integration Administrator’s Guide(SC23-4831-00)

v IBM Tivoli Access Manager for Business Integration Release Notes (GI11-0957-00)

v IBM Tivoli Access Manager for Business Integration Read Me First (GI11-0958-00)

IBM Tivoli Access Manager for Operating SystemsIBM Tivoli Access Manager for Operating Systems, available as a separatelyorderable product, provides a layer of authorization policy enforcement on UNIXsystems in addition to that provided by the native operating system. IBM TivoliAccess Manager for Operating Systems, like WebSEAL and IBM Tivoli AccessManager for Business Integration, is one of the resource managers that use theauthorization services of IBM Tivoli Access Manager for e-business.

The following documents associated with IBM Tivoli Access Manager forOperating Systems Version 4.1 are available on the Tivoli Information Center Website:

v IBM Tivoli Access Manager for Operating Systems Installation Guide (SC23-4829-00)

v IBM Tivoli Access Manager for Operating Systems Administration Guide

(SC23-4827-00)

v IBM Tivoli Access Manager for Operating Systems Problem Determination Guide(SC23-4828-00)

v IBM Tivoli Access Manager for Operating Systems Release Notes (GI11-0951-00)

v IBM Tivoli Access Manager for Operating Systems Read Me First (GI11-0949-00)

Preface xi

Page 14: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 14/116

Accessing publications onlineThe publications for this product are available online in Portable Document Format(PDF) or Hypertext Markup Language (HTML) format, or both in the TivoliSoftware Library: http://www.ibm.com/software/tivoli/library

To locate product publications in the library, click the Product manuals link on the

left side of the Library page. Then, locate and click the name of the product on theTivoli Software Information Center page.

Product publications include release notes, installation guides, user’s guides,administrator’s guides, and developer’s references.

Note: To ensure proper printing of PDF publications, select the Fit to page check box in the Adobe Acrobat Print window (which is available when you clickFile →Print).

Ordering publicationsYou can order many IBM Tivoli publications online at:http://www.elink.ibmlink.ibm.com/public/applications/

publications/cgibin/pbi.cgi

You can also order by telephone:

v In the United States: 800-879-2755

v In Canada: 800-426-4968

v In other countries, for a list of telephone numbers, seehttp://www.ibm.com/software/tivoli/order-lit/

Accessibility

Accessibility features help a user who has a physical disability, such as restrictedmobility or limited vision, to use software products successfully. With this product,you can use assistive technologies to hear and navigate the interface. You also canuse the keyboard instead of the mouse to operate all features of the graphical userinterface.

Contacting software support

Before contacting IBM Tivoli Software support with a problem, refer to the IBMTivoli Software support Web site at:http://www.ibm.com/software/sysmgmt/products/support/

If you need additional help, contact software support by using the methodsdescribed in the IBM Software Support Guide at the following Web site:

http://techsupport.services.ibm.com/guides/handbook.html

The guide provides the following information:

v Registration and eligibility requirements for receiving support

v Telephone numbers and e-mail addresses, depending on the country in whichyou are located

v A list of information you should gather before contacting customer support

xii IBM Tivoli Access Manager: Problem Determination Guide

Page 15: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 15/116

Conventions used in this book 

This reference uses several conventions for special terms and actions and foroperating system-dependent commands and paths.

Typeface conventions

The following typeface conventions are used in this reference:Bold Lowercase commands or mixed case commands that are difficult to

distinguish from surrounding text, keywords, parameters, options, namesof Java classes, and objects are in bold.

Italic Variables, titles of publications, and special words or phrases that areemphasized are in italic.

MonospaceCode examples, command lines, screen output, file and directory namesthat are difficult to distinguish from surrounding text, system messages,text that the user must type, and values for arguments or commandoptions are in monospace.

Operating system differencesThis book uses the UNIX convention for specifying environment variables and fordirectory notation. When using the Windows command line, replace $variable with%variable% for environment variables and replace each forward slash (/) with a

 backslash (\) in directory paths. If you are using the bash shell on a Windowssystem, you can use the UNIX conventions.

Preface xiii

Page 16: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 16/116

xiv IBM Tivoli Access Manager: Problem Determination Guide

Page 17: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 17/116

Chapter 1. Introduction to problem determination

Problem determination (or troubleshooting) is a process of determining why acertain product is not functioning in the expected manner. This guide provides

information about resources and techniques to aid in the identification andresolution of problems related to IBM Tivoli Access Manager for e-business.

Problem determination overview

Tivoli Access Manager provides an authentication and authorization framework forpermitting or restricting access to system resources located in a secure domain.This guide provides information about Tivoli Access Manager tools, resources, andtechniques that can aid in the identification and resolution of problems.

Subjects discussed include:

v Diagnostic and data collection tools

v Message logs

v Tracing tools and logs

v Product statistics

v Basic troubleshooting information

v Solutions to common Tivoli Access Manager problems

Preventing and preparing for problems

Problems can often be avoided with planning and preparation before deployingthe Tivoli Access Manager software. Before installing Tivoli Access Manager,review the latest Release Notes and the IBM Tivoli Access Manager installation

guides. These documents contain the following important information:v Supported operating system levels

v Prerequisite software requirements

v Required software patches

v Minimum and recommended memory requirements

v Disk space requirements

v Upgrade considerations

v Known problems, limitations, and recovery procedures

v Customer support contact information

If a problem does occur, it should be identified and resolved in a timely manner.

The following are suggestions that may help with problem resolution:

v Maintain updated information on the topology and configuration of your TivoliAccess Manager environment.

v Maintain updated information that describes the key system resources beingmanaged by Tivoli Access Manager and the security policies being applied tothem by Tivoli Access Manager

v Visit the IBM Customer Support Web site regularly to check for new fixpacksand other useful information; install new fixpacks as they become available

v Attempt to resolve the problem using the resources within this guide inconjunction with the IBM Tivoli Access Manager Error Message Reference

© Copyright IBM Corp. 2002, 2003 1

Page 18: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 18/116

v If it becomes necessary to contact IBM for assistance, gather relevant informationahead of time, as suggested within this guide

2 IBM Tivoli Access Manager: Problem Determination Guide

Page 19: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 19/116

Chapter 2. Diagnostic tools and scripts

Section topics:

v “Obtaining version information for installed components (pdversion)” on page 3

v “Reporting system information (pdinfo)” on page 3

v “Validating and maintaining the policy database (pdacld_dump)” on page 6

Obtaining version information for installed components (pdversion)

The pdversion command displays a list of IBM Tivoli Access Manager fore-business components and indicates the version number for any componentsinstalled on the current system. If you require assistance from Customer Support totroubleshoot a problem with your Tivoli Access Manager installation, you willneed to provide the information obtained from the pdversion command.

The pdversion executable is located in the following directory of the Tivoli Access

Manager Base installation:

UNIX:

/opt/PolicyDirector/bin/

Windows:

C:\Program Files\Tivoli\Policy Director\bin\

For example (UNIX):

# pdversionIBM Tivoli Access Manager Runtime 4.1.0.0IBM Tivoli Access Manager Policy Server 4.1.0.0IBM Tivoli Access Manager Web Portal Manager Not InstalledIBM Tivoli Access Manager Application Developer Kit 4.1.0.0IBM Tivoli Access Manager Authorization Server 4.1.0.0IBM Tivoli Access Manager Java Runtime Environment Not Installed

Reporting system information (pdinfo)

The purpose of the pdinfo utility is to gather and store current information aboutyour Tivoli Access Manager computer system. You can send the resulting .tar fileto technical support personnel to help them assist you in a problem-solving ortroubleshooting situation. The pdinfo utility operates in a Tivoli Access Managerenvironment on the following platforms: HP-UX, Solaris, AIX, Linux, andWindows NT/2000.

Note: You should not run the pdinfo utility in a production environment. Bydefault, the pdinfo utility kills all Tivoli Access Manager servers in order torecord coredump files. Refer to the discussion of the $EXECUTE_BLADESparameter later in this section for further information.

The pdinfo utility is part of the Tivoli Access Manager Base (PDRTE) installation.Other Tivoli Access Manager components, such as WebSEAL, contain embeddedPerl scripts that are executed automatically when you run the pdinfo utility, andprovide the appropriate system information for that component.

© Copyright IBM Corp. 2002, 2003 3

Page 20: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 20/116

The pdinfo.cfg configuration file allows you to customize the operation of thepdinfo utility.

The pdinfo utility does not automatically capture any trace log files which mayhave been created by the user. Any trace log files which have been generated needto be independently gathered and sent to Customer Support, in addition to the.tar file generated by the pdinfo utility.

Running the pdinfo utilityThe pdinfo utility is located in the following directory of the Tivoli AccessManager Base installation:

UNIX:

/opt/PolicyDirector/sbin/

Windows:

C:\Program Files\Tivoli\Policy Director\sbin\

You must be located in this directory to run the pdinfo utility or, alternatively, use

a full path name. When you run the pdinfo utility, you are prompted to providethe full path name for the system information output file.

For example (UNIX):

# cd /opt/PolicyDirector/sbin# ./pdinfoPlease enter the name (full path) of the desired output fileeg. /tmp/output.tar.gz:

Technical notes:

v You should not run the pdinfo utility in a production environment. By default,the pdinfo utility kills all Tivoli Access Manager servers in order to recordcoredump files. Refer to the discussion of the $EXECUTE_BLADES parameter

later in this section for further information.

v In a UNIX environment, you must be logged in as root (or an account with rootauthority) to successfully run the pdinfo utility.

v The directory containing the output file must have write access.

v Ensure there is adequate hard disk space available.

Contents of the tar fileThe .tar file resulting from the pdinfo utility contains the following files:

v System information output file

This output file name is specified in the pdinfo.cfg configuration file.

v Error log file

This file logs errors produced when running the pdinfo utility to. The pdinfoutility invokes other system commands to gather information. This file collectsany errors resulting from these system commands.

v Tivoli Access Manager configuration, database, and log files

All standard Tivoli Access Manager configuration, database, and log files arecollected in the .tar file.

v Tivoli Access Manager server core dumps

4 IBM Tivoli Access Manager: Problem Determination Guide

Page 21: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 21/116

The final task of the pdinfo utility is to kill the Tivoli Access Manager servers(except PDRTE) and log the resulting coredump files. Refer to the discussion of $EXECUTE_BLADES below for further information.

Using the pdinfo.cfg configuration fileYou can customize the information gathered by the pdinfo utility using thepdinfo.cfg

configuration file.

The pdinfo.cfg configuration file is located in the following directory:

UNIX:

/opt/PolicyDirector/etc/pdinfo/

Windows:

C:\Program Files\Tivoli\Policy Director\etc\pdinfo\

The configuration file contains the following information:

Output and error file names

You can specify the default names of the system information output file and thepdinfo-specific error file.

The default file names include:

$OUTPUT_FILE = "systeminfo.txt";$ERROR_FILE = "error.log";

Tivoli Access Manager component reporting scriptsThis section of the configuration file specifies the location of reporting scripts forother Tivoli Access Manager components:

$PDMGR_BLADE = "etc/pdinfo_pdmgr_blade.pl";$PDACLD_BLADE = "etc/pdinfo_pdacld_blade.pl";$PDWEB_BLADE = "etc/pdinfo_pdweb_blade.pl";

This information is out-of-date. The current version of the pdinfo tool no longeruses this configuration information. Instead, the pdinfo tool executes the set of reporting scripts which reside within the /etc/pdinfo directory of each TivoliAccess Manager product.

Information typesYou can specify the type of information gathered by the pdinfo utility. To enablespecific information gathering, set the appropriate parameter equal to “1”. Todisable specific information gathering, set the appropriate parameter equal to “0”or, alternatively, comment out the parameter line.

For example:

$GATHER_MACHINE_INFO = 1;$GATHER_NETWORK_INFO = 1;$GATHER_PROCESS_INFO = 1;$GATHER_INSTALLED_INFO = 1;$GATHER_PD_INFO = 1;$EXECUTE_BLADES = 1;

Parameter descriptions:

v Machine information ($GATHER_MACHINE_INFO) gathers the machine name,operating system version, system architecture, processors, RAM, virtual memorystatistics, and disk space.

Chapter 2. Diagnostic tools and scripts 5

Page 22: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 22/116

v Network information ($GATHER_NETWORK_INFO) gathers information aboutinterfaces, ARP cache, routing, and open sockets.

v Current processes information ($GATHER_PROCESS_INFO) gathers informationabout current processes running on the system.

v Installed software information ($GATHER_INSTALLED_INFO) gathersinformation about software currently installed on the system.

v

Tivoli Access Manager information ($GATHER_PD_INFO) gathers informationabout the current state of Tivoli Access Manager Base.

v Tivoli Access Manager component information ($EXECUTE_BLADES) runs thereporting scripts for other configured Tivoli Access Manager components.

This is the section of the pdinfo utility where the Tivoli Access Manager serversare killed in order to record the resulting coredump files. The kill operationoccurs within the Perl script for each server or ”blade”. You can therefore controlthe kill operation on a per-server basis by modifying the appropriate script forthat server.

Alternatively, you can disable the running of all Perl scripts by setting$EXECUTE_BLADES=0.

Example configuration file#### Constant definitions$OUTPUT_FILE = "systeminfo.txt";$ERROR_FILE = "error.log";

$PDMGR_BLADE = "etc/pdinfo_pdmgr_blade.pl";$PDACLD_BLADE = "etc/pdinfo_pdacld_blade.pl";$PDWEB_BLADE = "etc/pdinfo_pdweb_blade.pl";

### CHANGE VARIABLES HERE TO DETERMINE WHAT INFORMATION IS GATHERED### BY DEFAULT ALL INFORMATION IS GATHERED - TO PREVENT INFORMATION BEINGGATHERED, COMMENT OUT THE APPROPRIATE VARIABLE, OR SET IT TO 0

### MACHINE INFORMATION: Machine Name, O/S Version, System Architecture,Processors, RAM, VM Stats, Disk Space

$GATHER_MACHINE_INFO = 1;### NETWORK INFORMATION: interfaces, arp cache, routing, open sockets$GATHER_NETWORK_INFO = 1;

### CURRENT PROCESSES: Current processes running on the system$GATHER_PROCESS_INFO = 1;

### INSTALLED SOFTWARE: Current installed software on the system$GATHER_INSTALLED_INFO = 1;

### PD INFORMATION: information regarding the current state of PD$GATHER_PD_INFO = 1;

### EXECUTE PDWEB, ACLD, MGR, AND OTHER BLADES$EXECUTE_BLADES = 1;

Validating and maintaining the policy database (pdacld_dump)

The pdacld_dump executable is a serviceability utility used to validate andmaintain Tivoli Access Manager policy (authorization) databases. The utility isinstalled with the PDMgr package and is currently not translated.

6 IBM Tivoli Access Manager: Problem Determination Guide

Page 23: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 23/116

Some options to the pdacld_dump utility require an understanding of theauthorization database components and structure. It is recommended that youalways work with the advice and assistance of Tivoli Customer Support whenusing this utility.

The utility’s name, ″pdacld_dump,″ does not accurately describe its function. Theutility is not directly connected with the Tivoli Access Manager authorization

server (pdacld).

The pdacld_dump utility examines the content and structure of a Tivoli AccessManager authorization database. Tivoli Access Manager authorization databasesinclude the master database (master_authzn.db – controlled by the policy server,pdmgrd) and replica databases. Replica databases are used by all instances of theTivoli Access Manager authorization server (pdacld) and any C applicationsrunning in local mode.

Some of the functions provided by the utility include:

v Transform the binary content of the specified database file into readable text(directed to standard output or redirected to a file)

v

Produce a summary statement describing the condition of the specified databasev Examine the specified database for any corrupted content, defragment the

database structure, and produce a valid updated version of the database

v Provide two levels of validation checking

By default, the utility resides in the sbin directory.

UNIX:

/opt/PolicyDirector/sbin/

Windows:

C:\Program Files\Tivoli\Policy Director\sbin\

You must be located in the sbin directory to run the pdinfo utility or, alternatively,use a full path name.

Syntaxpdacld_dump -v

pdacld_dump -f <db-filename> [-s] [-r <new-db-file>] [-l <level>] [-t <obj-type>]

Table 1. pdacld_dump syntax 

Options andArguments

Description

–v This option returns the version of the utility.

–f <db-filename> This required option specifies the file name of the authorizationdatabase to be examined. A readable display of the databasecontents (″dump″) is sent, by default, to standard output.

Alternatively, you can redirect this output to a file.

A database analysis summary appears at the end of this output.

Windows: You must enclose the full path name for the databasefile name argument within double-quote marks.

Chapter 2. Diagnostic tools and scripts 7

Page 24: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 24/116

Table 1. pdacld_dump syntax (continued) 

Options andArguments

Description

–s This option prevents the full database content output (″dump″)and causes the utility to display only the summary of thedatabase analysis.

If you do not use this option, the summary appears at the end of the full content output.

–r <new-db-file> This option specifies the file name of a new database containingonly valid entries from the source database (specified by the –f option).

Additionally, the fresh structure of this new database filerepresents a defragmented version of the source database.

–l <level> This option specifies the level of validation checking. Valid choicesare ″1″ or ″2″. Level 2 is the default and highest level of validation.

v Level 1 performs checks to ensure that the ″Object Name:″ and″Object Type:″ fields match.

v Level 2 checks that ACLs have an appropriate map object andthat all cross-reference checking is still valid.

–t <obj-type> When used with the –s option, this option lists only databaseobjects of the specified object type.

The summary is also displayed.

Understanding the database summary reportThe pdacld_dump summary report reveals certain important information about thecondition of the authorization database:

v The DB Sequence Number changes each time the database is updated. If you

compare this number with the master authorization database sequence number,you can determine if the two databases are synchronized or not.

v The ″Dumped″ line reveals whether the file contains the complete database. It ispossible for a database to be truncated, and therefore missing data.

v The ″invalid objects were encountered″ line indicates the validity of the databasecontents.

_____________________________________________________________________Summary for /var/PolicyDirector/db/master_authzn.dbDumped 48 of 48 objects.

DB Sequence number :16821DB SSL Sequence number :1001

FrequencyCount vs ObjectType vs BasePrefix summary7:1281:/auth/pobject-map7:1282:/auth/acl-map1:1283:/auth/dbinfo0:1284:/auth/extern-auth0:1285:/auth/actions0:1287:/auth/pop0:1288:/auth/pobject-popmap0:1289:/auth/pop-map1:1290:/auth/pobjectspace21:1291:/auth/pobject0:1292:/auth/extattr7:1293:/auth/extended-acl

8 IBM Tivoli Access Manager: Problem Determination Guide

Page 25: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 25/116

1:1294:/auth/action-groups1:2566:/servers/azn/0:1286: NULL objects0:255: RESERVED-typed objects0:: v37 objects0:: unknown objects

0 invalid objects were encountered.

Database passes specified validation level check.

ExamplesShow version:

Entering the following command (UNIX):

# pdacld_dump -v

returns:

Policy Director ACL Database Viewer v4.1.0 (Build 020311)

Copyright (C) IBM Corporation 1994-2002. All Rights Reserved.

Display full database contents plus summary:

Entering the following command (UNIX):

# pdacld_dump -f /var/PolicyDirector/db/master_authzn.db

examines the master_authzn.db file and displays the contents of the database tostandard output. Note the ″Summary″ section near the end of this output. You caneliminate the database content dump and display only the summary report byadding the –s option to the example command line above.

The following abbreviated example output has been edited to show only the initialoutput entries and then the concluding summary section:

START OBJECTObject type: 1291Object size: 38Object name: /auth/pobject/Management/POPObject seqnum: 0IVPobj:

Description: POP Management root.Type: 16Is Policy Attachable: 1

END OBJECT

START OBJECT

Object type: 1291Object size: 18Object name: /auth/pobject/Management/Groups/ivmgrd-serversObject seqnum: 0IVPobj:

Description:Type: 0Is Policy Attachable: 1

END OBJECT

START OBJECTObject type: 1281Object size: 26

Chapter 2. Diagnostic tools and scripts 9

Page 26: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 26/116

Object name: /auth/pobject-mapObject seqnum: 0Protected object to ACL map:

Attached ACLs:default-root

END OBJECT

START OBJECT

Object type: 1291Object size: 41Object name: /auth/pobject/Management/PolicyObject seqnum: 0IVPobj:

Description: Policy Management root.Type: 16Is Policy Attachable: 1

END OBJECT

.

.

.

._____________________________________________________________________Summary for /var/PolicyDirector/db/master_authzn.dbDumped 48 of 48 objects.

DB Sequence number :16821DB SSL Sequence number :1001

FrequencyCount vs ObjectType vs BasePrefix summary7:1281:/auth/pobject-map7:1282:/auth/acl-map1:1283:/auth/dbinfo0:1284:/auth/extern-auth0:1285:/auth/actions0:1287:/auth/pop0:1288:/auth/pobject-popmap0:1289:/auth/pop-map1:1290:/auth/pobjectspace

21:1291:/auth/pobject0:1292:/auth/extattr7:1293:/auth/extended-acl1:1294:/auth/action-groups1:2566:/servers/azn/0:1286: NULL objects0:255: RESERVED-typed objects0:: v37 objects0:: unknown objects

0 invalid objects were encountered.

Database passes specified validation level check.

10 IBM Tivoli Access Manager: Problem Determination Guide

Page 27: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 27/116

Chapter 3. Using trace

IBM Tivoli Access Manager for e-business provides configurable tracing capabilitiesthat can aid in problem determination. Tracing can be activated at startup using

the routing files or can be activated dynamically at runtime using the pdadmintrace set command (see special conditions in the section below). Trace data isintended primarily for use by Customer Support personnel and may be requestedas part of diagnosing a reported problem.

However, it is possible experienced product administrators to use trace data todiagnose and correct problems in the environment. Tracing is best suited tosituations where a problem is easily reproduced, is short-lived in duration, and can

 be produced without significant trace generation from other system activity.

Section topics:

v “Mechanisms for controlling tracing” on page 11

v “Using the trace utility to capture actions” on page 11v “Using routing files to control trace” on page 16

Mechanisms for controlling tracing

v The pdadmin server task trace command can be used to dynamically controltrace operations for the Tivoli Access Manager authorization server, WebSEAL,and the Tivoli Access Manager Plug-in for Web Servers, as well as for customauthorization C API applications.

v A routing file can also be used to control tracing. Tivoli Access Manager Baseincludes its own routing file, as does WebSEAL. In order for modifications to aproduct’s routing file to take effect, the product must be stopped and restarted.

For example, tracing for the Tivoli Access Manager policy server cannot becontrolled dynamically with the pdadmin server task trace command. You mustuse the routing file to enable tracing for the policy server. The policy servermust be restarted for any routing file modifications to take effect.

Using the trace utility to capture actions

The trace utility allows you to capture information about error conditions andprogram control flow in Tivoli Access Manager Base, Tivoli Access ManagerWebSEAL, and Tivoli Access Manager Plug-in for Web Servers. This information isstored in a file and used for debugging purposes. The trace utility is providedprimarily to assist support personnel in diagnosing problems occurring with thefunctioning of the Tivoli Access Manager software.

As a user, you may find some of the Base, WebSEAL, and Web Server Plug-intracing components useful. However, the majority are of little benefit unless youare diagnosing complex problems with the assistance of technical supportpersonnel.

Note: Use trace with caution. It is intended as a tool to use under the direction of technical support personnel. Messages from trace are sometimes cryptic, arenot translated, and can severely degrade system performance.

© Copyright IBM Corp. 2002, 2003 11

Page 28: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 28/116

Basic trace command syntaxpdadmin> server task <server-name> trace <command>

The set of configured servers can be learned by issuing the following command:

pdadmin> server list

You can perform the following tasks with the pdadmin trace command:Table 2. trace commands 

trace list List all available trace components

trace set Enable the trace level and trace message destination for a component andits subordinates

trace show Show the name and level for all enabled trace components or for thespecified component

List all available trace componentsList the specified component or all components available to gather and report traceinformation.

trace list [<component>]

Enable traceUse the pdadmin trace set command to enable the gathering of trace informationfor the specified component and level.

trace set <component> <level> [file path=< file> | <log-agent>]

Table 3. trace arguments 

Argument Description

component The trace component name. Required. WebSEAL-specific componentsare prefixed with “pdweb”.

level Reporting level. Required. The level argument specifies the amount of 

detail gathered by the trace utility. The range is 1 to 9. Level 1 specifiesthe most detailed output and level 9 specifies the least detailed output.

 file The fully qualified name of the file to which trace data will be written.

logagent Optionally specifies a destination for the trace information gathered forthe specified component. Refer to the “Using event logging” chapter of the IBM Tivoli Access Manager Base Administrator’s Guide for completeconfiguration details.

Show enabled trace componentsList all enabled trace components or a specific enabled component. If a specifiedcomponent is not enabled, no output is displayed.

trace show [<component>]

Example:

pdadmin> server task webseald-<instance> trace set pdweb.debug 2pdadmin> server task webseald-<instance> trace showpdweb.debug 2

Understanding the trace entry formatThe following output is an example of a pdadmin trace entry as it would appearwithin a Tivoli Access Manager trace log file.

12 IBM Tivoli Access Manager: Problem Determination Guide

Page 29: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 29/116

2002-08-07-16:02:27.717+00:00I----- thread(2) trace.pd.acl.adminsvc:8e:/am410/src/libivacl/azn_admin_svc.cpp:921: CII EXIT:AznAdministrationSvc::azn_admin_perform_task()

This trace entry consists of the following fields:

Table 4. trace entry field names 

Field Name Description

time The time that the trace log entry was created.(″2002-08-07-16:02:27.717+00:00I″ in the example above)This time is in the following format:CCYY-MM-DD-hh:mm:ss.fff[+|-]II:iiwhere the digit groupings represent:CCYY - century and yearMM - monthDD - dayhh - hourmm - minutesss - secondsfff - fractions of a secondII:ii - a time inaccuracy factor

thread_id An identifier for the thread which generated the entry.(″thread(2)″ in the example above.)

subcomponent The subcomponent of the program from which the entrywas generated.(″pd.acl.adminsvc″ in the example above)

debug_level An indicator of the debug level of the trace entry.(″8″ in the example above.)Nine debug message levels can be enabled (1 through 9),Setting debug tracing at a particular level means thatall levels up to and including the specified level are enabled.For example, if the user sets the debug level at 4, then levels1, 2, and 3 are enabled as well.Setting the debug level to 0 disables the tracing.

src_file The name of the product source file which generatedthe entry.(″e:/am410/src/libivacl/azn_admin_svc.cpp″ in the example above)

src_line The line number within the product source file where theentry was generated.(″921″ in the example above)

text The text of the trace log entry.(″CII EXIT: AznAdministrationSvc::azn_admin_perform_task()″ in theexample above)

Example Base trace output (pd.acl)Below is a trace example which was generated by the following command pair(each entered as one line):

pdadmin> server task webseald-<instance> trace set pd.acl 9 \file path=c:\dirname\filenamepdadmin> acl create testacl

2002-08-07-16:02:27.717+00:00I----- thread(2) trace.pd.acl.adminsvc:8e:/am410/src/libivacl/azn_admin_svc.cpp:921: CII EXIT:AznAdministrationSvc::azn_admin_perform_task()

2002-08-07-16:02:27.732+00:00I----- thread(2) trace.pd.acl.adminsvc:1

Chapter 3. Using trace 13

Page 30: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 30/116

e:/am410/src/libivacl/azn_admin_svc_mtshandlers.cpp:439: AZN Status -- major =:0x00000000; minor =: 0x00000000

2002-08-07-16:02:27.748+00:00I----- thread(2) trace.pd.acl.aznapi:6e:/am410/src/libivacl/azn_error.cpp:579: CEI ENTRY: azn_error_get_message_id()

2002-08-07-16:02:27.748+00:00I----- thread(2) trace.pd.acl.aznapi:6e:/am410/src/libivacl/azn_error.cpp:610: CEI EXIT: azn_error_get_message_id()

2002-08-07-16:02:27.764+00:00I----- thread(2) trace.pd.acl.aznapi:4e:/am410/src/libivacl/azn_attrlist.cpp:790: API ENTRY: azn_attrlist_add_ulong()

2002-08-07-16:02:27.764+00:00I----- thread(2) trace.pd.acl.aznapi:4e:/am410/src/libivacl/azn_attrlist.cpp:833: API EXIT azn_attrlist_add_ulong()

2002-08-07-16:02:27.779+00:00I----- thread(2) trace.pd.acl.aznapi:6e:/am410/src/libivacl/azn_error.cpp:579: CEI ENTRY: azn_error_get_message_id()

2002-08-07-16:02:27.795+00:00I----- thread(2) trace.pd.acl.aznapi:6e:/am410/src/libivacl/azn_error.cpp:610: CEI EXIT: azn_error_get_message_id()

2002-08-07-16:02:27.795+00:00I----- thread(2) trace.pd.acl.adminsvc:1e:/am410/src/libivacl/azn_admin_svc_mtshandlers.cpp:441: status: 0x00000000

2002-08-07-16:02:27.810+00:00I----- thread(2) trace.pd.acl.aznapi:9e:/am410/src/libivacl/azn_attrlist.cpp:510:Attribute list - handle: 0x013f6f70, actual: 0x015f7888[[ list: 0x015ad238 to 0x015ad244 (0 attr_t structs) ]]

2002-08-07-16:02:27.826+00:00I----- thread(2) trace.pd.acl.aznapi:9e:/am410/src/libivacl/azn_attrlist.cpp:522:Entry 0: name: azn_admin_svc_pluginstatus, valcount: 1, list: 0x015b4358

2002-08-07-16:02:27.826+00:00I----- thread(2) trace.pd.acl.aznapi:9e:/am410/src/libivacl/azn_attrlist.cpp:534: val(0): size: 4

2002-08-07-16:02:27.842+00:00I----- thread(2) trace.pd.acl.aznapi:9e:/am410/src/libivacl/azn_attrlist.cpp:566: ulong: 0x0

2002-08-07-16:02:27.857+00:00I----- thread(2) trace.pd.acl.adminsvc:6e:/am410/src/libivacl/azn_admin_svc_mtshandlers.cpp:489: CEI EXIT:azn_admin_svc_serverperformtask_MTShandler_t::handleCommand()

Example WebSEAL trace output (pdweb.debug)Note: The pdweb.debug component only operates at level 2.

The following command (entered as one line) invokes the trace utility for thepdweb.debug component at level 2, and directs the output to a file, using theevent logging mechanism to specify a file log agent.

pdadmin> server task webseald-<instance> trace set pdweb.debug 2 \file path=/opt/pdweb/log/debug.log

Sample output of this command as it appears in the debug.log file (note theextensive text entries that follow the src_line value):

2002-09-11-23:42:19.725+00:00I----- thread(7) trace.pdweb.debug:2/amweb410/src/wand/wand/log.c:278: ------------- Browser ===> PD -------------Thread_ID:27GET /junction/footer.gif HTTP/1.1Accept: */*Referer: https://bevan/junction/Accept-Language: en-usAccept-Encoding: gzip, deflateIf-Modified-Since: Wed, 11 Jul 2001 21:11:14 GMTIf-None-Match: "abe09-3c8-3b4cc0f2"

14 IBM Tivoli Access Manager: Problem Determination Guide

Page 31: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 31/116

User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)Host: bevanConnection: Keep-Alive---------------------------------------------------

2002-09-11-23:42:19.736+00:00I----- thread(7) trace.pdweb.debug:2/amweb410/src/wand/wand/log.c:278: ------------- PD ===> BackEnd -------------Thread_ID:27

GET /footer.gif HTTP/1.1via: HTTP/1.1 bevan:443host: blade.cruz.ibm.com:444user-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)accept: */*accept-language: en-usaccept-encoding: gzip, deflateif-none-match: "abe09-3c8-3b4cc0f2"referer: https://bevan/blade/if-modified-since: Wed, 11 Jul 2001 21:11:14 GMTconnection: close

---------------------------------------------------

2002-09-11-23:42:19.739+00:00I----- thread(7) trace.pdweb.debug:2/amweb410/src/wand/wand/log.c:278: ------------- PD <=== BackEnd -------------Thread_ID:27HTTP/1.1 304 Not Modifieddate: Wed, 11 Sep 2002 23:34:17 GMTetag: "abe09-3c8-3b4cc0f2"server: IBM_HTTP_SERVER/1.3.19.1 Apache/1.3.20 (Unix)connection: close---------------------------------------------------

2002-09-11-23:42:19.740+00:00I----- thread(7) trace.pdweb.debug:2/amweb410/src/wand/wand/log.c:278: ------------- Browser <=== PD ------------Thread_ID:27HTTP/1.1 304 Not Modifieddate: Wed, 11 Sep 2002 23:34:17 GMTetag: "abe09-3c8-3b4cc0f2"

server: IBM_HTTP_SERVER/1.3.19.1 Apache/1.3.20 (Unix)

---------------------------------------------------

Location and name of the trace log fileWhere any given trace log file is located and what it is named depends uponwhich Tivoli Access Manager component is being traced. For the Tivoli AccessManager authorization server, the trace log file can be explicitly specified by theuser in the following command:

pdadmin> server task <server-name> trace set <component> <level> [file path=< file>]

where server-name is the name of the authorization server displayed by thepdadmin server list command, and file is the fully qualified trace file name.

Alternatively, if the Tivoli Access Manager Base routing file (discussed elsewherewithin this document) is being used to enable and disable tracing, then the locationand name of this trace log file can be defined within the routing file.

For WebSEAL, the trace log file can be explicitly specified by the user in thefollowing command:

pdadmin> server task <server-name> trace set <component> <level> [file path=< file>]

Chapter 3. Using trace 15

Page 32: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 32/116

where server-name is the name of the WebSEAL server displayed by the pdadminserver list command, and file is the fully qualified trace file name.

Alternatively, if the WebSEAL routing file (discussed elsewhere within thisdocument) is being used to enable and disable tracing, then the location and nameof this trace log file can be defined within the routing file.

For the Tivoli Access Manager Plug-in for Web Servers component, message logentries and trace log entries are always written, by default, to the same set of files.These include:

Windows:

(log for the authorization server)

<websvrplugin-install-dir>\log\msg__pdwebpi.log

(log for the IIS plug-in)

<websvrplugin-install-dir>\log\msg__pdwebpi-iis.log

UNIX:

(log for the authorization server)

/var/pdwebpi/log/msg__pdwebpi.log

(log for the watchdog server)

/var/pdwebpi/log/msg__pdwebpimgr.log

For the Tivoli Access Manager Plug-in for Edge Server component, message logentries and trace log entries are written to the same file. For this Tivoli AccessManager component, tracing is effectively enabled at all times. Tracing, therefore, isnot controlled via the pdadmin command. This log file is:

Windows:<edgesvrplugin-install-dir>\cp\logs\event.date

UNIX:

/opt/ibm/edge/cp/server_root/logs/event.date

Using routing files to control trace

An alternative method for enabling and disabling trace is by manually editing arouting file. The routing file is a file that can be used to define the name, location,and logging behavior of certain message log and trace log files, as well as messagelog files. More discussion of how to use the routing file to control message logging

is discussed in the″

Message logging″

chapter. The Tivoli Access Manager Base andWebSEAL components each have their own routing (or routing.template) filesdefined within their respective etc directories. If the file is namedrouting.template, it must first be copied to a file with the name routing.

The Tivoli Access Manager Plug-in for Web Servers component hardcodes theinformation that is typically contained within a routing file, therefore, it has norouting file of its own. However, it is possible to define an environment variablenamed PD_SVC_ROUTING_FILE that specifies the fully qualified name of arouting file whose values override the hardcoded defaults used by the Web

16 IBM Tivoli Access Manager: Problem Determination Guide

Page 33: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 33/116

Plug-in component. Keep in mind that the contents of this routing file can alsooverride the contents of the routing files for Tivoli Access Manager Base andWebSEAL, so use it with care.

The contents of a routing file are fairly self-descriptive. When using a routing fileto affect trace logging or message logging, you must stop and restart Tivoli AccessManager for the routing file change to take effect.

The format of a routing file entry that controls trace logging is:

comp:level,level,level:where:parameter

Table 5. routing file field names 

Field Name Description

comp The component to be traced.

In the discussion above, Tivoli Access Manager Base, WebSEAL, and theTivoli Access Manager Plug-in for Web Servers were referred to ascomponents. These are higher level components than what is now beingdescribed.

The term component is used here to refer to a logical piece of one of these higher level components for which trace logging may be enabled.

Component identifiers are short strings such as ″acl″, ″mgr″, or ″ias″. ″*″can be used to specify that tracing is to be enabled for all components.

Tivoli Access Manager field support personnel should be consulted forthe component identifiers which are relevant for any given problem.

level A list of sub-component levels for each component above.

Each level is separated from the next by a comma. Each level is of theform ″subcomp.n″ where ″subcomp″ is the name of a sub-componentassociated with the ″comp″ above. ″n″ is a number in the range ″1″through ″9″ which indicates the debug level selected for that

subcomponent.

The precise meaning of the debug level for any given sub-component isnot important. The general intention is that ascending from a lower levelto a higher level (1 to 2, for example) increases the level of informationdetail that is logged.

″*″ can be used to specify that tracing is to be enabled for allsub-components. Tivoli Access Manager field support personnel should

 be consulted for the sub-component identifiers which are relevant forany given problem.

where Specifies where the trace log output should be directed.

Valid values include STDERR, STDOUT, FILE (or TEXTFILE), and

DISCARD.parameter Used to specify the fully qualified filename where the trace log data

should be written if either FILE or TEXTFILE were specified above.

The string ″%ld″ can be incorporated into the file name. This will havethe effect of embedding the process-id into the file name.

If ″where″ is specified as STDERR, STDOUT, or DISCARD, then″parameter″ should be specified as a ″-″ to indicate that no parameterwas supplied.

Chapter 3. Using trace 17

Page 34: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 34/116

Example routing file contentsBelow is a sample of text taken from the routing file for Tivoli Access ManagerBase (UNIX platforms):

### The source code for this program is not published or otherwise divested# of its trade secrets, irrespective of what has been deposited with the

# U.S. Copyright Office.## COPYRIGHT NOTICE# Copyright (c) 1990, 1991, 1992, 1993, 1994, 1996 Open Software Foundation, Inc.# ALL RIGHTS RESERVED (DCE).## (c) Copyright IBM Corp. 1994-2002. All Rights Reserved.##

### This is a prototype Policy Director serviceability routing file.## Leading whitespace is ignored, as are lines whose first non-whitespace## character is a #. As installed, some default routing directives are## activated. If the PD_SVC_ROUTING_FILE environment variable is set, it names

## this file; otherwise the value is /opt/PolicyDirector/etc/routing.## (It is not an error if PD_SVC_ROUTING_FILE points to a non-existent file.)

## This file is consulted if no switches were given on the command line, or## if no environment variables (SVC_level or SVC_comp_DBG) could be found.

## The routing of production messages is specified like this:## level:where:parameter## "level" is severity level = FATAL ERROR WARNING NOTICE NOTICE_VERBOSE## or * (for all severities),## "where" is STDERR STDOUT FILE (or TEXTFILE) DISCARD.## "parameter" is the filename where "%ld" becomes the process-id.## To send all notices to the console, then:#*:FILE:/dev/console

## If FILE ends with ".n.m" then at most "n" files of "m" entries## each will be written where ".n" will be appended to each generation of## file. So to keep the last 1000 warnings around in 10 files for all## programs:#NOTICE:FILE.10.100:/var/PolicyDirector/log/msg__syslog.log

## The routing of debug messages is specified like this:## comp:level,level,level:where:parameter## "comp" is the component (acl, mgr, ias, etc.)## "level" is a list of sub-component levels for each component## "where" and "parameter" are as above.## Sub-components are listed in the sams file associated with the component.## Levels are subcomp.n, where n is 1 to 9. They are parsed in order, so## put sub-component wildcard entries first.## A component cannot be named twice -- component debug messages all go## into one place.

## So ACL tracing might be:#acl:*.1,mgmt.9:STDOUT:## Which sets low-level debugging of everything, and full debugging## for the management part.

## Each component can have its own entry. To turn on parts of MGR use## something like the following:#mgr:objmgmt.9,svrmgmt.4:STDOUT:## Other examples:#acl:*.2:FILE:/var/PolicyDirector/log/trace__acl.log#mgr:*.9:FILE:/var/PolicyDirector/log/trace__mgr.log

18 IBM Tivoli Access Manager: Problem Determination Guide

Page 35: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 35/116

## Debug tracing of Core components is meant to be used by the## developers of , not by users. users may use debug tracing## in applications which they write, following the same conventions## illustrated above for the Core.

## If you want more information about the status of the Core you## should activate routing for severity = NOTICE or NOTICE_VERBOSE.## See example routings below.

## In the following default routing directives,## for each severity level, messages will be logged across 32 files with## each file containing no more than 256 messages. The messages will## "wrap around" to the first file after the last file has reached its## limit or when is stopped and restarted. When a log file is## reused, the existing records will be erased. To avoid losing## messages when stopping and starting , use specifications## of the type under "Sequential Logging" below.##

#FATAL:STDOUT:-;FILE.32.256:/var/PolicyDirector/log/msg__fatal.log#ERROR:STDOUT:-;FILE.32.256:/var/PolicyDirector/log/msg__error.log#WARNING:STDOUT:-;FILE.32.256:/var/PolicyDirector/log/msg__warning.log#NOTICE:STDOUT:-;FILE.32.256:/var/PolicyDirector/log/msg__notice.log#NOTICE_VERBOSE:STDOUT:-;FILE.32.256:/var/PolicyDirector/log/msg__verbose.log

## Sequential Logging## To avoid the loss of logged messages, use the following type of routing## specification. Sequential logs will grow without limit, so they## must be erased or pruned periodically.

FATAL:STDOUT:-;FILE:/var/PolicyDirector/log/msg__fatal.logERROR:STDOUT:-;FILE:/var/PolicyDirector/log/msg__error.logWARNING:STDOUT:-;FILE:/var/PolicyDirector/log/msg__warning.logNOTICE:FILE.10.100:/var/PolicyDirector/log/msg__notice.log#NOTICE_VERBOSE:STDOUT:-;FILE:/var/PolicyDirector/log/msg__verbose.log

#### To avoid conflict with## setup during configuration and boot, do not use the STDERR

## message location in routing file directives.###Routing Entries for the Trace Output## Route to STDOUT of each process#*:*.9:STDOUT:-

## Route to a per-process text file and STDOUT of each process#*:*.9:TEXTFILE:/var/PolicyDirector/log/trace__%ld.log;STDOUT:-

## Route to a per-process text file#*:*.9:TEXTFILE.10.1000:/var/PolicyDirector/log/trace__%ld.log

Tivoli Access ManagerPlug-in for Edge Server does not make use of a routing file.The plug-in’s message logging and trace log settings are defined within the objectspace definition configuration file, osdef.conf.

Relevant sections of the osdef.conf file are illustrated below.

# Indicates the category of logging messages that the plug-in will send to# the event log file. Only messages matching the specified category will# be sent to the log file. This option is valid only in the [Global]# section of this file. The default value is ’EWI’.## Log message flag options:# ’E’ - Error messages

Chapter 3. Using trace 19

Page 36: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 36/116

# ’W’ - Warning messages# ’I’ - Informational messages# ’D’ - Debugging messages#logging_flags = EWI

# Indicates the verbosity level for informational and debugging log# messages. This value can range from 0 to 5. A higher number means# that more messages will be logged. This option is valid only in the

# [Global] section of this file. The default value is ’3’.#logging_level = 3

20 IBM Tivoli Access Manager: Problem Determination Guide

Page 37: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 37/116

Chapter 4. Using AutoTrace

AutoTrace is a multi-platform set of debug trace and analysis tools designed toreduce the time required to identify and resolve software problems. The AutoTrace

tool is to be used under the direction of an IBM field support representative. Themajority of the information within this chapter is of interest only to IBM fieldsupport personnel. Customer-oriented directions for reporting AutoTraceinformation to IBM Customer Support are contained in the section entitled ″UsingAutoTrace to collect trace data.″

Section topics:

v “AutoTrace overview” on page 21

v “Building the Tivoli Access Manager product with AutoTrace” on page 22

v “Description of AutoTrace files” on page 23

v “Verifying that AutoTrace is functioning properly” on page 27

v

“Using AutoTrace to collect trace data” on page 29v “AutoTrace output format and example” on page 30

v “Useful AutoTrace commands” on page 31

v “How to create a trace ID file” on page 34

AutoTrace overview

AutoTrace is a multi-platform set of debug trace and analysis tools designed toreduce the time required to identify and resolve software problems. AutoTrace hasnot only been identified as a tool that can improve the serviceability of the IBMTivoli Access Manager product, it also seems likely to become a common debugtrace tool across the set of Tivoli software products. The version of AutoTrace

incorporated into this release of Tivoli Access Manager is version 3.1.5.

Typically the AutoTrace tool is to be used under the direction of an IBM fieldsupport representative.

Features of AutoTrace 3.1.5 include:

v AutoTrace 3.1.5 automatically adds trace instrumentation during a product’s build. At a minimum, instrumented code includes function entry and exit points, but may also include code blocks and thrown and caught exceptions. AutoTracealso traces the values of simple data types that are passed into and returnedfrom each instrumented function.

Debug trace instrumentation beyond what AutoTrace provides automatically canalso be added to product source code manually by developers using theAutoTrace API.

v AutoTrace becomes an integral part of the product’s build environment. It doesnot modify product source code to add its debug trace instrumentation. Instead,it adds its instrumentation to the preprocessor output of the compiler which isthen compiled. All instrumented executables are then linked with an AutoTraceshared library.

v During the product’s build, AutoTrace assigns a unique trace ID (integer) to eachinstrumented function (one trace ID is associated with both the function entryand the function exit). A trace ID is also assigned to each trace hook added tothe code manually using the AutoTrace API.

© Copyright IBM Corp. 2002, 2003 21

Page 38: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 38/116

A file containing a list of these trace ID’s can be used to enable/disable tracingfor the associated trace points. Individual trace ID’s can also beenabled/disabled to fine-tune the set of functions being traced.

How a file of trace ID’s can be created will be discussed later.

v AutoTrace records trace data within a memory segment that is shared betweenthe process performing the tracing and the AutoTrace atctl(.exe) process which

owns/manages the memory segment.The AutoTrace config file is used to define one or more shared memorysegments—also known as tracing ″channels″. The config file is also used toconfigure which products or which processes trace to any given channel.

AutoTrace provides facilities for copying a snapshot of the trace data within achannel to a file. This can be accomplished either via an AutoTrace command orfrom product code via the AutoTrace API. The contents of this file can then beviewed using other AutoTrace tools.

v AutoTrace includes powerful trace analysis and trace profiling utilities whichallow the trace data snapshots to be analyzed either locally or remotely. Theseinclude:

– A text report generator which generates a text report from the captured trace

snapshot– An HTML reporter which generates an HTML report from the trace snapshot

and enables it to be viewed either locally or remotely

– A snapshot file viewer (the most powerful of the trace analysis tools)

Each of these tools reports configurable detail about a trace entry including:

– Time stamp

– Process ID

– Thread ID

– Program name

– Source file name and line number

– Product ID

AutoTrace works alongside the Tivoli Access Manager debug trace facilities whichwere introduced in previous releases. It does not replace them.

Building the Tivoli Access Manager product with AutoTrace

AutoTrace version 3.1.5 has been integrated with the Tivoli Access Manager Base,Tivoli Access Manager WebSEAL, and Tivoli Access Manager Plug-in for WebServers products on the Windows NT, AIX, and Solaris platforms. AutoTraceautomatically adds debug trace instrumentation to all C and C++ code. Allexecutables and shared libraries built as part of the Tivoli Access Manager Baseand WebSEAL are linked with an AutoTrace shared library named libatrc. Thislibrary is installed along with the Tivoli Access Manager runtime.

As AutoTrace instruments each function within the build, it adds to a database filewhich contains information about each instrumented function. This informationincludes the function name, the assigned trace ID, the name of the file containingthe function, and an associated function signature computed from several itemsincluding the name of the function, the names and types of the function’sparameters, and the function’s return type. This database is used by otherAutoTrace tools that do not participate in the build process. This file can be

22 IBM Tivoli Access Manager: Problem Determination Guide

Page 39: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 39/116

thought of as a database of information which associates a given product functionwith a specific trace ID. A trace ID is an integer used to enable and disable tracingfor the associated function.

AutoTrace also supports an exclude file which contains a list of functions whichare not to be automatically instrumented by AutoTrace. These might includefunctions which are invoked frequently and/or functions which are known to be

reliable. The AutoTrace exclude file is mentioned here only to alert the reader thatnot all functions are necessarily instrumented by AutoTrace.

Description of AutoTrace files

The following AutoTrace files are installed as part of the Tivoli Access Managerruntime:

Table 6. AutoTrace file descriptions 

File Description

atserv.exe(Windows NT only)

An NT service required by AutoTrace. Unless this service isinstalled and started, the AutoTrace control program(atctl(.exe)) will fail to initialize. This service is installed andstarted when the Tivoli Access Manager Runtime is installed.

libatrc.dll(Windows)libatrc.a(AIX)libatrc.so(Solaris)libatrc.sl(HP-UX)

The AutoTrace shared library with which each productexecutable and shared library is linked.

atctl(.exe) The AutoTrace control program. When the atctl(.exe) processinitializes, it reads information from two AutoTrace configurationfiles named config and product. It uses this information toallocate the AutoTrace shared-memory segment(s) where tracedata is gathered. The atctl program will be initializedautomatically each time the system is booted. atctl(.exe)commands are also invoked by the user to control tracing.

atinstall.exe(Windows NT only)

Enables the automatic installation and starting of  atserv.exe.

atprintf(.exe) A tool that can be used to manually add comments or additionaltrace information to a trace channel.

atrpt.(exe) Used for viewing and analyzing the contentsof a trace snapshot file. The AutoTracedatabase is used by atrpt(.exe)and can be specified via either:- the –d command line option

- the ATRC_DB environment variable- atrpt(.exe) also allows the user to specifythe location of the database file afteratrpt(.exe) has been started.

AccessManagerBaseAutoTraceDatabaseFile.obfuscatedAccessManagerWebSEALAutoTraceDatabaseFile.obfuscated

Obfuscated copies of the AutoTrace database files which maskTivoli Access Manager file and function names.

Chapter 4. Using AutoTrace 23

Page 40: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 40/116

Table 6. AutoTrace file descriptions (continued) 

File Description

config file The config file contains information which configures thenumber of tracing channels and the size of each, as well as whichproducts and/or processes which will trace to each. Thedirectory which contains this file is communicated to AutoTraceon the atctl init <dir-name> command each time the machine isrebooted. Up to 255 tracing channels can be defined.

This file along with the product file establishes the configurationof the AutoTrace control channel and tracing channels.

Typically each product will ship its own config file. Tivoli AccessManager Base, Tivoli Access Manager WebSEAL, and TivoliAccess Manager Plug-in for Web Servers each ship their ownconfig files.

product file The product file contains an entry which contains the product’sassigned 8-digit ID and the product short name. The directorywhich contains this file must be communicated to AutoTraceusing the atctl init <dir-name> command each time the machine

is rebooted.This file along with the config file establishes the configurationof the AutoTrace control channel and tracing channels.

Typically each product will ship its own product file. TivoliAccess Manager Base, Tivoli Access Manager WebSEAL, andTivoli Access Manager Plug-in for Web Servers each ship theirown product files.

The following AutoTrace files are not included with Tivoli Access Manager:

Table 7. AutoTrace files not included with Tivoli Access Manager 

File Descriptionatdb(.exe) Used for manipulating and extracting information from the

AutoTrace database. The location of the AutoTrace database can be specified to atdb(.exe) either with the –d command lineoption, or via the ATRC_DB environment variable.

atprof(.exe) The AutoTrace trace profiler. This tool enables the user todetermine which functions in the instrumented product are themost heavily used. This information can be used to tune the setof trace ID’s contained within either a ″first failure data capture″trace ID file or an AutoTrace exclude file.

AccessManagerBaseAutoTraceDatabaseFileAccessManagerWebSEALAutoTraceDatabaseFileAMWebPIAutoTraceDatabaseFile

The master AutoTrace database files created by the respectiveproduct builds. They contain information about eachinstrumented function which includes the function name, theassigned trace ID, the name of the file containing the function,and an associated function signature computed from severalitems including the name of the function, the names and types of the function’s parameters and the function’s return type. Afunction’s signature is computed during the build to enableAutoTrace to determine whether a trace ID already exists for thisfunction within the database.

24 IBM Tivoli Access Manager: Problem Determination Guide

Page 41: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 41/116

Description of the AutoTrace product fileThe product file contains an entry which associates a product’s IBM assigned8-digit ID with a product short name. The assigned 8-digit ID is a component of every AutoTrace trace hook added to that product. The product short name is usedon various AutoTrace commands to identify trace hooks or trace ID’s for thatparticular product. The product short name for Tivoli Access Manager Base is

HPD. The product short name for Tivoli Access Manager WebSEAL is DPW. Theproduct short name for Tivoli Access Manager Plug-in for Web Servers is AMZ.

The directory which contains the product file must be communicated to AutoTraceusing the atctl init <dir-name> command each time the machine is rebooted. TheTivoli Access Manager Base, WebSEAL, and Plug-in for Web Servers do thisautomatically.

This file, along with the config file, establishes the configuration of the AutoTracetracing channels. Typically each product ships its own product file. Tivoli AccessManager Base, WebSEAL, and Plug-in for Web Servers each ship their own productfiles.

The AutoTrace product file for Tivoli Access Manager Base contains the followinginformation:

# Product Name Home49420005 HPD -

The AutoTrace product file for Tivoli Access Manager WebSEAL contains thefollowing information:

# Product Name Home49420006 DPW -

The AutoTrace product file for Tivoli Access Manager Plug-in for Web Serverscontains the following information:

# Product Name Home

4942000D AMZ -

The product file for each of the products above can be found in that product’s etcdirectory.

Description of the AutoTrace config fileThe config file contains information which configures the number of tracingchannels and the size of each, as well as the products and/or processes which willtrace to each. The directory which contains this file must be communicated toAutoTrace on the atctl init <dir-name> command each time the machine isrebooted. Up to 255 tracing channels can be defined.

This file along with the product file establishes the configuration of the AutoTracetracing channels.

Typically each product instrumented by AutoTrace will ship its own AutoTraceconfig file. This is true for Tivoli Access Manager Base, Tivoli Access ManagerWebSEAL, and Tivoli Access Manager Plug-in for Web Servers.

The AutoTrace config file for Tivoli Access Manager Base contains the followinginformation which defines:

v A default size for each AutoTrace tracing channel

Chapter 4. Using AutoTrace 25

Page 42: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 42/116

v Trace channel 37 for the pdmgrd process

v Trace channel 38 for the pdacld process

v Trace channel 39 for Tivoli Access Manager Base (HPD)

Refer to the config file itself for more details regarding the nature of its contents.

default:size 1024Knlen 16auth 0

process pdmgrd:chan 37

process pdacld:chan 38

product HPD:chan 39

The AutoTrace config file for Tivoli Access Manager WebSEAL contains thefollowing information:

process webseald:chan 35

process DPW:chan 36

The AutoTrace config file for Tivoli Access Manager Plug-in for Web Serverscontains the following information:

process AMZ:chan 18

The config file for each of the products above can be found in that product’s etcdirectory.

Configuration information within a product’s AutoTrace config file can be used todirect all of that product’s trace output to a single trace channel. For example:

product ABC:chan 5

It is also possible to direct the trace output of a specific process of that product toits own trace channel. For example:

process myABCprocess:chan 6

When channels for both a product and a process within a given product arespecified to AutoTrace, the process stanza will take precedence over the product

stanza. In other words, trace data generated by the process will be directed to thetrace channel associated with that process. Trace data generated by other processeswithin that product will be directed to the trace channel associated with theproduct.

Assume that AutoTrace has been configured to trace two different products.Product DEF and product HIJ are configured as follows within their respectiveAutoTrace config files:

26 IBM Tivoli Access Manager: Problem Determination Guide

Page 43: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 43/116

product DEF:chan 7

process myDEFprocess:chan 8

product HIJ:chan 9

Let’s also assume that myDEFprocess calls functions which reside within a shared

library of product HIJ. If trace ID’s are enabled within myDEFprocess and withinproduct HIJ’s shared library, then the trace information for both will be recordedwithin trace channel 8.

Description of the AutoTrace exclude fileThe AutoTrace exclude file is used only during the product’s build. It is notshipped with the product.

The AutoTrace exclude file contains a list of  functionname filename pairs thatidentify functions which are not to be automatically instrumented by AutoTraceduring the product’s build. Functions that are candidates for inclusion within theexclude file include heavily used functions and those functions which are generally

regarded as stable.

The AutoTrace exclude file is described here only to serve as a reminder that notevery product function has necessarily been instrumented by AutoTrace. Thecontents of the exclude file has no effect on any trace hooks that have beenmanually added to the code via the AutoTrace API. No manual AutoTrace hookshave been used by Tivoli Access Manager in this release.

Verifying that AutoTrace is functioning properly

Initializing atserv.exe (Windows only)On the Windows NT platform only, AutoTrace 3.1.5 provides a Windows NT

service that manages the shared memory segment containing the tracing channels.The executable name is atserv.exe. This service should be installed and startedautomatically by the installation of the Tivoli Access Manager runtime.

From the Windows Control Panel, click on the Services icon and check whetheratserv.exe was successfully installed and started. It will be listed as the″AutoTrace Runtime″. If not, locate the following file:

C:\Program Files\Tivoli\Policy Director\sbin\atinstall.exe

Execute the following command:

MSDOS> atinstall.exe ––quietinstall "c:\Program Files\Tivoli\Policy Director\sbin"

This command should cause atserv.exe to be installed as a service and to bestarted. Observe that ––quietinstall option uses two dashes.

Initializing atctl.(exe)The AutoTrace control program (atctl(.exe)) is located in the .../PolicyDirector/sbin directory. This program should automatically be initialized after theTivoli Access Manager Base product is installed, and again after each system boot.

Verify that the AutoTrace control program has been properly initialized by enteringthe following commands.

Chapter 4. Using AutoTrace 27

Page 44: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 44/116

The atctl info command should produce output similar to the following whenwhen atctl(.exe) has initialized successfully:

C:\Program Files\Tivoli\Policy Director\sbin> atctl infoCHAN SIZE VER NLIFE0 1M 3 035 1M 3 037 1M 3 038 1M 3 0

A channel defined to AutoTrace using the atctl init command may not appearwithin the atctl info output until AutoTrace has actually attempted to trace to thatchannel.

The atctl config command should produce output similar to the following whenTivoli Access Manager Base, Tivoli Access Manager WebSEAL, and Tivoli AccessManager Plug-in for Web Servers are properly installed and configured:

C:\Program Files\Tivoli\Policy Director\sbin> atctl config# last modified: Wed May 29 16:04:06 2002default:

size 1Mnlen 16

product 49420005: # HPDchan 39product 49420006: # DPW

chan 36product 4942000D: # AMZ

chan 18process webseald:

chan 35process pdmgrd:

chan 37process pdacld:

chan 38

The atctl product command should produce output similar to the following whenTivoli Access Manager Base, Tivoli Access Manager WebSEAL, and Tivoli AccessManager Plug-in for Web Servers are properly installed and configured:

C:\Program Files\Tivoli\Policy Director\sbin> atctl product# prod name home49420005 HPD C:\PROGRA~1\Tivoli\POLICY~1\etc49420006 DPW C:\PROGRA~1\Tivoli\PDWeb\etc4942000D AMZ C:\PROGRA~1\Tivoli\PDWebPI\etc

If any of the above commands fail, cd to the Policy Director\sbin directorywhere atctl(.exe) is located and enter the following commands:

(to reinitialize AutoTrace for Tivoli Access Manager Base)

atctl init ..\etc

(to reinitialize AutoTrace for Tivoli Access Manager WebSEAL)

atctl init ..\..\PDWeb\etc

(to reinitialize AutoTrace for Tivoli Access Manager Plug-in for Web Servers)

atctl init ..\..\PDWebPI\etc

where the paths specified are the paths to that product’s config and product files.

The atctl info, atctl config, and atctl product commands above should now allexecute successfully.

28 IBM Tivoli Access Manager: Problem Determination Guide

Page 45: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 45/116

Using AutoTrace to collect trace data

1. Create a file of trace ID’s which are appropriate for the problem beingdebugged. These trace ID’s are used to enable and disable debug tracing withinthe associated functions.

Techniques for how to create a trace ID file are described later in this chapter.Typically a file of trace ID’s will be provided to the customer by a

representative from IBM field support.

2. The selected trace ID’s can be enabled using the following command:

atctl on < product-shortname> <traceID-filename>

HPD is the product short name for Tivoli Access Manager Base. DPW is theproduct short name for Tivoli Access Manager WebSEAL. AMZ is the productshort name for Tivoli Access Manager Plug-in for Web Servers. The short namefor any given product can be learned from its associated AutoTrace product file.

Test whether these trace ID’s were successfully enabled using the followingcommand:

atctl trace < product-shortname>

A list of the trace IDs which have been enabled should appear. These shouldmatch the trace IDs contained within the trace ID file that was used to enablethem.

3. Run the failing product scenario to generate trace data.

4. The following command can be used to capture a snapshot of the trace datawithin the desired tracing channel to a file:

atctl snap <channel-num>

The trace data will be copied to a snap file within the current directory withthe name snap%c_%u.at where %c is the channel number, and %u is anautomatically assigned unique number between 00 and 99. When a channel’s

trace contents are ″snapped″, all trace data within that channel is cleared.5. A text version of the snap file can be generated using the following command:

atrpt -t <snapfile-spec> > textfile

Alternatively, a snap file viewer can be started using the command atrpt withno arguments. This command presents the user with a GUI that provides theuser with a means for selecting the snap file to be viewed and the associatedAutoTrace database file.

All AutoTrace commands that need to know the name of the AutoTracedatabase file look to find that information within an environment variablenamed ATRC_DB. If this environment variable has not been defined to identify

the AutoTrace database file, the name of the AutoTrace database file must beexplicitly specified using the –d command line option.

IBM customers should stop at this point and deliver the AutoTrace snap file toqualified IBM customer support personnel for interpretation.

Chapter 4. Using AutoTrace 29

Page 46: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 46/116

AutoTrace output format and example

The following is an example of an AutoTrace trace entry as it would appear if thesnapshot of the trace channel were formatted such that all available fields wereincluded in the formatted output. This can be accomplished using the followingcommand:

atrpt -t -O* -d AccessManagerBaseAutoTraceDatabaseFile < full-snapfile-path>

where -O* specifies that all available fields be included in the output.

11279 49420005 19605 pdmgrd 3136 0 15:30:28.578386 |ira_ldap_search_ext_s(ld=0x1d7b6b8, base=0x1bdcdf8, scope=2, filter=0x1d713e0,attrs=0x12f278, attrsonly=0, serverctrls=(nil), clientctrls=(nil),timeoutp=0x12f140, sizelimit=2048, res=0x12f14c) [ira_ldap.c,502]

Which AutoTrace database file (Base, WebSEAL, or Plug-in for Web Servers) isspecified on the atrpt command is dependent on the snap file being formatted. Forexample, if the snap file was captured from a WebSEAL trace channel, then theWebSEAL AutoTrace database file must be used to format it.

This AutoTrace trace entry consists of the following fields:

Table 8. AutoTrace entry descriptions 

Field Name Description

line_number The line number of this entry within the trace.(″11279″ in the example above)

product_id The AutoTrace product ID assigned to the product which generatedthe trace entry.(″49420005″ in the example above)

trace_id The AutoTrace trace ID associated with the trace hook whichgenerated the trace entry.(″19605″ in the example above)

process_name The name of the process which generated the trace entry.

(″pdmgrd″ in the example above)

process_id The process ID of the process which generated the trace entry.(″3136″ in the example above)

thread_id The thread ID of the thread which generated the trace entry.(″0″ in the example above)

absolute_time The actual time at which the trace entry was captured.(″15:30:28.578386″ in the example above)

text The text of the trace entry.(″ira_ldap_search_ext_s(ld=0x1d7b6b8, base=0x1bdcdf8, scope=2,filter=0x1d713e0, attrs=0x12f278, attrsonly=0, serverctrls=(nil),clientctrls=(nil), timeoutp=0x12f140, sizelimit=2048, res=0x12f14c)″in the example above.)

src_file The name of the product source file which generated the entry.(″ira_ldap.c″ in the example above)

src_line The line number within the product source file where the entrywas generated.(″502″ in the example above)

Below is a portion of an actual AutoTrace trace file. All of the formattingperformed by the AutoTrace report generator, atrpt, has been lost in the outputpresented below.

30 IBM Tivoli Access Manager: Problem Determination Guide

Page 47: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 47/116

11278 49420005 16335 pdmgrd 3136 0 15:30:28.578383 |ira_internal_search(ld=0x1d7b6b8, base=0x1bdcdf8, scope=2, filter=0x1d713e0,attrs=0x12f278, sizelimit=2048, entries=0x12f298) [ira_entry.c,2349]

11279 49420005 19605 pdmgrd 3136 0 15:30:28.578386 |ira_ldap_search_ext_s(ld=0x1d7b6b8, base=0x1bdcdf8, scope=2, filter=0x1d713e0,attrs=0x12f278, attrsonly=0, serverctrls=(nil), clientctrls=(nil),timeoutp=0x12f140, sizelimit=2048, res=0x12f14c) [ira_ldap.c,502]

11280 49420005 19605 pdmgrd 3136 0 15:30:28.578388 |ira_ldap_search_ext_s: line 636 ira_ldap.c [ira_ldap.c,636]

11281 49420005 19605 pdmgrd 3136 0 15:30:28.585992 |ira_ldap_search_ext_s -> 0 [ira_ldap.c,670]

11282 49420005 16335 pdmgrd 3136 0 15:30:28.585996 |ira_internal_search: line 2378 ira_entry.c [ira_entry.c,2378]

11283 49420005 16335 pdmgrd 3136 0 15:30:28.585999 |ira_internal_search: line 2381 ira_entry.c [ira_entry.c,2381]

11284 49420005 16335 pdmgrd 3136 0 15:30:28.586003 |ira_internal_search: line 2391 ira_entry.c [ira_entry.c,2391]

11285 49420005 16323 pdmgrd 3136 0 15:30:28.586005 |fill_entry(ld=0x1d7b6b8, e=0x1bdb760, msg=0x1bdcd58) [ira_entry.c,1683]

11286 49420005 16322 pdmgrd 3136 0 15:30:28.586064 |fill_attrs(ld=0x1d7b6b8, attrs=0x1bdb764, msg=0x1bdcd58) [ira_entry.c,1608]

11287 49420005 16322 pdmgrd 3136 0 15:30:28.586072 |fill_attrs: line 1664 ira_entry.c [ira_entry.c,1664]

11288 49420005 16322 pdmgrd 3136 0 15:30:28.586074 |fill_attrs -> 0 [ira_entry.c,1668]

11289 49420005 16323 pdmgrd 3136 0 15:30:28.586076 |fill_entry -> 0 [ira_entry.c,1719]

11290 49420005 16335 pdmgrd 3136 0 15:30:28.586078 |ira_internal_search: line 2393 ira_entry.c [ira_entry.c,2393]

11291 49420005 16335 pdmgrd 3136 0 15:30:28.586081 |ira_internal_search: line 2430 ira_entry.c [ira_entry.c,2430]

11292 49420005 16335 pdmgrd 3136 0 15:30:28.586084 |ira_internal_search -> 0 [ira_entry.c,2434]

It is not expected that customers will be performing AutoTrace traces except underthe direction of IBM Customer Support personnel. Therefore, customers willtypically be sending their AutoTrace snap files to IBM Customer Support foranalysis, rather than attempting to format them themselves.

Useful AutoTrace commands

Table 9. AutoTrace commands 

atctl remove all

Clear any configuration or trace data that may exist within shared memory. If thiscommand is run, atctl init must be re-run.

Chapter 4. Using AutoTrace 31

Page 48: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 48/116

Table 9. AutoTrace commands (continued) 

atctl init <dir-name>

Initialize AutoTrace using the config and product files within the directory dir-name.AutoTrace must be initialized in this fashion once per system reboot. This commandinitializes the AutoTrace shared memory (control channel 0 and the tracing channel(s)).Typically, each product is responsible for issuing its own atctl init command specifying its

own copy of  config and product.

atctl info

Examine the channel information to determine whether atctl init was successful.

atctl productDisplay the following information for each product initialized:- product ID- product name- home directoryatctl obtains ″product ID″ and ″product name″ from the product file.

atctl config

Display the current configuration (that is, the information within control channel 0).

atctl config <config-file>

Process the config file and store its information into control channel 0.

atctl version

Display the AutoTrace version.

atctl on HPD all

Activate all trace IDs associated with the HPD product.

atctl off HPD all

Deactivate all trace IDs associated with the HPD product.

atctl on HPD <traceID-file>

Activate all trace IDs associated with the HPD product that are specified within the filetraceID-file. The traceID-file can be a list of integers and/or integer ranges (14025-14050represents an integer range).

atctl off HPD <traceID-file>

Deactivate all trace IDs associated with the HPD product that are specified within the filetraceID-file.

atctl trace HPD

List all trace IDs which are active for product HPD.

atctl snap <channel-num>

Capture a snapshot of the trace data within the specified channel. The data will be copiedto a ″snap file″ within the current directory with the name snap%c_%u.at where %c is thechannel number, and %u is an automatically assigned unique number between 00 and 99.The snap file can then be examined with atrpt. When a channel’s trace contents are″snapped″, all trace data within that channel is cleared.

32 IBM Tivoli Access Manager: Problem Determination Guide

Page 49: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 49/116

Table 9. AutoTrace commands (continued) 

atctl snap <channel-num> < file-spec>

Capture a snapshot of the trace data within the specified channel. Place the trace data inthe specified snap file. The snap file can then be examined with atrpt. When a channel’strace contents are ″snapped″, all trace data within that channel is cleared.

atctl suspend <channel-num>Suspend tracing to a given channel.

atctl resume <channel-num>

Resume tracing to a given channel.

atctl reset <channel-num>

Resets the given channel, thus erasing all trace data that it contains.

atrpt -ti <snapfile-spec>

Display information about the captured snapshot (date/time/machine/channel).

atrpt -t <snapfile-spec> > textfile

Create a text copy of the captured trace snapshot.

NOTE: Some trace fields, such as thread id, are not, by default, included within the atrptoutput.

To add the thread id:

atrpt -t -O tid <snapfile-spec> > textfile

To add the filename and line number where each trace entry was generated:

atrpt -t -O filename -O line <snapfile-spec> > textfile

atrpt <snapfile-spec>

Display the contents of a snapfile using the snap file viewer.

REMINDER: The atrpt command requires the complete file pathname of the AutoTracedatabase file generated by the build. This file may be specified either via the –d < file-spec>argument on the atrpt command line, or it may be specified via the ATRC_DB environmentvariable.

atdb get id <id>

If  id is a function name, return its associated trace ID. If  id is a trace ID, return itsassociated function name.

atdb get id *

Get all of the trace IDs in the AutoTrace database file along with all of the associated

function and file names. This is a good way to begin creating a trace ID file.

Chapter 4. Using AutoTrace 33

Page 50: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 50/116

Table 9. AutoTrace commands (continued) 

atdb group <inputfile(s)> > ffdcfile

Read a file of function name/file name pairs, and generate a file which contains theassociated trace IDs from the AutoTrace database file. A function name may be specifiedeither as simple function name, or as a simple function name and the associated file nameseparated by one or more spaces. The wildcard characters ″*″ and ″?″ may also be used

within function and file names.

This is how a trace ID file would typically be created for collective activation/deactivationat runtime.

More than one input file can be specified:

atdb group inputfile1 inputfile2 > ffdcfile

REMINDER: The atdb command requires the complete file pathname of the AutoTracedatabase file generated by the build. This file may be specified either via the –d < file-spec>argument on the atdb command line, or it may be specified via the ATRC_DB environmentvariable.

atdb is not shipped with the Tivoli Access Manager product. It is intended to be used only by IBM field support personnel.

How to create a trace ID file

The information within this section is intended for IBM field support personnel.

An AutoTrace trace ID is a unique integer value associated with a function withinthe compiled product code. Information about each trace ID is stored within theAutoTrace database file created during the product build process. To view all of these trace IDs along with their associated functions, enter the following command:

atdb –d <database-name> get id *

where database-name represents the name of the AutoTrace database for the productof interest.

Separate AutoTrace databases exist for the Tivoli Access Manager Base andWebSEAL. The name of the AutoTrace database file for the Tivoli Access ManagerBase is AccessManagerBaseAutoTraceDatabaseFile. An obfuscated version of thisfile which masks function and file names is shipped with the product. It cannot beused to create a trace ID file.

A file of trace IDs is used for collectively activating or deactivating the associatedtrace points. Two techniques are described below for creating a trace ID file. Bothuse AutoTrace tools for extracting trace IDs from the AutoTrace database file.

Technique 1Use the following command to get all of the trace IDs from the AutoTrace databasealong with all associated function and file names and copy this information intothe file traceIDfile:

atdb –d <database-name> get id * > traceIDfile

This file can then be manually edited to include only the set of desired trace IDs.

34 IBM Tivoli Access Manager: Problem Determination Guide

Page 51: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 51/116

Use the following command to activate tracing for the trace IDs defined within thetraceIDfile (where HPD represents the short name of the Tivoli Access ManagerBase product):

atctl on HPD traceIDfile

Use a different short name if activating trace points for another product.

Technique 2Create a file of ″function-name and file-name″ pairs for the functions you want totrace. In this discussion, this file is called functionlistfile. Use the atdbcommand to extract the trace IDs associated with those functions from theAutoTrace database file as follows:

atdb –d <database-name> group functionlistfile > traceIDfile

The wildcard characters ″*″ and ″?″ can be used within either the ″function-name″or the ″file-name″. For the Windows platform, when specifying a specific filename,use the .c or .cpp file name extension, as you would expect. On UNIX platforms,use a .i file name extension instead (for example, myfilename.i ).

Use the following command to activate tracing for the trace IDs defined within thetraceIDfile (where HPD represents the short name of the Tivoli Access ManagerBase product).

atctl on HPD traceIDfile

Use a different short name if activating trace points for another product.

Technique 2 is superior to technique 1.

Known problemsAutoTrace allocates each tracing channel separately from shared memory. TheSolaris platform has an environment variable which, by default, limits allocations

of shared memory to a maximum of 1024k bytes.

Modifying the AutoTrace config file to specify a channel size greater than 1024kwill cause the allocation of the AutoTrace shared memory segment to fail unlessthe Solaris environment variable has been updated to allow shared memoryallocations which are larger. This is accomplished by adding the following line tothe /etc/system file and rebooting:

set shmsys:shminfo_shmmax=0xffffffff

If the allocation of the AutoTrace shared memory segment should fail, theperformance of the associated Tivoli Access Manager product will be noticeablyslower. AutoTrace reports its failed shared memory allocation attempt within thesystem error log.

Chapter 4. Using AutoTrace 35

Page 52: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 52/116

36 IBM Tivoli Access Manager: Problem Determination Guide

Page 53: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 53/116

Chapter 5. Message logging

Section contents:

v “Message log file overview” on page 37

v “Logging Base serviceability messages” on page 40

v “Logging WebSEAL serviceability messages” on page 43

v “Configuring default HTTP logging” on page 45

v “Configuring HTTP logging using event logging” on page 48

Message log file overview

IBM Tivoli Access Manager message log files fall into a few general categorieswhich are described below.

Installation log filesTivoli Access Manager installation log files contain messages that are generatedduring the installation of the product. If a failure occurs during the installation of the product, these log files should be consulted. Installation log files include:

Table 10. Installation log files 

Windows < base-install-dir>\log\msg__PDInstall.log

AIX /smit.log

Solaris Refer to the contents of the pkginfo files stored beneath the/var/sadm/pkg directory

HP-UX /var/adm/sw/swinstall.log

Linux /tmp/install.log

When easy installation is used to install the Tivoli Access Manager product, anadditional install log file is created. This install log file is:

Table 11. Easy installation log files 

Windows %TEMP%/msg__amismp.log

UNIX /tmp/msg__amismp.log

Configuration log filesThe Tivoli Access Manager pdconfig command is used to configure the TivoliAccess Manager product. Messages generated during the configuration process arestored within Tivoli Access Manager configuration log files. If a failure occursduring the configuration of the product, these log files should be consulted.

For the Tivoli Access Manager Base product, the configuration log files include:

Table 12. Base configuration log files 

Windows < base-install-dir>\log\msg__config.log

UNIX none

© Copyright IBM Corp. 2002, 2003 37

Page 54: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 54/116

For the the WebSEAL product, the configuration log files include:

Table 13. WebSEAL configuration log files 

Windows <webseal-install-dir>\log\msg__config.log

UNIX /var/pdweb/msg__config.log

Tivoli Access Manager component log filesTivoli Access Manager has several runtime log files that contain messagesgenerated during runtime by specific Tivoli Access Manager components. Ingeneral, each message within these runtime log files has an associated severitylevel which is one of the following:

Table 14. Log file severity levels 

FATAL An unrecoverable error has occurred. The process usuallyterminates, and a core file may be produced. Special recoveryaction may be required. Depending on the nature of the failure,Customer Support may need to be consulted.

ERROR An unexpected, non-terminal, or correctable event. The product

continues to function, but some services or functionality may not be available. Administrative action may be required.

WARNING An event has occurred that is possibly not the desired orrequested result. The program continues to function normally. Theevent may convey information useful for administrative action toavoid a potential error condition. For example, a low memory ordisk space condition.

NOTICE An event has taken place that does not directly require action. Theevent conveys general information about running state or normalactions.

NOTICE_VERBOSE This is similar to NOTICE, but the event may contain moredetailed information.

For the Tivoli Access Manager policy server process, these component runtime logfiles include:

Table 15. Policy server log files 

Windows < base-install-dir>\log\msg__ivmgrd.log

UNIX /var/PolicyDirector/log/msg__ivmgrd.log

For the Tivoli Access Manager authorization server process, these componentruntime log files include:

Table 16. Authorization server log files 

Windows < base-install-dir>\log\msg__ivacld.log

UNIX /var/PolicyDirector/log/msg__ivacld.log

For the Tivoli Access Manager WebSEAL process, these component runtime logfiles include:

Table 17. WebSEAL log files 

Windows <webseal-install-dir>\log\msg__webseald.log

38 IBM Tivoli Access Manager: Problem Determination Guide

Page 55: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 55/116

Table 17. WebSEAL log files (continued) 

UNIX /var/pdweb/log/msg__webseald.log

For the Tivoli Access Manager Plug-in for Web Servers component, the messagelog files and the trace log files are the same. These include:

Table 18. Tivoli Access Manager Plug-in for Web Servers log files Windows Authorization Server log:

<websvrplugin-install-dir>\log\msg__pdwebpi.logIIS plug-in log:<websvrplugin-install-dir>\log\msg__pdwebpi-iis.log

UNIX Authorization Server log:/var/pdwebpi/log/msg__pdwebpi.logWatchDog server log:/var/pdwebpi/log/msg__pdwebpimgr.log

For the Tivoli Access Manager Plug-in for Edge Server component, the message logfiles and the trace log files are the same. For this Tivoli Access Manager

component, tracing is effectively enabled at all times. These log files include:

Table 19. Tivoli Access Manager Plug-in for Edge Server 

Windows <edgesvrplugin-install-dir>\cp\logs\event.date

UNIX /opt/ibm/edge/cp/server_root/logs/event.date

General runtime log filesTivoli Access Manager has several general purpose runtime log files that containmessages generated during runtime. Messages logged by the Tivoli AccessManager policy server to msg__ivmgrd.log and messages logged by the TivoliAccess Manager authorization server to msg__ivacld.log also appear within these

logs. These messages are also categorized either as FATAL, ERROR, WARNING,NOTICE, or NOTICE_VERBOSE. These runtime log files include:

Table 20. General runtime log files 

Windows < base-install-dir>\log\msg__fatal.log< base-install-dir>\log\msg__error.log< base-install-dir>\log\msg__warning.log< base-install-dir>\log\msg__notice.log< base-install-dir>\log\msg__verbose.log

UNIX /var/PolicyDirector/log/msg__fatal.log/var/PolicyDirector/log/msg__error.log/var/PolicyDirector/log/msg__warning.log/var/PolicyDirector/log/msg__notice.log

/var/PolicyDirector/log/msg__verbose.log

For the Tivoli Access Manager Java runtime components, a different set of runtimelog files is used. These runtime log files include:

Chapter 5. Message logging 39

Page 56: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 56/116

Table 21. Java runtime log files 

Windows < base-install-dir>\log\msg__amj_fatal.log< base-install-dir>\log\msg__amj_error.log< base-install-dir>\log\msg__amj_warning.log< base-install-dir>\log\msg__amj_notice.log< base-install-dir>\log\msg__amj_noticeverbose.log

UNIX /var/PolicyDirector/log/msg__amj_fatal.log/var/PolicyDirector/log/msg__amj_error.log/var/PolicyDirector/log/msg__amj_warning.log/var/PolicyDirector/log/msg__amj_notice.log/var/PolicyDirector/log/msg__amj_noticeverbose.log

More detailed information about any given runtime log message can be found inthe IBM Tivoli Access Manager Error Message Reference. The exceptions to thisinclude:

v Tivoli Access Manager Plug-in for Edge Server

Refer to the IBM Tivoli Access Manager Plug-in for Edge Server User’s Guide .

v Tivoli Access Manager for WebSphere Application Server

Refer to the IBM Tivoli Access Manager for WebSphere Application Server User’sGuide.

v Tivoli Access Manager for WebLogic Server

Refer to the IBM Tivoli Access Manager for WebLogic Server User’s Guide.

Logging Base serviceability messages

Tivoli Access Manager Base serviceability messages are controlled by the TivoliAccess Manager Base routing file. The routing file is located in the followingdirectory:

UNIX:

/opt/PolicyDirector/etc/

Windows:

C:\Program Files\Tivoli\Policy Director\etc\

The routing file is an ASCII file that contains additional documentation in theform of comment lines. Entries in this configuration file determine the types of serviceability messages that are logged. Enable any entry by removing thecomment character (#).

The format of a routing file entry that controls message logging is:

level:where:parameter

Table 22. Routing file entry descriptions 

Field Name Description

level Specifies a category of messages which are to be logged. Tivoli AccessManager messages are categorized as either FATAL, ERROR,WARNING, NOTICE, or NOTICE_VERBOSE.

″*″ may be used to specify all message categories.

where Specifies where the trace log output should be directed. Valid valuesinclude STDERR, STDOUT, FILE (or TEXTFILE), and DISCARD.

40 IBM Tivoli Access Manager: Problem Determination Guide

Page 57: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 57/116

Table 22. Routing file entry descriptions (continued) 

Field Name Description

parameter Used to specify the fully qualified filename where the trace log datashould be written if either FILE or TEXTFILE were specified above.

The string ″%ld″ can be incorporated into the file name. This will havethe effect of embedding the process-id into the file name.

If ″where″ is specified as STDERR, STDOUT, or DISCARD, then″parameter″ should be specified as a ″-″ to indicate that no parameterwas supplied.

The routing file includes the following default entries:

UNIX:

FATAL:STDOUT:-;FILE:/var/PolicyDirector/log/msg__fatal.logERROR:STDOUT:-;FILE:/var/PolicyDirector/log/msg__error.logWARNING:STDOUT:-;FILE:/var/PolicyDirector/log/msg__warning.logNOTICE:FILE.10.100:/var/PolicyDirector/log/msg__notice.log#NOTICE_VERBOSE:STDOUT:-;FILE:/var/PolicyDirector/log/msg__verbose.log

Windows:

FATAL:STDERR:-;FILE:%PDDIR%/log/msg__fatal.logERROR:STDERR:-;FILE:%PDDIR%/log/msg__error.logWARNING:STDERR:-;FILE:%PDDIR%/log/msg__warning.logNOTICE:FILE.10.100:%PDDIR%/log/msg__notice.log#NOTICE_VERBOSE:STDOUT:-;FILE:%PDDIR%/log/msg__verbose.log

Note: On a Windows system, the special environment variable PDDIR is set at runtime to the Tivoli Access Manager installation directory.

By default, when Tivoli Access Manager Base runs in the foreground, messages arehandled in the following manner:

1. Messages are sent to the screen (STDOUT, STDERR)

2. Messages are sent to the appropriate configured log file entries in the logdirectory (msg__fatal.log, msg__error.log, msg__warning.log, msg__notice.log,msg__verbose.log)

By default, when Tivoli Access Manager Base runs in the background, messagesare handled in the following manner:

1. Messages are redirected from STDOUT and STDERR and sent to theappropriate server log file as defined in the [logging] stanza of the appropriateserver configuration file:

Table 23. Policy server and authorization server log files 

ServerConfigurationFile Log File Location

Policy Server(pdmgrd)

ivmgrd.conf UNIX:[logging]log-file=/var/PolicyDirector/log/msg__ivmgrd.logWindows:[logging]log-file=<install-path>\log\msg__ivmgrd.log

Chapter 5. Message logging 41

Page 58: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 58/116

Table 23. Policy server and authorization server log files (continued) 

ServerConfigurationFile Log File Location

AuthorizationServer(pdacld)

ivacld.conf UNIX:[logging]log-file=/var/PolicyDirector/log/msg__ivacld.logWindows:

[logging]log-file=<install-path>\log\msg__ivacld.log

2. Messages are also sent to the appropriate configured log file entries in the logdirectory (msg__fatal.log, msg__error.log, msg__warning.log, msg__notice.log,msg__verbose.log)

To enable msg__verbose.log, uncomment the NOTICE_VERBOSE line.

The FILE syntax for the NOTICE message controls log roll over and file recycling:

FILE.< max-files>.< max-records>

The max-file value specifies the number of files that are used.

The max-records value specifies the maximum number of entries per file.

In the default example above, FILE.10.100 means there are 10 files created, eachwith a maximum of 100 entries. The files are named:

msg__notice.log.1msg__notice.log.2...msg__notice.log.10

The messages “wrap around” to the first file after the last file has reached its limitor when the server is stopped and restarted. When a log file is reused, the existingrecords are written over (erased).

Message log entry format and exampleThe following is an example of message log entry taken from a Tivoli AccessManager serviceability log file.

2002-08-06-17:41:56.720+00:00I----- 0x1354A41E webseald ERRORivc socket mtsclient.cpp 1923 0x00000c40Could not connect to the server servername, on port 7135.

42 IBM Tivoli Access Manager: Problem Determination Guide

Page 59: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 59/116

This message log entry consists of the following fields:

Table 24. Message log entry descriptions 

Field Name Description

time The time that the trace log entry was created.(″2002-08-06-17:41:56.720+00:00I″ in the example above)This time is in the following format:

CCYY-MM-DD-hh:mm:ss.fff[+|-]II:iiwhere the digit groupings represent:CCYY - century and yearMM - monthDD - dayhh - hourmm - minutesss - secondsfff - fractions of a secondII:ii - a time inaccuracy factor

message_number A unique message number associated with the message being logged.(″0x1354A41E″ in the example above)

process_ID The name of the process which generated the trace entry.

(″webseald″ in the example above)

severity_level An indicator of the severity of the message log entry.(″ERROR″ in the example above)This field uses one of the following values:″FATAL″, ″ERROR″, ″WARNING″, ″NOTICE″ or ″NOTICE_VERBOSE.

component The component name associated with the program that generatedthe entry.(″ivc″ in the example above)

subcomponent The subcomponent of the program that generated the entry(″socket″ in the example above)

src_file The name of the product source file that generated the entry.(″mtsclient.cpp″ in the example above)

src_line The line number within the product source file where the entrywas generated.(″1923″ in the example above)

thread_id The thread ID of the thread that wrote the message, expressed as ahexadecimal number.(″0x00000c40″ in the example above)

text The text of the trace log message.(″CII EXIT: AznAdministrationSvc::azn_admin_perform_task()″ in theexample above)

Logging WebSEAL serviceability messages

Tivoli Access Manager WebSEAL serviceability messages are controlled by theTivoli Access Manager WebSEAL routing file. The routing file is located in thefollowing directory:

UNIX:

/opt/pdweb/etc/

Windows:

C:\Program Files\Tivoli\PDWeb\etc\

Chapter 5. Message logging 43

Page 60: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 60/116

The routing file is an ASCII file that contains additional documentation in theform of comment lines. Entries in this configuration file determine the types of serviceability messages that are logged. Enable any entry by removing thecomment character (#).

The format of a routing file entry that controls message logging is:

level:where:parameter

Table 25. Routing file entry descriptions 

Field Name Description

level Specifies a category of messages which are to be logged. Tivoli AccessManager messages are categorized as either FATAL, ERROR,WARNING, NOTICE, or NOTICE_VERBOSE.

″*″ may be used to specify all message categories.

where Specifies where the trace log output should be directed. Valid valuesinclude STDERR, STDOUT, FILE (or TEXTFILE), and DISCARD.

parameter Used to specify the fully qualified filename where the trace log datashould be written if either FILE or TEXTFILE were specified above.

The string ″%ld″ can be incorporated into the file name. This will havethe effect of embedding the process-id into the file name.

If ″where″ is specified as STDERR, STDOUT, or DISCARD, then″parameter″ should be specified as a ″-″ to indicate that no parameterwas supplied.

The routing file includes the following default entries:

UNIX:

FATAL:STDERR:-ERROR:STDERR:-

WARNING:STDERR:-#NOTICE:FILE.10.100:/opt/pdweb/log/msg__notice.log#NOTICE_VERBOSE:FILE.10.100:/opt/pdweb/log/msg__notice.log

Windows:

FATAL:STDERR:-ERROR:STDERR:-WARNING:STDERR:-#NOTICE:FILE.10.100:%PDWEBDIR%/log/msg__notice.log#NOTICE_VERBOSE:FILE.10.100:%PDWEBDIR%/log/msg__notice.log

Note: On a Windows system, the special environment variable PDWEBDIR is setat run time to the WebSEAL installation directory.

By default, when WebSEAL runs in the foreground, all messages are sent to thescreen (STDERR).

By default, when WebSEAL runs in the background, messages are redirected fromSTDERR and sent to the WebSEAL server log file as defined in the [logging] stanzaof the webseald.conf configuration file:

44 IBM Tivoli Access Manager: Problem Determination Guide

Page 61: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 61/116

Table 26. WebSEAL log file 

ServerConfigurationFile Log File Location

WebSEALServer(webseald)

webseald.conf UNIX:[logging]server-log=/var/pdweb/log/msg__webseald.logWindows:

[logging]server-log=C:\Program Files\Tivoli\PDWeb\log\msg__webseald.log

To enable msg__verbose.log, uncomment the NOTICE_VERBOSE line.

The FILE syntax for the NOTICE message controls log roll over and file recycling:

FILE.< max-files>.< max-records>

The max-file value specifies the number of files that are used.

The max-records value specifies the maximum number of entries per file.

In the default example above, FILE.10.100 means there are 10 files created, eachwith a maximum of 100 entries. The files are named:

msg__notice.log.1msg__notice.log.2...msg__notice.log.10

The messages ″wrap around″ to the first file after the last file has reached its limitor when the server is stopped and restarted. When a log file is reused, the existingrecords are written over (erased).

Configuring default HTTP logging

Tivoli Access Manager WebSEAL maintains three conventional HTTP log files thatrecord activity rather than messages:

v request.log

v agent.log

v referer.log

By default, these log files are located under the following directory:

UNIX:

/var/pdweb/www/log/

Windows:

C:\Program Files\Tivoli\PDWeb\www\log\

Parameters for configuring standard HTTP logging are located in the [logging]stanza of the webseald.conf configuration file.

The following table illustrates the relationship between the HTTP log files and theconfiguration file parameters:

Chapter 5. Message logging 45

Page 62: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 62/116

Table 27. HTTP log files and configuration file parameters 

Log FilesLocation

ParameterEnable/Disable Parameter

( = yes or no)

request.log requests-file requests

referer.log referers-file referers

agent.log agents-file agents

For example, the entry for the default location of the request.log file appears asfollows:

UNIX:

requests-file = /var/pdweb/www/log/request.log

Windows:

requests-file = \Program Files\Tivoli\PDWeb\www\log\request.log

Enabling and disabling HTTP loggingBy default, all HTTP logging is enabled:

[logging]requests = yesreferers = yesagents = yes

Each log can be enabled or disabled independently from the others. If anyparameter is set to ″no″, logging is disabled for that file.

Once enabled as described above, default WebSEAL HTTP logging is actuallyhandled by the Tivoli Access Manager event logging mechanism, as described in“Configuring HTTP logging using event logging” on page 48.

Specifying the timestamp typeYou can choose to have the timestamps in each log recorded in Greenwich MeanTime (GMT) instead of the local time zone. By default, the local time zone is used:

[logging]gmt-time = no

To use GMT timestamps, set:

gmt-time = yes

Specifying log file rollover thresholdsThe max-size parameter specifies the maximum size to which each of the HTTP

log files may grow and has the following default value (in bytes):[logging]max-size = 2000000

When a log file reaches the specified value — known as its rollover threshold —the existing file is backed up to a file of the same name with an appended currentdate and timestamp. A new log file is then started.

The various possible max-size values are interpreted as follows:

v If the max-size value is less than zero (< 0), then a new log file is created witheach invocation of the logging process and every 24 hours from that instance.

46 IBM Tivoli Access Manager: Problem Determination Guide

Page 63: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 63/116

v If the max-size value is equal to zero (= 0), then no rollovers are performed andthe log file grows indefinitely. If a log file already exists, new data is appendedto it.

v If the max-size value is greater than zero (> 0), then a rollover is performedwhen a log file reaches the configured threshold value. If a log file already existsat startup, new data is appended to it.

Specifying the frequency for flushing log file buffersLog files are written to buffered data streams. If you are monitoring the log files inreal time, you may want to alter the frequency with which the server forces a flushof the log file buffers.

By default, log files are flushed every 20 seconds:

[logging]flush-time = 20

If you specify a negative value, a flush will be forced after every record is written.

HTTP common log format (for request.log)Every response (success or failure) sent back by the Tivoli Access Manager serveris recorded with a one-line entry in the request.log file using following HTTPCommon Log Format:

host - authuser [date] request status bytes

where:

host Specifies the IP address of the requesting machine.

authuser This field takes the value of the From: header of the receivedHTTP request. The value ″unauth″ is used for an unauthenticateduser.

date Specifies the date and time of the request.

request Specifies the first line of the request as it came from the client.

status Specifies the HTTP status code sent back to the requestingmachine.

bytes Specifies the number of bytes sent back to the requesting machine.This value—either the unfiltered content size or a zero size—isconfigured with the log-filtered-pages parameter.

Displaying the request.log fileThe request.log records standard logging of HTTP requests, such as informationon URLs that have been requested and information on the client (for example, IP

address) that made the request.

The following example shows a sample version of a request.log file:

130.15.1.90 - - [26/Jan/2002:17:23:33 -0800] "GET /~am/a_html/ HTTP/1.0" 403 77130.15.1.90 - - [26/Jan/2002:17:23:47 -0800] "GET /icons HTTP/1.0" 302 93130.15.1.90 - - [26/Jan/2002:17:23:59 -0800] "GET /icons/ HTTP/1.0" 403 77130.15.1.90 - - [26/Jan/2002:17:24:04 -0800] "GET /~am/a_html/ HTTP/1.0" 403 77130.15.1.90 - - [26/Jan/2002:17:24:11 -0800] "GET /~am/ HTTP/1.0" 403 77

Chapter 5. Message logging 47

Page 64: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 64/116

Displaying the agent.log fileThe agent.log file records the contents of the User_Agent: header in the HTTPrequest. This log reveals information about the client browser, such as architectureor version number, for each request.

The following example shows a sample version of a agent.log file:

Mozilla/4.01 [en] (WinNT; U)Mozilla/4.01 [en] (WinNT; U)Mozilla/4.01 [en] (WinNT; U)Mozilla/4.01 [en] (WinNT; U)

Displaying referer.logThe referer.log records the Referer: header of the HTTP request. For eachrequest, the log records the document that contained the link to the requesteddocument.

The log uses the following format:

referer -> object

This information is useful for tracking external links to documents in your Webspace. The log reveals that the source indicated by referer contains a link to a pageobject. This log allows you to track stale links and to find out who is creating linksto your documents.

The following example shows a sample version of a referer.log file:

http://manuel/maybam/index.html -> /pics/tivoli_logo.gifhttp://manuel/maybam/pddl/index.html -> /pics/tivoli_logo.gifhttp://manuel/maybam/ -> /pddl/index.htmlhttp://manuel/maybam/ -> /pddl/index.htmlhttp://manuel/maybam/pddl/index.html -> /pics/tivoli_logo.gifhttp://manuel/maybam/ -> /pddl/index.html

Configuring HTTP logging using event loggingWebSEAL HTTP logging can be configured in the [aznapi-configuration] stanza of the webseald.conf configuration file by using the logcfg event logging parameterto define one or more log agents (loggers) that gathers a specific category of loginformation from the event pool and directs this information to a destination:

[aznapi-configuration]logcfg = <category>:{stdout|stderr|file|pipe|remote} [[< param>[=<value>]][,< param>[=<value>]]...]

Refer to the ″Using event logging″ chapter of the IBM Tivoli Access Manager Base Administrator’s Guide for complete event logging configuration details.

The values for category that are appropriate for HTTP logging include:v http

All HTTP logging information

v http.clf

HTTP request information in common log format

v http.ref

HTTP Referer header information

v http.agent

HTTP User_Agent header information

48 IBM Tivoli Access Manager: Problem Determination Guide

Page 65: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 65/116

v http.cof

The NCSA combined format captures HTTP request information (with timestamp) and appends the quoted referer and agent strings to the standardcommon log format.

The following log agent configurations are enabled when the default WebSEALHTTP logging parameters are enabled (see “Configuring default HTTP logging” on

page 45). Note that the log agent configurations accept the values of therequests-file, referers-file, agents-file, flush-time, and max-size parameters fromthe webseald.conf [logging] stanza:

request.log (common log format):

logcfg = http.clf:file path=<requests-file>,flush=<flush-time>, \rollover=< max-size>,log=clf,buffer_size=8192,queue_size=48

referer.log:

logcfg = http.ref:file path=<referers-file>,flush=<flush-time>, \rollover=< max-size>,log=ref,buffer_size=8192,queue_size=48

agent.log (common log format):logcfg = http.agent:file path=<agents-file>,flush=<flush-time>, \rollover=< max-size>,log=agent,buffer_size=8192,queue_size=48

Because default HTTP logging is configured in a different stanza ([logging]) thanevent logging configuration ([aznapi-configuration]), it is possible to have twoduplicate entries for each event appear in a log file when both logging mechanismsare enabled.

The event logging mechanism provides you with much more flexibility ingathering HTTP log information and in customizing its output.

Chapter 5. Message logging 49

Page 66: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 66/116

50 IBM Tivoli Access Manager: Problem Determination Guide

Page 67: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 67/116

Chapter 6. Tracing and logging for Tivoli Access ManagerJava runtime components

Tracing and message logging is controlled differently for the components which areinstalled as part of the IBM Tivoli Access Manager Java runtime.

Tracing and message logging for these components is controlled through aninstalled file named PDJLog.properties which can be found in the< java-home-dir>/etc directory.

The contents of this file enable the user to control:

v Whether tracing is on or off for each of the Tivoli Access Manager Java runtimecomponents

v What tracing level is to be performed for each Tivoli Access Manager Javaruntime component

v

Where the trace output is to be directedv What Java classes within each Tivoli Access Manager Java runtime component

are to be traced

v Whether message logging is on or off for each class of messages (FATAL,ERROR, WARNING, NOTICE, or NOTICEVERBOSE)

v Where the message log output for each class of messages is to be directed

The PDJLog.properties file defines several ″loggers″, each of which is associatedwith one of the components installed by the Tivoli Access Manager Java runtime.These loggers include:

v PDJadminTraceLogger

v PDJasn1TraceLogger

v PDJutilTraceLogger

v PDJtsTraceLogger

v PDJauthzTraceLogger

v PDJsvrsslcfgTraceLogger

For each trace logger, the PDJLog.properties file defines an isLogging attributewhich, when set to the value ″true″, enables tracing for that Tivoli Access Manager

 Java runtime component. A value of ″false″ disables tracing for that component.

PDJLog.properties defines a ″parent″ logger called PDJTraceLogger, which alsohas an isLogging attribute. If the isLogging attribute of each of the individualloggers is set to ″false,″ the isLogging attribute of  PDJTraceLogger can be used to

enable and disable tracing for all of the Tivoli Access Manager Java runtimecomponents simultaneously.

For each trace logger, the PDJLog.properties file defines a mask attribute thatdetermines what levels of tracing are enabled. Valid mask values are 1 through 9.Again, the meaning of any given mask value is unimportant. The general intentionis that ascending from a lower mask value to a higher mask value (1 to 2, forexample) increases the level of information detail that is logged.

© Copyright IBM Corp. 2002, 2003 51

Page 68: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 68/116

Setting a logger’s mask value at a particular level means that all tracing levels upto and including the specified level are enabled. For example, if the user sets themask to a level of 4, then tracing levels 1, 2, and 3 are enabled as well.

PDJLog.properties defines a PDJTraceAllMaskFilter attribute that can also beassigned a mask value of 1 through 9. If the mask attribute of each of theindividual loggers is unset, the PDJTraceAllMaskFilter attribute can be used to

define the same tracing level for all of the Tivoli Access Manager Java runtimecomponents simultaneously.

PDJLog.properties also contains a class attribute for each Tivoli Access Manager Java runtime component, which enables the user to specify which of thatcomponent’s Java classes are to be traced. If no list is specified, all of thatcomponent’s classes are traced. Class names should be separated by spaces.

Message logging for the Tivoli Access Manager Java runtime components iscontrolled in a similar fashion. For each class of messages (FATAL, ERROR,WARNING, NOTICE, or NOTICEVERBOSE), PDJLog.properties defines anattribute that enables the user to specify whether that class of messages is to belogged. The user can also specify where these messages are to be logged.

By default the Tivoli Access Manager Java runtime trace output is directed to thefollowing file:

Windows:

< base-install-dir>\log\trace__amj.log

UNIX:

/var/PolicyDirector/log/trace__amj.log

By default the Tivoli Access Manager Java runtime message log output is directedto the following files:

Windows:

< base-install-dir>\log\msg__amj_fatal.log< base-install-dir>\log\msg__amj_error.log< base-install-dir>\log\msg__amj_warning.log< base-install-dir>\log\msg__amj_notice.log< base-install-dir>\log\msg__amj_noticeverbose.log

UNIX:

/var/PolicyDirector/log/msg__amj_fatal.log/var/PolicyDirector/log/msg__amj_error.log/var/PolicyDirector/log/msg__amj_warning.log/var/PolicyDirector/log/msg__amj_notice.log/var/PolicyDirector/log/msg__amj_noticeverbose.log

Below is an example of the format of a trace log entry:

19:15:38.772 ----- PROGRAM DEBUG4 jadmin MyApp MyApp Main MainHere is our entry...

19:15:38.772 - Timestamp in GMT formatPROGRAM - Program Name (forthcoming)DEBUG4 - Debug Leveljadmin - ComponentMyApp - Subcomponent (i.e. Class)MyApp - Logging class

52 IBM Tivoli Access Manager: Problem Determination Guide

Page 69: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 69/116

Main - Logging methodMain - Thread IDHere is our entry... - The text of the trace log entry

Below is a an example of the contents of the PDJLog.properties file:

# 3 43 1.3.1.5 src/com/tivoli/pd/jras/pdjlog/PDJLog.properties,# pd.jras, am410, 020715a 7/10/02 10:45:14## Licensed Materials - Property of IBM# 5748-XX8# (c) Copyright International Business Machines Corp. 2001# All Rights Reserved# US Government Users Restricted Rights - Use, duplicaion or disclosure# restricted by GSA ADP Schedule Contract with IBM Corp.

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJTraceLogger. It is the parent of all of the other# trace loggers. The only value that should potentially be modified# for this section is the isLogging value.# To turn trace on for all the PDJRTE components: (1) Set the isLogging value# for the PDJTraceLogger to true and (2) comment out below the isLogging# value entry for the individual PDJRTE component trace loggers such as the

# PDJadminTraceLogger.#-----------------------------------------------------------------------

baseGroup.PDJTraceLogger.isLogging=false

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJMessageLogger. The only value that should potentially# be modified for this section is the isLogging value, which is set to true# by default. To turn on messaging for individual handlers attached to# this logger, set the isLogging value for each desired handler, such# as PDJNoticeFileHandler.##-----------------------------------------------------------------------

baseGroup.PDJMessageLogger.isLogging=true

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJTraceAllMaskFilter.# The mask key determines the level at which trace is captured. The valid# trace levels are one of the numerals 1-9. The trace levels are nested.# Specifying a value of 9 includes levels 1-8, specifying a value of 7# includes levels 1-7, and so on.# To set the same mask for all the PDJRTE components:# (1) Set the mask for the PDJTraceAllMaskFilter to the desired mask,# and (2) comment out below the AllMaskFilter entries for the individual# components such as the PDJadminAllMaskFilter.#-----------------------------------------------------------------------

baseGroup.PDJTraceAllMaskFilter.mask=9

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJTraceFileHandler.# The fileName key specifies the file to which the trace output for all# the PDJRTE components is written out. Specify the base# file name only. The fully-qualified location will be:## < pd-home>/<var-dir>/PolicyDirector/log/< file-name>## The default fileName is trace__amj.log.

Chapter 6. Tracing and logging for Tivoli Access Manager Java runtime components 53

Page 70: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 70/116

# If either PDJLog.properties or PD.properties is not found, no logging will# take place.#-----------------------------------------------------------------------

#baseGroup.PDJTraceFileHandler.fileName=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to

# configure a PDJFatalFileHandler.# The fileName key specifies the file to which the fatal error output# for all the PDJRTE components is written out. Specify the base# file name only. The fully-qualified location will be:## < pd-home>/<var-dir>/PolicyDirector/log/< file-name>## The default fileName is msg__amj_fatal.log.# If either PDJLog.properties or PD.properties is not found, no logging will# take place.#-----------------------------------------------------------------------

#baseGroup.PDJFatalFileHandler.fileName=baseGroup.PDJFatalFileHandler.isLogging=true

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJErrorFileHandler.# The fileName key specifies the file to which the error output# for all the PDJRTE components is written out. Specify the base# file name only. The fully-qualified location will be:## < pd-home>/<var-dir>/PolicyDirector/log/< file-name>## The default fileName is msg__amj_error.log.# If either PDJLog.properties or PD.properties is not found, no logging will# take place.#-----------------------------------------------------------------------

#baseGroup.PDJErrorFileHandler.fileName=

baseGroup.PDJErrorFileHandler.isLogging=true

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJWarningFileHandler.# The fileName key specifies the file to which the warning output# for all the PDJRTE components is written out. Specify the base# file name only. The fully-qualified location will be:## < pd-home>/<var-dir>/PolicyDirector/log/< file-name>## The default fileName is msg__amj_warning.log.# If either PDJLog.properties or PD.properties is not found, no logging will# take place.

#-----------------------------------------------------------------------

#baseGroup.PDJWarningFileHandler.fileName=baseGroup.PDJWarningFileHandler.isLogging=true

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJNoticeFileHandler.# The fileName key specifies the file to which the notice output# for all the PDJRTE components is written out. Specify the base# file name only. The fully-qualified location will be:

54 IBM Tivoli Access Manager: Problem Determination Guide

Page 71: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 71/116

## < pd-home>/<var-dir>/PolicyDirector/log/< file-name>## The default fileName is msg__amj_notice.log.# If either PDJLog.properties or PD.properties is not found, no logging will# take place.#-----------------------------------------------------------------------

#baseGroup.PDJNoticeFileHandler.fileName=baseGroup.PDJNoticeFileHandler.isLogging=false

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJNoticeVerboseFileHandler.# The fileName key specifies the file to which the verbose notice output# for all the PDJRTE components is written out. Specify the base# file name only. The fully-qualified location will be:## < pd-home>/<var-dir>/PolicyDirector/log/< file-name>## The default fileName is msg__amj_noticeverbose.log.# If either PDJLog.properties or PD.properties is not found, no logging will# take place.#-----------------------------------------------------------------------

#baseGroup.PDJNoticeVerboseFileHandler.fileName=baseGroup.PDJNoticeVerboseFileHandler.isLogging=false

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJConsoleHandler.## To enable all trace and message output to the console:# (1) set the isLogging attribute for the PDJConsoleHandler.islogging# to true, and# (2) comment out the console handler entries for the other trace and

# message handler entries in this file.## Setting the isLogging property of the console handlers will add them in# with the other handlers, i.e. if the file handlers are turned on, turning# on the console handlers will not turn them off.##-----------------------------------------------------------------------

baseGroup.PDJConsoleHandler.isLogging=false

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJTraceConsoleHandler.#-----------------------------------------------------------------------

#baseGroup.PDJTraceConsoleHandler.isLogging=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJMessageConsoleHandler.#-----------------------------------------------------------------------

#baseGroup.PDJMessageConsoleHandler.isLogging=

#-----------------------------------------------------------------------

Chapter 6. Tracing and logging for Tivoli Access Manager Java runtime components 55

Page 72: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 72/116

# This section shows the key/value pairs that may be specified to# configure a PDJadminTraceLogger.#-----------------------------------------------------------------------

#baseGroup.PDJadminTraceLogger.isLogging=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to

# configure a PDJasn1TraceLogger.#-----------------------------------------------------------------------

#baseGroup.PDJasn1TraceLogger.isLogging=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJutilTraceLogger.#-----------------------------------------------------------------------

#baseGroup.PDJutilTraceLogger.isLogging=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJtsTraceLogger.#-----------------------------------------------------------------------

#baseGroup.PDJtsTraceLogger.isLogging=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure a PDJauthzTraceLogger.#-----------------------------------------------------------------------

#baseGroup.PDJauthzTraceLogger.isLogging=

#-----------------------------------------------------------------------

# This section shows the key/value pairs that may be specified to# configure a PDJsvrsslcfgTraceLogger.#-----------------------------------------------------------------------

#baseGroup.PDJsvrsslcfgTraceLogger.isLogging=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJAdminAllMaskFilter.#-----------------------------------------------------------------------

#baseGroup.PDJadminAllMaskFilter.mask=

#-----------------------------------------------------------------------

# This section shows the key/value pairs that may be specified to# configure an PDJutilAllMaskFilter.#-----------------------------------------------------------------------

#baseGroup.PDJutilAllMaskFilter.mask=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJasn1AllMaskFilter.#-----------------------------------------------------------------------

#baseGroup.PDJasn1AllMaskFilter.mask=

56 IBM Tivoli Access Manager: Problem Determination Guide

Page 73: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 73/116

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJtsAllMaskFilter.#-----------------------------------------------------------------------

#baseGroup.PDJtsAllMaskFilter.mask=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJauthzAllMaskFilter.#-----------------------------------------------------------------------

#baseGroup.PDJauthzAllMaskFilter.mask=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJsvrsslcfgAllMaskFilter.#-----------------------------------------------------------------------

#baseGroup.PDJsvrsslcfgAllMaskFilter.mask=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJadminClassFilter. Classes in PDJLog are treated as# subcomponents. Modify the "classes" value to turn on/off the logging# of different components. A blank value or absent classes qualifier means# all components will be logged.#-----------------------------------------------------------------------

#baseGroup.PDJadminClassFilter.classes=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJutilClassFilter. Classes in PDJLog are treated as# subcomponents. Modify the "classes" value to turn on/off the logging# of different components. A blank value or absent classes qualifier means# all components will be logged.#-----------------------------------------------------------------------

#baseGroup.PDJutilClassFilter.classes=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJasn1ClassFilter. Classes in PDJLog are treated as# subcomponents. Modify the "classes" value to turn on/off the logging

# of different components. A blank value or absent classes qualifier means# all components will be logged.#-----------------------------------------------------------------------

#baseGroup.PDJasn1ClassFilter.classes=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJtsClassFilter. Classes in PDJLog are treated as# subcomponents. Modify the "classes" value to turn on/off the logging# of different components. Absence of this qualifier means all components# will be logged.

Chapter 6. Tracing and logging for Tivoli Access Manager Java runtime components 57

Page 74: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 74/116

#-----------------------------------------------------------------------

#baseGroup.PDJtsClassFilter.classes=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to

# configure an PDJauthzClassFilter. Classes in PDJLog are treated as# subcomponents. Modify the "classes" value to turn on/off the logging# of different components. Absence of this qualifier means all components# will be logged.#-----------------------------------------------------------------------

#baseGroup.PDJauthzClassFilter.classes=

#-----------------------------------------------------------------------# This section shows the key/value pairs that may be specified to# configure an PDJsvrsslcfgClassFilter. Classes in PDJLog are treated as# subcomponents. Modify the "classes" value to turn on/off the logging# of different components. Absence of this qualifier means all components# will be logged.#-----------------------------------------------------------------------

#baseGroup.PDJsvrsslcfgClassFilter.classes=

58 IBM Tivoli Access Manager: Problem Determination Guide

Page 75: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 75/116

Chapter 7. Using WebSEAL statistics

IBM Tivoli Access Manager WebSEAL provides a series of built-in softwaremodules that, when enabled, can monitor specific server activity and collect

information about those activities. At any instance in time, you can display thestatistics information gathered since that module was enabled. In addition, you candirect this statistics information to log files.

When you display statistics information, you see a ″snapshot″ of the informationsince the module was enabled. The information gathered by WebSEAL statisticsprovides a relative view of the activity being recorded. If statistics are captured atregular intervals over a period of time, you can generate a graphical view of therelative relationship of the server activities.

pdadmin stats command syntax

Use the pdadmin stats command to manage statistics components. This sectiondescribes the valid operations for the pdadmin stats command:

Basic pdadmin stats commandpdadmin> server task webseald-<instance> stats <command>

You can perform the following tasks with the pdadmin stats command:

Table 28. pdadmin stats commands 

stats on Enable statistics dynamically, by component

stats off Disable statistics, by component or all components at once

stats show List enabled components

stats get Display current statistics values by component or all components at once

stats reset Reset statistics values by component or all components at once

stats list List all available statistics components

Enable statistics dynamicallyYou can enable statistics reporting dynamically with the pdadmin stats oncommand or statically with configuration parameters in the webseald.confconfiguration file.

Use the pdadmin stats on command to enable statistics gathering and set thestatistics report frequency, count, and destination for a component.

stats on <component> [<interval> [<count>]] [<logagent>]

Table 29. pdadmin stats command arguments 

Argument Description

component The statistic component name. Required. Statistics are gathered inWebSEAL memory for this component. Statistics for this component canalso be recorded in a log file by specifying the optional arguments tothis command.

© Copyright IBM Corp. 2002, 2003 59

Page 76: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 76/116

Table 29. pdadmin stats command arguments (continued) 

Argument Description

interval The time interval between reports of information. This argument isoptional and results in statistics being sent to a log file. When thisoption is specified, statistics are sent, by default, to standard out of theWebSEAL server, which is the WebSEAL log file. You can specifyanother output location using the logagent argument. If  interval is notspecified, no statistics are sent to any log file. However, the statisticcomponent is still enabled. You can obtain reports dynamically at anytime using pdadmin stats get.

count The number of reports sent to a log file. This argument is optional andrequires that the interval argument be specified. If  interval is specifiedwithout count, the duration of reporting is indefinite. After the countvalue is reached, reporting to a log file stops. However, the statisticcomponent is still enabled. You can obtain reports dynamically at anytime using pdadmin stats get.

logagent Optionally specifies a destination for the statistics information gatheredfor the specified component. Refer to the ″Using event logging″ chapterof the IBM Tivoli Access Manager Base Administrator’s Guide for completeconfiguration details.

Note: By default, the pdweb.threads, pdweb.doccache, and pdweb.jmtcomponents are always enabled and cannot be disabled.

See also “Enabling statistics statically using event logging” on page 67.

Example 1:

This example enables the pdweb.http component. Because the interval option is notspecified, you can only obtain statistics information for this componentdynamically using pdadmin stats get.

pdadmin> server task webseald-<instance> stats on pdweb.http

Example 2:

This example enables the pdweb.http component. Because the interval argument isspecified, the information is sent (by default) to the standard WebSEAL log file.The interval and count arguments cause the log file to accumulate 100 entriesrepresenting statistics reports 20 seconds apart.

pdadmin> server task webseald-<instance> stats on pdweb.http 20 100

Example 3:

This example enables the pdweb.http component. The logagent argument uses

event logging configuration to specify a destination file for the statisticsinformation. The interval argument (without a count value) sends statisticsinformation for this component to the log file every 20 seconds indefinitely. Thegrowth of the log file is controlled by the rollover_size parameter. Refer to the″Using event logging″ chapter of the IBM Tivoli Access Manager Base Administrator’sGuide for complete event logging configuration details.

pdadmin> server task webseald-<instance> stats on pdweb.http 20 filepath=/tmp/jmt-stats.log,rollover_size=-1,flush_interval=20

Example 4:

60 IBM Tivoli Access Manager: Problem Determination Guide

Page 77: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 77/116

This example illustrates a limitation of managing statistics dynamically. The firstcommand enables the pdweb.http component and directs statistics information tothe A.log file. The second command attempts to activate a second log file, B.log.However, this action actually results in deactivating A.log while activating B.log.

pdadmin> server task webseald-<instance> stats on pdweb.http 20 file path=/tmp/A.logpdadmin> server task webseald-<instance> stats on pdweb.http 20 file path=/tmp/B.log

Disable statisticsDisable statistics gathering for a component or all components at once.

stats off [<component>]

Example:

pdadmin> server task webseald-<instance> stats off pdweb.sescache

Note: By default, the pdweb.threads, pdweb.doccache, and pdweb.jmtcomponents are always enabled and cannot be disabled.

Show enabled statistics components

List all enabled statistics components or a specific enabled component. If aspecified component is not enabled, no output is displayed.

stats show [<component>]

Example 1:

pdadmin> server task webseald-<instance> stats showpdweb.authnpdweb.doccachepdweb.jmtpdweb.sescachepdweb.threads

Example 2:

pdadmin> server task webseald-<instance> stats show pdweb.authnpdweb.authn

Get current statistic values dynamicallyShow the current values of statistics being gathered by a component or all enabledcomponents.

stats get [<component>]

Example:

pdadmin> server task webseald-<instance> stats get pdweb.threadsactive:4total:50

Reset statistics valuesReset the values being gathered by an enabled component or all enabledcomponents at once.

stats reset [<component>]

Example:

pdadmin> server task webseald-<instance>> stats reset pdweb.threads

Chapter 7. Using WebSEAL statistics 61

Page 78: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 78/116

List all available statistics componentsList all components available to gather and report statistics.

stats list

Example:

pdadmin> server task webseald-<instance> stats list

pd.ras.stats.monitorpd.log.EventPool.queuepd.log.file.clfpd.log.file.refpd.log.file.agentpdweb.authnpdweb.authzpdweb.httppdweb.httpspdweb.threadspdweb.jmtpdweb.sescachepdweb.doccachepdweb.jct.1

Statistics components and activity typesThis section describes the statistics components available to Tivoli Access ManagerWebSEAL.

pdweb.authn componentThe pdweb.authn statistics component gathers information related to WebSEALauthentication. The following table describes the types of information available:

Table 30. pdweb.authn information types 

Type Description

pass The total number of successful authentications

fail The total number of failed authentications

pwd exp The total number of authentication attempts made with an expiredpassword

max The maximum time for a single authentication process

avg The average time for a single authentication process

total The total time for all authentication processing

Example:

pdadmin> server task webseald-<instance> stats get pdweb.authnpass : 2fail : 1pwd exp : 0max : 0.178avg : 0.029total : 0.382

pdweb.authz componentThe pdweb.authz statistics component gathers information related to WebSEALauthorization. The following table describes the types of information available:

62 IBM Tivoli Access Manager: Problem Determination Guide

Page 79: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 79/116

Table 31. pdweb.authz information types 

Type Description

pass The total number of successful authorization requests (how manyresources were successfully accessed)

fail The total number of failed authorization requests

Example:

pdadmin> server task webseald-<instance> stats get pdweb.authzpass : 2fail : 1

pdweb.http componentThe pdweb.http statistics component gathers information related to WebSEALHTTP communication. The following table describes the types of informationavailable:

Table 32. pdweb.http information types 

Type Description

reqs The total number of HTTP requests received

max-worker The maximum time used by a single worker thread to process an HTTPrequest

total-worker The total time used by all worker threads that process HTTP requests

max-webseal The maximum time used to process a single HTTP request - measuredinside the worker thread, after the request headers have been read, andeliminating connection setup overhead

total-webseal The total time used to process all HTTP requests - measured inside theworker threads, after the request headers have been read, and eliminatingconnection setup overhead

Example:

pdadmin> server task webseald-<instance> stats get pdweb.httpreqs : 0max-worker : 0.000total-worker : 0.000max-webseal : 0.000total-webseal : 0.000

pdweb.https componentThe pdweb.https statistics component gathers information related to WebSEALHTTPS communication. The following table describes the types of informationavailable:

Table 33. pdweb.https information types 

Type Description

reqs The total number of HTTPS requests received

max-worker The maximum time used by a single worker thread to process an HTTPSrequest

total-worker The total time used by all worker threads that process HTTPS requests

max-webseal The maximum time used to process a single HTTPS request - measuredinside the worker thread, after the request headers have been read, andeliminating connection setup overhead

Chapter 7. Using WebSEAL statistics 63

Page 80: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 80/116

Table 33. pdweb.https information types (continued) 

Type Description

total-webseal The total time used to process all HTTPS requests - measured inside theworker threads, after the request headers have been read, and eliminatingconnection setup overhead

Example:

pdadmin> server task webseald-<instance> stats get pdweb.httpsreqs : 0max-worker : 0.000total-worker : 0.000max-webseal : 0.000total-webseal : 0.000

pdweb.threads componentThe pdweb.threads statistics component gathers information related to WebSEALworker thread activity. This component is always enabled by default and cannot bedisabled. The following table describes the types of information available:

Table 34. pdweb.threads information types 

Type Description

active The total number active worker threads handling requests

total The total number of configured worker threads

Example:

pdadmin> server task webseald-<instance> stats get pdweb.threadsactive : 0total : 50

pdweb.jmt componentThe pdweb.jmt statistics component gathers information related to the WebSEAL

 junction mapping table. This component is always enabled by default and cannot be disabled. The following table describes the types of information available:

Table 35. pdweb.jmt information types 

Type Description

hits The total number of requests that required URL mapping via the junctionmapping table

Example:

pdadmin> server task webseald-<instance> stats get pdweb.jmt

hits : 5

pdweb.sescache componentThe pdweb.sescache statistics component gathers information related to theWebSEAL session/credential cache activity. The following table describes the typesof information available:

64 IBM Tivoli Access Manager: Problem Determination Guide

Page 81: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 81/116

Table 36. pdweb.sescache information types 

Type Description

hit The number of requests that resulted in a session cache hit - that is, theuser had a session cache entry and it was successfully referenced

miss The number of requests that missed a session cache hit

add The number of entries that have been added to the session cache

del The number of entries that have been deleted from the cache

inactive The number of entries removed from the cache because the inactivitytimeout value had expired

lifetime The number of entries removed from the cache because the lifetimetimeout value had expired

LRU expired The number of times a ″Least Recently Used″ cache entry is expired orremoved to make room for a new entry.

Example:

pdadmin> server task webseald-<instance> stats get pdweb.sescache

hit : 0miss : 0add : 0del : 0inactive : 0lifetime : 0LRU expired : 0

pdweb.doccache componentThe pdweb.doccache statistics component gathers information related to WebSEALdocument caching activity. This component reports statistics for all MIME typesenabled in the [content-cache] stanza of the webseald.conf configuration file. Thiscomponent is always enabled by default and cannot be disabled.

The following table describes the types of global information available for allMIME types:

Table 37. pdweb.doccache information types 

Type Description

General Errors The number of errors reported by the pdweb.doccachecomponent when the there are memory allocation failures,initialization failures, and invalid MIME type header values

Uncachable The number of instances when there is no cache defined for theMIME type of the document to be cached

Pending Deletes The number of entries marked for deletion, but are still in use

Pending Size The number of bytes used by entries marked for deletion but arestill in use

Misses The number of times a URL is looked up in the document cacheand not found. A found cached document eliminates the need toaccess the real document again.

Cache MIME type The MIME type of documents stored in this cache.

Chapter 7. Using WebSEAL statistics 65

Page 82: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 82/116

Cache MIME type statistics:

Table 38. Cache MIME type statistics 

Type Description

Max size The maximum combined byte size of all documents in the cache.

Max entry size The maximum byte size for any single cached document. If thedocument size exceeds this internally calculated value, it is notcached.

Size The total byte count for all documents currently residing in thecache

Count The current number of entries in the cache.

Hits The number of successful lookups (documents successfully foundin the cache)

Stale hits The number of successful lookups that found an entry that was tooold and was purged instead

Create waits The number of times subsequent requests for a document are blocked (made to wait) while the document content is initially being cached

Cache no room The number of times a document that is valid for caching cannotfit into the cache because there are too many entries being createdat the same time

Additions The number of successful new entries in the cache

Aborts The number of times the creation of a new cache entry is aborted because of problems or a header that indicates the entry shouldnot be cached

Deletes The number of cache entries deleted because the entry is stale(expired) or the creation was aborted

Updates The number of entries that have had expiry times updated

Too big errors The number of attempts to cache documents that exceed the

maximum entry size (and therefore are not cached)MT errors The number of times more than one thread tries to create the same

entry in the cache (MT=Multi-Threading)

Example:

pdadmin> server task webseald-<instance> stats get pdweb.doccacheGeneral Errors : 0Uncachable : 0Pending Deletes: 0Pending Size : 0Misses : 0Cache MIME type : text/html

Max size : 2048000

Max entry size : 128000Size : 0Count : 0Hits : 0Stale hits : 0Create waits : 0Cache no room : 0Additions : 0Aborts : 0Deletes : 0Updates : 0Too big errors : 0MT errors : 0

66 IBM Tivoli Access Manager: Problem Determination Guide

Page 83: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 83/116

pdweb.jct.# componentThe pdweb.jct.# statistics component gathers information related to WebSEAL

 junctions. The following table describes the types of information available:

Table 39. pdweb.jct.# information types 

Type Description

[/] The actual junction name (listed as a number in the command)reqs The total number of requests routed across this junction

max The maximum time consumed in a single request across this junction

total The total time consumed by requests across this junction

Example:

pdadmin> server task webseald-<instance> stats get pdweb.jct.1[/]reqs : 0max : 0.000total : 0.000

Enabling statistics statically using event logging

Statistics can be statically configured in the [aznapi-configuration] stanza of thewebseald.conf configuration file by using the stats parameter to enable thestatistics interface and using the logcfg event logging parameter to direct thestatistics information to a destination:

[aznapi-configuration]stats = <component> [interval [count]]logcfg = stats.<component>:<destination>

Refer to “Enable statistics dynamically” on page 59 for information on the intervaland count arguments.

Refer to the ″Using event logging″ chapter of the IBM Tivoli Access Manager Base Administrator’s Guide for complete event logging configuration details.

Example 1:

In this example, the stats parameter enables the component and any interval andcount arguments. The logcfg parameter specifies the destination for thisinformation. Refer to the ″Using event logging″ chapter of the IBM Tivoli Access

 Manager Base Administrator’s Guide for complete configuration details.

[aznapi-configuration]stats = pdweb.jmt 20logcfg = stats.pdweb.jmt:file path=/tmp/jmt.log,rollover_size=-1,flush=20

Example 2:

In this example, multiple stats parameters enable multiple components. Multiplelogcfg parameters specify multiple destinations. Note that, unlike the dynamicstats on command, you can specify several destination files for the samecomponent. Refer to the ″Using event logging″ chapter of the IBM Tivoli Access

 Manager Base Administrator’s Guide for complete event logging configuration details.

[aznapi-configuration]stats = pdweb.jmt 20stats = pdweb.authn 40

Chapter 7. Using WebSEAL statistics 67

Page 84: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 84/116

stats = pdweb.jct.1 50logcfg = stats.pdweb.jmt:file path=/tmp/jmtA.log,rollover_size=-1,flush=20logcfg = stats.pdweb.jmt:file path=/tmp/jmtB.log,rollover_size=-1,flush=20logcfg = stats.pdweb.authn:file path=/tmp/an.log,rollover_size=-1,flush=20logcfg = stats.pdweb.jct.1:file path=/tmp/jct.log,rollover_size=-1,flush=20

68 IBM Tivoli Access Manager: Problem Determination Guide

Page 85: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 85/116

Chapter 8. Basic troubleshooting

Section topics:

v “Verifying software installation” on page 69

v “Verifying the functionality of individual components” on page 70

Verifying software installation

The installation of Tivoli Access Manager involves the installation andconfiguration of a number of prerequisite software products and Tivoli AccessManager components. Operational failures for the Tivoli Access Manager productcan result from the failure to install all required prerequisite software, or the failureto install the correct level of this software.

Always re-examine your installed software and software levels if either:

v A new Tivoli Access Manager installation fails to work properly

v An existing Tivoli Access Manager installation fails to work properly afterupdating one or more prerequisite products or Tivoli Access Managercomponents

The Tivoli Access Manager installation guides and Release Notes provide detailedinformation about specific software requirements that must be satisfied beforeTivoli Access Manager can be successfully installed and configured. Theserequirements include supported operating systems, prerequisite software, andrequired patches. Be sure to note the release or level of each software item.

The Tivoli Access Manager technical documentation also describes exactly whatsoftware must be installed on any given Tivoli Access Manager system. For

example, the IBM Tivoli Access Manager Base Installation Guide informs you that asystem containing the Tivoli Access Manager policy server must, at a minimum,have the following software installed:

v IBM Global Security Toolkit (GSKit)

v IBM SecureWay Directory LDAP Client

v Tivoli Access Manager Runtime

v Tivoli Access Manager Policy Server

Tools exist that enable you to determine whether the correct software, and level, isinstalled upon any given Tivoli Access Manager system. Tivoli Access Managersupplies a utility named pdversion that automatically lists all Tivoli AccessManager components installed on that system along with their version numbers.

The pdversion utility does not list any prerequisite software, such as the IBMSecureWay Directory LDAP Client or the IBM Global Security Toolkit (GSKit). Inmost cases, the presence or absence of this prerequisite software can be determinedusing native operating system utilities.

v For the Windows platforms, use the Add/Remove Programs icon on theWindows Control Panel

v For AIX, use the lslpp –l command

v For Solaris, use the pkginfo –l command

v For Linux, use the rpm –qa command

© Copyright IBM Corp. 2002, 2003 69

Page 86: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 86/116

v For HP-UX, use the swlist command

Even with these tools, certain packages on certain platforms may still be difficult tolocate. GSKit on the Windows platforms, for example, adds no entry to theWindows Add/Remove Program list. On the Windows platform, GSKit is typicallyinstalled in the following directory:

C:\Program Files\IBM\gsk5

The following command can be used to display the GSKit version:

MSDOS> C:\Program Files\IBM\gsk5\bin\gsk5ver.exe

Use these tools in conjunction with the software requirements listed within theTivoli Access Manager installation guides to ensure that all the required software isinstalled upon each machine within your Tivoli Access Manager configuration.

Verifying the functionality of individual components

A typical installation of the Tivoli Access Manager product involves a number of prerequisite products and Tivoli Access Manager components. Tivoli Access

Manager will experience operational problems if any of these prerequisite productsare not installed and configured properly.

This section describes how a user can verify whether certain prerequisite softwareproducts or Tivoli Access Manager components are generally operating properly. If any operational irregularities are discovered, refer to the appropriate installationand configuration instructions for that product or Tivoli Access Managercomponent.

The following table illustrates the components that are required with various typesof Tivoli Access Manager systems.

Table 40. Required components for Tivoli Access Manager system types 

Tivoli Access ManagerSystem Type

Required Software Components

Policy Server – IBM Global Security Toolkit (GSKit)– Tivoli Access Manager runtime– IBM SecureWay Directory client– Tivoli Access Manager policy server

Runtime system – IBM Global Security Toolkit (GSKit)– Tivoli Access Manager runtime– IBM SecureWay Directory client

Development system – IBM Global Security Toolkit (GSKit)– Tivoli Access Manager runtime– IBM SecureWay Directory client– Tivoli Access Manager application development kit

– Tivoli Access Manager Java runtime(required for developing Java applications)

Authorization server – IBM Global Security Toolkit (GSKit)– Tivoli Access Manager runtime– IBM SecureWay Directory client– Tivoli Access Manager authorization server

Note: If you plan to install Active Directory as your user registry, the IBMSecureWay Directory client is not required on Tivoli Access Managersystems in your secure domain.

70 IBM Tivoli Access Manager: Problem Determination Guide

Page 87: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 87/116

IBM Global Security Toolkit (GSKit)The two most common problems with the IBM Global Security Toolkit include:

v GSKit was not installed

v The wrong version of GSKit is installed (perhaps left over from a previous TivoliAccess Manager installation)

You can validate the correct installation of GSKit by executing the followingcommand:

gsk5ver(.exe)

This command invokes all of the GSKit shared libraries and displays versioninformation about each library.

LDAP serverCommunication between the IBM SecureWay LDAP client and the LDAP servercan be tested using the ldapsearch command. This command also reveals theversion of the LDAP server software. This command (entered as one line) can beexecuted from any machine with a copy of the IBM SecureWay Directory Client

installed.ldapsearch -h <ldapserver-hostname> -p 389 -D "<ldapadminDN>" \-w <ldapadmin-password> -b "" -s base objectclass=*

The output of this command varies depending on which supported LDAP serveryou are using. The following output is generated by an IBM SecureWay LDAPServer:

namingcontexts=CN=SCHEMAnamingcontexts=CN=LOCALHOSTsubschemasubentry=cn=schemasupportedextension=1.3.18.0.2.12.1supportedextension=1.3.18.0.2.12.3supportedextension=1.3.18.0.2.12.5supportedextension=1.3.18.0.2.12.6supportedcontrol=2.16.840.1.113730.3.4.2supportedcontrol=1.3.18.0.2.10.5supportedcontrol=1.2.840.113556.1.4.473supportedcontrol=1.2.840.113556.1.4.319security=noneport=389supportedsaslmechanisms=CRAM-MD5supportedldapversion=2supportedldapversion=3ibmdirectoryversion=4.1ibm-ldapservicename=ldapserverhostname.domainnameibm-adminid=CN=ROOTibm-servertype=masteribm-supportedacimechanisms=1.3.18.0.2.26.2vendorname=International Business Machines (IBM)

vendorversion=4.1ibm-sslciphers=N/A

Alternatively, if you have SSL configured for your LDAP server, you can test theSSL connection using the following command (entered as one line):

ldapsearch -h <ldapserver-hostname> -p 636 -D "<ldapadminDN>" \-w <ldapadmin-password> -Z -K <client-keyfile> \-P <key-password> -b "" -s base objectclass=*

The output should be similar to that listed above.

Chapter 8. Basic troubleshooting 71

Page 88: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 88/116

IBM SecureWay Directory clientYou can determine the version of the IBM SecureWay Directory client by issuingthe following command:

ldapsearch –e

The verification procedure described above for the LDAP server used the IBM

SecureWay Directory client that had been installed on the LDAP server systemitself. However, unless you are using Active Directory for your Tivoli AccessManager user registry, eachTivoli Access Manager system requires the installationof the IBM SecureWay Directory client, not just the machine with the LDAP serverinstalled.

The verification procedure described above for the LDAP server, is alsoappropriate for verifying the functionality of the IBM SecureWay Directory clienton each Tivoli Access Manager system.

Active Directory

Client/server configurationServer

Start the MMC for Active Directory using the following menus:

Start > Program > Administrative Tools > Active Directory Users and Computers

If the Active Directory management console started successfully and you can browse all the objects in Active Directory, the Active Directory server hascompleted its configuration correctly. Otherwise, you can perform the proceduresfor unconfiguration and reconfiguration of the Active Directory described in theIBM Tivoli Access Manager Base Installation Guide.

Client

The client has to be configured into an existing Active Directory domain in orderto perform Tivoli Access Manager configuration. To ensure that the client machineis part of the Active Directory domain, you can use System Properties to ensurethe correct configuration of the client machine:

Start > Settings > Control Panel > System > System Properties

In the Control Panel, double click on System icon. The System Properties windowappears. On the System Properties windows, click the Network Identificationmenu. If the Domain on the Network Identification contains the correct ActiveDirectory domain, it indicates the client machine is properly configured into theActive Directory domain.

Version numberServer

Active Directory is included with Windows 2000 Advance Server installation.Therefore, only one version of this software is possible. After successful ActiveDirectory server configuration, the Active Directory server is started automaticallyduring the reboot process.

Client

72 IBM Tivoli Access Manager: Problem Determination Guide

Page 89: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 89/116

Active Directory is included with Windows 2000 Advance Server installation.Therefore, only one version of this software is possible.

Confirm client-server connection through ADSIInstall ″Windows 2000 Support Tools″ in the SUPPORT\TOOLS directory on theWindows 2000 operating system CD. From that directory, start setup.exe andfollow the installation guide to complete the ″Windows 2000 Support Tools″

installation.

Activate the ADSI Edit MMC window by starting:

Start > Program > Windows 2000 Support Tools > Tools > ADSI Edit

The user can set up the server connection by selecting:

Action > Settings

Additionally, select the Advanced button on the Connection window to input theadministrator ID and password to connect to the remote Active Directory server. If the connection is successful, the client machine will be able to communicate withthe Active Directory server using ADSI.

Domino Server

Client/server configurationFor both the Notes client and the Domino server configuration, the user can referto either Domino 5 Administration Help or Notes 5 Help documentation forinstructions and information.

Version numberServer

To find out the Domino server version number, issue the following command onthe Domino server console (or the command prompt on the server startingwindow)

show server

The list of Domino server information, including server version number, isdisplayed on the Domino server console.

Client

Refer to the IBM Tivoli Access Manager Base Installation Guide to install a LotusNotes client on the client machine and configure it to the Domino server used forTivoli Access Manager configuration. Upon completion of the Notes clientconfiguration, the Notes GUI windows is displayed. Display the Notes clientversion number information from the Notes client information window by

selecting the following GUI menus:Help > About Notes

Confirm client-server connection through the Notes APIsOnce both the Notes client and Domino server have been configured successfully,the Notes client should be able to open any database in Domino server.

After starting the Notes client GUI, select the following menus to open anydatabase in the Domino server:

File > Database > Open

Chapter 8. Basic troubleshooting 73

Page 90: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 90/116

From the pop-up Open Database window, you can select the correct Dominoserver and double-click on the database that you want to open. If the database inthe selected Domino server opens, the Notes client is able to communicate withDomino server using Notes APIs.

Tivoli Access Manager servers

At a high level, you can determine which Tivoli Access Manager servers have beenconfigured and which are running.

On UNIX platforms, use the following command:

# pd_start status

On the Windows platforms, this can be checked by examining the Tivoli AccessManager entries that appear in the Services window. On the Windows NT platformthe Services icon can be found on the Control Panel. On the Windows 2000platform, the Services icon can be found under Control Panel/AdministrativeTools.

Tivoli Access Manager policy serverThe pdadmin command can be used to verify the proper operation of the policyserver. Enter the following command at the command line:

pdadmin –a sec_master –p <sec_master-password>

At the pdadmin prompt, enter the following commands (example output shown):

pdadmin> server listwebseald-< machinename>(if WebSEAL is configured on the machine)

pdadmin> object list/Management/WebSEAL(if WebSEAL is configured on the machine)

pdadmin> acl listdefault-websealdefault-rootdefault-gsodefault-policydefault-configdefault-managementdefault-replica

pdadmin> user list * 25sec_masterivmgrd/masterwebseald/< machinename>

Tivoli Access Manager runtimeThe Tivoli Access Manager runtime may be installed on a runtime system withonly GSKit and the IBM SecureWay LDAP client. In this case, the verificationprocedure for the Tivoli Access Manager runtime is the same as that described forthe policy server in the section above.

Tivoli Access Manager WebSEALYou can use a browser to verify that WebSEAL is operating properly. Enter thefollowing URL into your browser:

74 IBM Tivoli Access Manager: Problem Determination Guide

Page 91: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 91/116

https://<webseal-machinename>

Since a port number is not specified, it is assumed WebSEAL is listening on port443 (HTTPS).

Your browser may give you the following warnings:

1. The certificate received from this Web server was issued by a company that

you have not yet chosen to trust

2. The name within the certificate received from WebSEAL does not match thename of the system from which it was received

If these warnings occur, they simply indicate that you have not yet purchased yourown server certificate for your WebSEAL server. Your browser is complaining thatit has received a default server certificate from WebSEAL which contains defaultnames for the issuing certificate authority and the name of the Web server.

Next, the browser prompts you to specify a Tivoli Access Manager user name andpassword. Enter sec_master for the user name and the password that youconfigured for sec_master during installation. If authentication is successful, an

image labeled Tivoli Access Manager for WebSEAL appears.

Junctioned third-party Web serverVerifying the correct operation of a junctioned third-party Web application server issimilar to that for WebSEAL itself (as described in the section above). Enter thefollowing URL into your browser to verify that the third-party Web server isfunctioning properly:

http://< junctioned-webserver-machinename>

Since a port number is not specified, it is assumed the server is listening on port 80(HTTP). If successful, the index.html page of the third-party Web server should

appear.

The WebSEAL junction for the web server is created using a pdadmin commandsimilar to the one in the following example (entered as one line). In this example,the junction point for the third-party Web server within the WebSEAL file space isnamed /myjunction:

pdadmin> server task webseald-<webseal-machinename> create -t tcp \-p < junctioned-server-port> -h < junctioned-webserver-machinename> \-c iv_user,iv_groups /myjunction

Try accessing the index.html page on the junctioned Web server, by accessing itthrough WebSEAL using the following URL:

https://<webseal-machinename>/myjunction

Your browser may again issue warnings about the WebSEAL server certificate thatwas received and prompts you again for a user name and password. Entersec_master for the user name and the appropriate password. If successful, theindex.html page of the third-party Web server appears.

Tivoli Access Manager Plug-in for Web ServersYou can use your browser to verify that Tivoli Access Manager Plug-in for WebServers is operating properly.

Chapter 8. Basic troubleshooting 75

Page 92: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 92/116

Enter the following URL into your browser:

http://<websvrplugin-machinename>

Since a port number is not specified, it is assumed that the Plug-in for Web Serversis listening on port 80 (HTTP).

Next your browser prompts you to specify a Tivoli Access Manager user name andpassword. Enter sec_master for the user name and the password that youconfigured for sec_master during installation. If successful, the default Web serverpage appears.

76 IBM Tivoli Access Manager: Problem Determination Guide

Page 93: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 93/116

Chapter 9. Solutions to common problems

Before listing some of the common Tivoli Access Manager problems, it isworthwhile to mention that most common problems are the result of installation

and configuration problems such as:

v Failure to install all of the software required by the Tivoli Access Managerproduct. This required software can include:

– Operating system software

– Operating system patches

– Prerequisite software products

– Prerequisite software product patches

v Failure to install the correct level of any of the software above

v Failure to install all of the required software components for any given type of Tivoli Access Manager system

v Failure to install or configure any of the above items properly

v Failure to adhere to all hardware prerequisites

The information contained within the Tivoli Access Manager technicaldocumentation is your best defense against the occurrence of any problems.

Section topics:

v “Tivoli Access Manager Base common problems” on page 77

v “LDAP common problems” on page 81

v “Tivoli Access Manager WebSEAL common problems” on page 83

v “Tivoli Access Manager for WebSphere Application Server common problems”on page 83

v “Tivoli Access Manager for WebLogic Server common problems” on page 86

v “Migration common problems” on page 86

Tivoli Access Manager Base common problems

v “Unable to configure the policy server” on page 77

v “Unable to create new user” on page 78

v “Invalid account” on page 79

v “Insufficient disk space causes problems” on page 79

v “″Not found″ error during user/group create” on page 80

v “Unexpected permit/deny access to resources” on page 80

Unable to configure the policy server

Problem:I am unable to configure the policy server.

Solution:There are many possible causes for this problem. A few are listed below:

Suggestion 1:

© Copyright IBM Corp. 2002, 2003 77

Page 94: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 94/116

The policy server may be unable to communicate with the configured user registry.Examine the following possible causes:

Verify that the user registry has not been stopped.

If using LDAP for your user registry, verify that the LDAP client on your machinecan still communicate with the LDAP server. Issue an LDAP command such as the

following (entered as one line) to learn whether the LDAP server is responsive:ldapsearch -h <ldapserver-hostname> -p 389 -D "<ldapadmin-DN>" \-w <ldapadmin-password> -b "" -s base objectclass=*

Keep in mind that the output of this command can vary depending upon whichsupported LDAP server you are using. The command above assumes that theLDAP server has been configured to listen on port 389. Also verify that the userregistry is still configured to communicate over the same port that was specified toTivoli Access Manager during the configuration of the Tivoli Access Managerruntime.

Suggestion 2:

If you are configuring Tivoli Access Manager to use the IBM Secureway LDAPserver version 4.1 as its user registry, Tivoli Access Manager may be having troublecreating the LDAP object associated with its required secAuthority=default LDAPsuffix.

This release of Access Manager includes a patch file named apply_ldap41_patchwhich must be run to correct this problem. Refer to the IBM Tivoli Access Manager

 for e-business Release Notes where this patch file is described.

Suggestion 3:

If you select LDAP for your user registry when configuring theTivoli Access

Manager runtime component, failure to specify the DN of an instantiated LDAPobject for your LDAP DN for the GSO database can result in a configurationfailure when later attempting to configure the Tivoli Access Manager policy server.

Suggestion 4:

If you are using Domino for the Tivoli Access Manager user registry and theconfiguration of your policy server fails, ensure that you have installed andconfigured the Lotus Notes client on your system and that the file nNOTES.dll can

 be found in your system path. If  nNOTES.dll cannot be found in the system path,explicitly add it to the system path.

Unable to create new user

Problem:I am having difficulty creating a new Tivoli Access Manager user.

Solution:One of the most common error messages seen when creating a new user is:

Could not perform the administration request.Error: Password rejected due to the Minimum Non-Alphabetic Characters policy(status 0x13212131)

78 IBM Tivoli Access Manager: Problem Determination Guide

Page 95: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 95/116

This error indicates, for example, that the password ″abc″ specified whenattempting to create the new user does not comply with one of the ″user″password policies currently defined to Tivoli Access Manager. To view the helptext associated with the Tivoli Access Manager policy commands, enter thefollowing command:

pdadmin> help policy

The password policy error above can be solved using one of the solutions below:

v Determine the minimum non-alphabetic character policy currently defined forTivoli Access Manager:

pdadmin> get min-password-non-alphas

Then create the user using a password that contains the required minimum of non-alpha characters:

v Modify the Tivoli Access Manager non-alphabetic character policy prior tocreating the user:

pdadmin> set min-password-non-alphas <number>

Invalid accountProblem:I am unable to authenticate with a Tivoli Access Manager user identity I haverecently created.

Solution:Tivoli Access Manager user definitions are initially created with the attribute:

Account valid = no

This condition is frequently the cause of authentication failures. Use the followingcommand to change the user’s Account valid attribute to ″yes″:

pdadmin> user modify <user-name> account-valid yes

Use the following command to verify that the change was made successfully:

pdadmin> user show <user-name>

Insufficient disk space causes problems

Problem:Insufficient disk space can cause Tivoli Access Manager problems with symptomsare difficult to predict.

Solution:Ensure that adequate disk space exists on whatever drives (Windows) or filesystems (UNIX) you used to install Tivoli Access Manager.

On the Windows platforms, concerns for adequate disk space includes the driveswhich contain the following directories:

v The Tivoli Access Manager installation directory

v The WebSEAL installation directory

v The Tivoli Access Manager Plug-in for Web Servers installation directory

v The Tivoli Access Manager Plug-in for Edge Server installation directory

Chapter 9. Solutions to common problems 79

Page 96: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 96/116

On UNIX platforms, concerns for adequate disk space includes the file systemsthat contain the /opt and the /var directories. Use the df –k command to displaythe free disk space for each file system. The –k option causes the disk space to bedisplayed in kilobytes.

″Not found″ error during user/group create

Problem:When creating a user or a group I sometimes receive the following genericmessage which does not communicate the nature of the error:

Error: Not found. (status 0x14c012f2)

Solution:When creating a user or a group, this message tells you that the distinguishedname (DN), which was specified in the user create or group create command,cannot be created within your LDAP user registry. Verify you have typed thecorrect DN.

You can use the dmt tool, shipped with the IBM Secureway LDAP client, to assistin verifying that the suffix of the DN, specified in the create command, matches an

existing suffix in the LDAP registry.

Unexpected permit/deny access to resources

Problem:Accesses to a protected system resource are either being unexpectedly granted ordenied.

Solution:It is always wise to first validate that the Tivoli Access Manager processes arestarted and running normally. Also check to ensure that the Tivoli Access Managermessage log files do not flag any operational problems. If Tivoli Access Managerseems operationally sound, the problem is likely due to the policies that have beendefined and applied to that system resource.

There are two Tivoli Access Manager policy mechanisms that can be used tocontrol access to your protected system resources, ACLs and POPs. Use thepdadmin command to learn which ACL within your protected object spacehierarchy controls access to the protected resource in question.

If an ACL is directly attached to the protected object in question, then it defines theACL policy for that object. If an ACL is not directly attached to the protected objectin question, then the controlling ACL is the nearest one above in the protectedobject hierarchy.

The following command lists all ACLs which are defined in Tivoli Access Manager:padmin> acl list

The following command enables you to learn where each of those ACLs isattached within the protected object space hierarchy.

pdadmin> acl find <acl-name>

Examine the controlling ACL using the following command to ensure it is correctfor the type of enforcement desired. Correct the ACL definition if needed:

pdadmin> acl show <acl-name>

80 IBM Tivoli Access Manager: Problem Determination Guide

Page 97: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 97/116

Tivoli Access Manager access control depends on two conditions:

v The ACL that controls the requested object must contain appropriate accesspermissions for the requesting user.

v The requested object must be accessible to the requesting user.

Accessibility to protected objects is controlled by the traverse (T) permission. Thetraverse permission is only applied to container objects in the protected object

space. The traverse permission specifies that a user, group, any-other, orunauthenticated user, identified in the ACL entry, has permission to passthrough this container object in order to gain access to a protected resourceobject below in the hierarchy.

A protected object is accessible to a requester if the requester possesses the traversepermission on each ACL attached to container objects above the requested resourceon the path towards root and including root.

Additionally, use the pdadmin command to learn which POP (if any) within yourprotected object space hierarchy controls access to the protected resource inquestion.

If a POP is directly attached to the protected object in question, then it defines thePOP policy for that object. If a POP is not directly attached to the protected objectin question, then the controlling POP is the nearest one above in the protectedobject hierarchy.

The following command lists all of the POPs which are defined for Tivoli AccessManager:

padmin> pop list

The following command enables you to learn where a particular POP is attachedwithin the protected object space hierarchy:

pdadmin> pop find < pop-name>

Examine the controlling POP using the following command to ensure it is correctfor the type of enforcement desired. Correct the POP definition if needed.

pdadmin> pop show < pop-name>

LDAP common problems

v “LDAP does not start after secAuthority=Default suffix is created” on page 81

v “Insufficient LDAP access privileges to perform operation” on page 82

v “IBM Directory Server error log warnings” on page 82

LDAP does not start after secAuthority=Default suffix is

createdProblem:IBM Directory Server does not start after secAuthority=Default suffix is created.

Solution:Tivoli Access Manager maintains some of its internal data in the LDAP serverunder the secAuthority=Default suffix. This suffix is automatically created if youchoose the easy installation method as described in the IBM Tivoli Access ManagerBase Installation Guide.

Chapter 9. Solutions to common problems 81

Page 98: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 98/116

If you do not use easy installation and choose the native installation method toinstall the IBM Directory Server, the apply_ldap41_patch command must beexecuted after the package has been installed and before the server is configured orstarted.

If you do not execute this command, the LDAP schema will not be updated todefine the secAuthority attribute type used to define the secAuthority=Default

suffix.

When the schema has not been updated, the server cannot start and will log anerror message in the slapd.errors file indicating that the secAuthority attribute isnot defined.

The slapd.errors file is located in the /tmp directory on UNIX platforms and inthe <ldap-install-dir>\tmp directory on Windows (where ldap-install-dir is thedirectory path in which IBM Directory Server was installed).

Insufficient LDAP access privileges to perform operation

Problem:I am experiencing an error which reads:

Insufficient LDAP access privileges to perform operation.

Solution:This message indicates that a supplied LDAP suffix does not have the correctACLs attached. Possible reasons for this are:

v During configuration, Tivoli Access Manager was unable to attach ACLs to theexisting suffix because the directory entries necessary to instantiate the suffixhad not been created

v The suffix was added after the initial configuration of the Tivoli Access Managermanagement server and the appropriate ACLs were not added manually as isrequired

The DMT tool shipped with the IBM Secureway Directory Client can be used tocheck the suffix and add the appropriate ACLs manually. For information on howto accomplish this, see the IBM Tivoli Access Manager Base Administrator’s Guide.

Fix the ACLs on the suffix and then retry the failing command.

IBM Directory Server error log warnings

Problem:IBM Directory Server error log indicates several ″does not exist″ warnings.

Solution:When the Tivoli Access Manager policy server is configured, the policy server isfirst unconfigured as part of this configuration process to ensure that it has beencleaned up completely.

The unconfiguration step attempts to remove entries in the LDAP registry.

When the policy server has not yet completed configuration, these entries have notyet been created and may not exist in the LDAP registry. The IBM Directory Serverlogs these entry removal attempts as warnings in its error log. These warnings aretherefore normal and can be safely ignored.

82 IBM Tivoli Access Manager: Problem Determination Guide

Page 99: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 99/116

Tivoli Access Manager WebSEAL common problems

v “WebSEAL does not respond on ports 80 or 443” on page 83

v “Multiple logins required during e-community” on page 83

WebSEAL does not respond on ports 80 or 443

Problem:WebSEAL does not respond on either port 80 or port 443.

Solution:Investigate whether you have another Web server installed on the WebSEALsystem. For example, the IBM HTTP Server is commonly installed as part of theinstallation of an IBM Secureway LDAP server. If another Web server is installedon the same system as WebSEAL, reconfigure it to listen upon ports other thanthose configured for use by WebSEAL.

Multiple logins required during e-community

Problem:WebSEAL e-community users are prompted to login more than once.

Solution:This problem can occur if two WebSEAL servers are configured in the samedomain—one configured as the Master Authentication Server (MAS) and the otherconfigured to use the MAS server for authentication. Attempts to access the lattermay require the user to login more than once if the difference in system times

 between the two WebSEAL servers is too great. Ensure that the system time oneach WebSEAL server participating in an e-community is synchronized.

Tivoli Access Manager for WebSphere Application Server common

problems

v “WebSphere administrative server does not start” on page 83

v “WebSphere server does not start after configuration” on page 84

v “WebSphere server does not start after unconfiguration” on page 84

v “Migration fails on Windows files with short name” on page 85

v “Client authentication lost due to session expiration” on page 85

v “Unable to access WebSphere Application Server sample applications” on page85

WebSphere administrative server does not start

Problem:After configuring Tivoli Access Manager for WebSphere, the WebSphereadministrative server will not start, and throws a null exception.

Explanation:The WebSphere administrative server requires that Tivoli Access Manager forWebSphere be configured to enable static role caching. When static role caching isdisabled, the WebSphere administrative server will not start.

Solution:Enable static role caching. See the IBM Tivoli Access Managerfor WebSphere

 Application Server User’s Guide for more information.

Chapter 9. Solutions to common problems 83

Page 100: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 100/116

WebSphere server does not start after configuration

Problem:After configuring Tivoli Access Manager for WebSphere, the WebSphereApplication Server will not start.

Explanation:

There are two possible reasons:v During Tivoli Access Manager for WebSphere configuration, the new

administrative group pdwas-admin was not added to the appropriate ACLs.

v During Tivoli Access Manager for WebSphere configuration, the newadministrative group pdwas-admin was added to the appropriate ACLs, but theACLs were not updated to all authorization servers. This problem can onlyoccur in secure domains that have more than one authorization server

Solution:There are two possible solutions:

v If pdwas-admin was not added to the appropriate ACLs, add it now.

v If pdwas-admin was added to the appropriate ACLs, and the secure domain

contains more than one authorization server, and the authorization servers werenot updated, update them now.

See the IBM Tivoli Access Managerfor WebSphere Application Server User’s Guide formore information.

WebSphere server does not start after unconfiguration

Problem:After unconfiguring Tivoli Access Manager for WebSphere and the Tivoli AccessManager Java runtime, the WebSphere Application Server might not start. Thisproblem occurs very intermittently. WebSphere Application Server fails to load thesecurity collaborator, com.ibm.ejs.security.EJSSecurityCollaborator.

Workaround:Disable WebSphere Application Server security and restart WebSphere ApplicationServer.

1. Go to the system running DB2. Log on as the user name with which DB2 wasinstalled. For example:

# su - db2inst1

A usage message is displayed

2. Enter the following commands as shown in bold font, where was40 is the nameof the host system:

db2 => connect to was40 user db2inst1

Enter current password for db2inst1:

Database Connection InformationDatabase Server = DB2/LINUX 7.2.0SQL authorization ID = DB2INST1Local database alias = WAS40

db2 => update ejsadmin.securitycfg_table set securityenabled = 0DB20000I The SQL command completed successfully

db2 => commitDB20000I The SQL command completed successfully

3. Start WebSphere Application Server.

84 IBM Tivoli Access Manager: Problem Determination Guide

Page 101: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 101/116

Migration fails on Windows files with short name

Problem:The migration utility does not work on filenames containing a tilde (~). This cancause problems when attempting to migrate a Windows short file name.

Solution:

Rename the filenames to omit the tilde (~)

Client authentication lost due to session expiration

Problem:Tivoli Access Manager provides a default SSL timeout value for connections to theTivoli Access Manager policy server. When this timeout value is exceeded duringexecution of the migration utility, you might see the following message:

The server lost the client’s authentication,probably because of session expiration.

Solution:When this message occurs, run the migration utility again using the -t minutes

option. The migration utility uses a default of 60 minutes. This value should be nogreater than the current SSL timeout between the authorization API client and thepolicy server.

You can determine the SSL timeout value by examining the parameterssl-v3-timeout, located under the [ssl] stanza in the Tivoli Access Managerconfiguration file ivmgrd.conf. The default value for ssl-v3-timeout is 7200seconds (120 minutes). When this default value is set, ensure that the SSL timeoutset by the migration utility -t flag is at least 60 minutes.

For more information, see the IBM Tivoli Access Manager Base Adminstrator’s Guide.

Unable to access WebSphere Application Server sampleapplications

Problem:Sample applications provided by WebSphere Application Server cannot be accessedafter Tivoli Access Manager for WebSphere security is enabled.

Explanation:WebSphere Application Server automatically installs and configures a defaultserver and sample applications. During the automatic installation, the name of theapplication, such as Sample Application, is internally changed to$hostName$_sampleApp. When the Tivoli Access Manager for WebSphere usermigrates the sample application EAR file, such as sampleApp.ear, a Tivoli Access

Manager object space entry is created for Sample Application. However,WebSphere Application Server uses $hostName$_sampleApp to display theapplication on the console, and to resolve authorization requests.

Workaround:Complete the following steps:

1. Modify the application.xml file in the sampleApp.ear file to change the displayname to match the name that is presented on the WebSphere ApplicationServer administration console. To do this, use WebSphere Application Serverapplication assembly tool to change the application name and then save theEAR file.

Chapter 9. Solutions to common problems 85

Page 102: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 102/116

For example, find the tag:

<DisplayName>Sample Applicaton</DisplayName>

and change it to:

<DisplayName>$hostName$_sampleApp</DisplayName>

Note: Substitute the name of the host system for the variable $hostname$. Forexample, if the system host name is diamond, the DisplayName of theapplication should be diamond_sampleApp.

2. Run the migration utility on the modified EAR file.

Tivoli Access Manager for WebLogic Server common problems

v “Authorization service won’t start on HP-UX” on page 86

v “WebLogic Server throws memory exception” on page 86

Authorization service won’t start on HP-UXOn HP-UX 11i only, if the Tivoli Access Manager authorization service does not

start, check that theSHLIB_PATH

environment variable is set correctly. This variablemust include the location of the IBM Global Security Toolkit (GSKIT) libraries. Thelocation can be either /usr/lib or /opt/ibm/gsk5/lib.

If  SHLIB_PATH is not set, enter:

export SHLIB_PATH=/opt/ibm/gsk5/lib

If  SHLIB_PATH is set but does not contain either of the two directories, add one of the directories to the path.

WebLogic Server throws memory exception

Problem:

A java.lang.OutofMemory exception is thrown.

Explanation:When running a large number of Access Manager for WebLogic Server sessions,BEA WebLogic Server may run out of heap space.

Solution:Increase the maximum heap size option for the Java Virtual Machine (JVM) in thestartWebLogic script. For example:

%JAVA_HOME%\bin\java -ms64m -mx128m

Consult the BEA product documentation for recommended heap size, based onapplication architecture and the number of memory-intensive processes running on

the host system. Applications should be stress-tested to determine the appropriateheap size for their environment. See the following URL for performance tuningconsiderations for thread counts and heap size:

http://edocs.bea.com/wls/docs61/perform/index.html

Migration common problems

v “Cannot create users and groups after migration” on page 87

86 IBM Tivoli Access Manager: Problem Determination Guide

Page 103: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 103/116

Cannot create users and groups after migration

Problem:Tivoli Access Manager does not have authority to create users/groups after IBMDirectory migration.

Solution:

When the IBM Directory Server component is chosen as the user registry and youare migrating from a previous version of Tivoli Access Manager, the IBM DirectoryServer component must be migrated first.

The migration of the IBM Directory Server component is performed by followingthe instructions in the Migration section of the IBM Directory Server Version 4.1Installation Guide for Multiplatforms.

These instructions guide you through the process of backing up the current datawith the db2ldif utility, upgrading the IBM Directory Server component, andrestoring the data with the bulkload utility.

When you use the bulkload utility, you should specify the –A yes option to have it

properly process Access Control List (ACL) updates. If the ACLs are not loadedproperly, Tivoli Access Manager will not have the authority to perform the neededtasks to create and maintain user and group information.

If bulkload fails to update the ACLs properly and these symptoms occur, you cancreate the ACLs manually by following the ″Applying Access Manager ACLs tonew LDAP suffixes″ procedure in the ″Using the LDAP registry″ chapter of theIBM Tivoli Access ManagerBase Administrator’s Guide. Apply the ACLs to all existingLDAP suffixes and secAuthority=Default entries below all defined users in theLDAP server using the Directory Management Tool. This will restore the properauthority to allow Tivoli Access Manager to continue.

Chapter 9. Solutions to common problems 87

Page 104: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 104/116

88 IBM Tivoli Access Manager: Problem Determination Guide

Page 105: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 105/116

Chapter 10. Contacting customer support

If you encounter a problem with the Tivoli Access Manager for e-business productthat cannot be resolved using the problem determination information provided in

this guide, contact Customer Support. The proper method for contacting CustomerSupport is described in the Customer Support Handbook , which is located at thefollowing Web site:

http://www.tivoli.com/support/handbook/index.html

The handbook provides information about how to contact Customer Support,depending upon the severity of your problem, and provides the followinginformation:

v Registration and eligibility

v Telephone numbers and e-mail addresses, depending upon the country in whichyou are located

v The types of information you should gather before contacting support

Throughout this guide, you are directed to collect certain kinds of information toassist Customer Support in finding a solution to your problem. Collect thisinformation, but do not send it to Customer Support until you are directed to doso. The procedures and methods for transmitting such data undergo periodicchange; it is best to get the latest instructions from Customer Support.

General Data to Collect for Customer Support

The following is a list of the types of data that should be collected and madeavailable to Customer Support should they request it:

v

Brief description of the class of problem, such as install, configuration, audit,system failure, or performance

v Hardware configuration (machine make and model number), operating systemtype, version number, patch levels

v Language/locale information

v Tivoli Access Manager level, including any fixpacks and efixes

v Tivoli Access Manager and LDAP server—the systems on which they arelocated, network connectivity to these systems, version number (includingfixpacks) of the servers

v Are any of the systems (Tivoli Access Manager server, LDAP server) configuredwith multiple IP addresses?

v Detailed description of the problem, whether the problem can be recreated, andif so, the steps required to recreate the problem

v Output from the pdversion utility

v Output from the pdinfo utility

v Collect any trace logs which have been generated

Typically trace logs will be generated only at the request of IBM customersupport personnel.

v Timeframe in which the problem occurred (to relate it back to the log entries)

© Copyright IBM Corp. 2002, 2003 89

Page 106: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 106/116

Collecting defect information for pdadmin and C APIs

This section describes how to collect problem determination information whenreporting a defect on the Tivoli Access Manager administration C andauthorization C APIs and the pdadmin command line interface.

Basic procedurev Turn on tracing on the machine that is running the affected Tivoli Access

Manager server (could be pdmgrd and/or pdacld depending on the API beingused).

v Restart pdmgrd and/or pdacld.

v Turn on tracing on the machine where you are executing pdadmin or theapplication/test case that invokes the C APIs (administration C API orauthorization C API).

v Restart pdadmin or the application/test case that invokes the C APIs.

v Execute the operations that failed.

v Follow steps listed below to collect the trace files:

Note: One way to turn tracing on is to edit the < base-install-dir>\etc\routingfile and uncomment the last entry in this file. You can also turn tracing ondynamically for authorization API applications, but this approach will notwork for pdmgrd or for applications that use only the administration C APIand not the authorization C API.

Getting the pdmgrd trace file# cd < base-install-dir>\log# cat ivmgrd.pid

(Make note of the pdmgrd process ID from the output of this command)

Collect the < pdmgrd-process-id>.trace.log.* files

Getting the pdacld trace file# cd < base-install-dir>\log# cat ivacld.pid

(Make note of the pdacld process ID from the output of this command)

Collect the < pdacld-process-id>.trace.log.* files

Getting the pdadmin and C API trace fileNote down the process ID of the pdadmin process or the process ID of the

application/test case that invokes the C API.# cd < base-install-dir>\log

Collect the < process-id>.trace.log.* files

Getting the message filesCollect the following files from the log directory on all the machines that runTivoli Access Manager servers and applications:

90 IBM Tivoli Access Manager: Problem Determination Guide

Page 107: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 107/116

msg__notice*logmsg__warning*logmsg__error*logmsg__fatal*log

Collecting defect information for Java APIs

This section describes how to collect problem determination information whenreporting a defect on the Tivoli Access Manager administration Java andauthorization Java APIs.

Basic procedurev Turn on tracing on the machine that is running the affected Tivoli Access

Manager server (could be pdmgrd and/or pdacld depending on the API beingused).

v Restart pdmgrd and/or pdacld.

v Turn on tracing on the machine where you are executing the application/testcase that invokes the Java APIs (administration Java API or authorization JavaAPI).

v Restart the application/test case that invokes the Java APIs.v Execute the operations that failed.

v Follow steps listed below to collect the trace files:

Note: One way to turn tracing on is to edit the < base-install-dir>\etc\routingfile and uncomment the last entry in this file. You can also turn tracing ondynamically for authorization API applications, but this approach will notwork for pdmgrd or for applications that use only the administration JavaAPI and not the authorization Java API.

Enabling tracing and messaging for Java runtime componentsThe < java_home>\PolicyDirector\PDJLog.properties file contains theconfiguration of loggers, filters, handlers, and formatters for the various TivoliAccess Manager Java components. Edit this file as shown below to enable tracingand messaging for all the Tivoli Access Manager Java runtime components:

Set the isLogging value to ″true″ for the PDJTraceLogger by updating thefollowing line:

baseGroup.PDJTraceLogger.isLogging = true

By default, all the trace and message log files are located in the log directory, andare named as follows:

Table 41. Java runtime log files 

Windows < base-install-dir>\log\msg__amj_fatal.log< base-install-dir>\log\msg__amj_error.log< base-install-dir>\log\msg__amj_warning.log< base-install-dir>\log\msg__amj_notice.log< base-install-dir>\log\msg__amj_noticeverbose.log

UNIX /var/PolicyDirector/log/msg__amj_fatal.log/var/PolicyDirector/log/msg__amj_error.log/var/PolicyDirector/log/msg__amj_warning.log/var/PolicyDirector/log/msg__amj_notice.log/var/PolicyDirector/log/msg__amj_noticeverbose.log

Chapter 10. Contacting customer support 91

Page 108: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 108/116

Getting the pdmgrd trace file# cd < base-install-dir>\log# cat ivmgrd.pid

(Make note of the pdmgrd process ID from the output of this command)

Collect the < pdmgrd-process-id>.trace.log.* files

Getting the pdacld trace file# cd < base-install-dir>\log# cat ivacld.pid

(Make note of the pdacld process ID from the output of this command)

Collect the < pdacld-process-id>.trace.log.* files

92 IBM Tivoli Access Manager: Problem Determination Guide

Page 109: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 109/116

Appendix. Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document inother countries. Consult your local IBM representative for information on theproducts and services currently available in your area. Any reference to an IBMproduct, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product,program, or service that does not infringe any IBM intellectual property right may

 be used instead. However, it is the user’s responsibility to evaluate and verify theoperation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matterdescribed in this document. The furnishing of this document does not give youany license to these patents. You can send license inquiries, in writing, to:

IBM Director of LicensingIBM Corporation500 Columbus AvenueThornwood, NY 10594U.S.A

For license inquiries regarding double-byte (DBCS) information, contact the IBMIntellectual Property Department in your country or send inquiries, in writing, to:

IBM World Trade Asia CorporationLicensing2-31 Roppongi 3-chome, Minato-ku

Tokyo 106, Japan

The following paragraph does not apply to the United Kingdom or any othercountry where such provisions are inconsistent with local law: INTERNATIONALBUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION ″AS IS″WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFNON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULARPURPOSE. Some states do not allow disclaimer of express or implied warranties incertain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors.Changes are periodically made to the information herein; these changes will be

incorporated in new editions of the publication. IBM may make improvementsand/or changes in the product(s) and/or the program(s) described in thispublication at any time without notice.

Any references in this information to non-IBM Web sites are provided forconvenience only and do not in any manner serve as an endorsement of those Websites. The materials at those Web sites are not part of the materials for this IBMproduct and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

© Copyright IBM Corp. 2002, 2003 93

Page 110: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 110/116

Licensees of this program who wish to have information about it for the purposeof enabling: (i) the exchange of information between independently createdprograms and other programs (including this one) and (ii) the mutual use of theinformation which has been exchanged, should contact:

IBM Corporation2Z4A/101

11400 Burnet RoadAustin, TX 78758USA

Such information may be available, subject to appropriate terms and conditions,including in some cases, payment of a fee.

The licensed program described in this document and all licensed materialavailable for it are provided by IBM under terms of the IBM Customer Agreement,IBM International Program License Agreement or any equivalent agreement

 between us.

Any performance data contained herein was determined in a controlled

environment. Therefore, the results obtained in other operating environments mayvary significantly. Some measurements may have been made on development-levelsystems and there is no guarantee that these measurements will be the same ongenerally available systems. Furthermore, some measurement may have beenestimated through extrapolation. Actual results may vary. Users of this documentshould verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources.IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.Questions on the capabilities of non-IBM products should be addressed to the

suppliers of those products.

All statements regarding IBM’s future direction or intent are subject to change orwithdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily businessoperations. To illustrate them as completely as possible, the examples include thenames of individuals, companies, brands, and products. All of these names arefictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, whichillustrates programming techniques on various operating platforms. You may copy,modify, and distribute these sample programs in any form without payment toIBM, for the purposes of developing, using, marketing or distributing applicationprograms conforming to the application programming interface for the operatingplatform for which the sample programs are written. These examples have not

 been thoroughly tested under all conditions. IBM, therefore, cannot guarantee orimply reliability, serviceability, or function of these programs. You may copy,modify, and distribute these sample programs in any form without payment toIBM for the purposes of developing, using, marketing, or distributing applicationprograms conforming to IBM’s application programming interfaces.

94 IBM Tivoli Access Manager: Problem Determination Guide

Page 111: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 111/116

Each copy or any portion of these sample programs or any derivative work, mustinclude a copyright notice as follows:

© (your company name) (year). Portions of this code are derived from IBM Corp.Sample Programs. © Copyright IBM Corp. _enter the year or years_. All rightsreserved.

Trademarks

The following terms are trademarks or registered trademarks of InternationalBusiness Machines Corporation in the United States, other countries, or both:

AIXDB2IBMIBM logoSecureWayTivoliTivoli logo

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

 Java and all Java-based trademarks and logos are trademarks or registeredtrademarks of Sun Microsystems, Inc. in the United States and other countries.

UNIX is a registered trademark of The Open Group in the United States and othercountries.

Other company, product, and service names may be trademarks or service marksof others.

Appendix. Notices 95

Page 112: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 112/116

96 IBM Tivoli Access Manager: Problem Determination Guide

Page 113: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 113/116

Index

Aagent.log 45

event logging format 49example 48

agents 45agents-file 45API (C)

collecting defect data 90AutoTrace

AutoTrace file descriptions 23 building 22config file 23config file description 25create trace ID file 34exclude file description 27initializing atctl.(exe) 27initializing atserv.exe 27

output format and example 30overview 21product file 23product file description 25useful commands 31using 29

aznapi-configuration stanza 48, 67

Ccommon log format (request.log) 47customer support

collecting data 89collecting defect data for pdadmin and C APIs 90contacting 89

Ddiagnostic tools

pdacld_dump 6pdinfo 3pdversion 3

Eerror messages

serviceability 43event logging

HTTP logging 48

statistics 67event poolhttp 48http.agent 48http.clf 48http.cof 48http.ref 48

Fflush-time 47

Ggmt-time 46

HHTTP common log format 47http event pool 48HTTP logging

default 45using event logging 48

http.agent event pool 48http.clf event pool 48http.cof event pool (NCSA) 48http.ref event pool 48

Llogcfg 48, 67logging

default HTTP 45HTTP (event logging) 48

Mmax-size 46message logging

Base serviceability messages 40component log files 38configuration log files 37default HTTP logging 45installation log files 37

 Java Runtime components 51log entry format 42overview 37runtime log files 39using event logging mechanism 48WebSEAL serviceability messages 43

messages 43msg__error.log 41, 44msg__fatal.log 41, 44msg__notice.log 41, 44msg__warning.log 41, 44routing file 43

msg__error.log 41, 44msg__fatal.log 41, 44msg__notice.log 41, 44

msg__warning.log 41, 44

NNCSA combined format 48

Ppdacld_dump 6

database summary report 8syntax 7

© Copyright IBM Corp. 2002, 2003 97

Page 114: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 114/116

pdadmincollecting defect data 90

pdadmin trace 11pdinfo 3

contents of tar file 4pdinfo.cfg configuration file 5running the utility 4

pdinfo.cfg 4, 5

pdversion 3pdweb.authn statistics 62pdweb.authz statistics 62pdweb.doccache statistics 65pdweb.http statistics 63pdweb.https statistics 63pdweb.jct.# statistics 67pdweb.jmt statistics 64pdweb.sescache statistics 64pdweb.threads statistics 64problem determination

overview 1problem prevention 1

Rreferer.log 45event logging format 49example 48

referers 45referers-file 45related publications xrequest.log 45

event logging format 49example 47

requests 45requests-file 45routing file 43

Sserviceability messages 43msg__error.log 41, 44msg__fatal.log 41, 44msg__notice.log 41, 44msg__warning.log 41, 44routing file 43

statisticsactivity types 62components 62disable (stats off) 61display (stats get) 61enable (stats on) 59enable using event logging 67list (stats list) 62

logcfg parameter 67pdweb.authn 62pdweb.authz 62pdweb.doccache 65pdweb.http 63pdweb.https 63pdweb.jct.# 67pdweb.jmt 64pdweb.sescache 64pdweb.threads 64reset (stats reset) 61show (stats show) 61stats command syntax 59

statistics (continued)stats commands 59stats parameter 67

statistics (WebSEAL) 59stats 67stats, commands 59

Ttracecapturing actions 11controlling trace with routing files 16

 Java Runtime components 51syntax 12trace entry format 12trace log file 15

trace, pdadmin trace 11troubleshooting

Active Directory 72Domino Server 73GSKit 71IBM SecureWay Directory client 72

 junctioned servers 75

LDAP server 71Tivoli Access Manager Plug-in for Web Servers 75Tivoli Access Manager policy server 74Tivoli Access Manager runtime 74Tivoli Access Manager servers 74Tivoli Access Manager WebSEAL 74verifying component functionality 70verifying software installation 69

WWebSEAL

statistics 59

98 IBM Tivoli Access Manager: Problem Determination Guide

Page 115: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 115/116

Page 116: am41_pdg

7/28/2019 am41_pdg

http://slidepdf.com/reader/full/am41pdg 116/116

Printed in U.S.A.