IBM Tivoli Identity Manager Version 4.6 Performance Tuning...

IBM Tivoli Identity Manager Version 4.6 Performance Tuning Guide

Issue Date: 2007 June 1 – Fourth Edition

Publication Number: SC23-5272-03

Copyright Notice

Copyright IBM Corporation 2007. All rights reserved. May only be used pursuant to a Tivoli Systems Software License Agreement, an IBM Software License Agreement, or Addendum for Tivoli Products to IBM Customer or License Agreement. No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated into any computer language, in any form or by any means, electronic, mechanical, magnetic, optical, chemical, manual, or otherwise, without prior written permission of IBM Corporation. IBM Corporation grants you limited permission to make hardcopy or other reproductions of any machine-readable documentation for your own use, provided that each such reproduction shall carry the IBM Corporation copyright notice. No other rights under copyright are granted without prior written permission of IBM Corporation. The document is not intended for production and is furnished “as is” without warranty of any kind. All warranties on this document are hereby disclaimed, including the warranties of merchantability and fitness for a particular purpose.

U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corporation.

Trademarks

IBM, the IBM logo, Tivoli, the Tivoli logo, AIX, DB2, and WebSphere Application Server are trademarks or registered trademarks of International Business Machines Corporation or Tivoli Systems Inc. in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Other company, product, and service names may be trademarks or service marks of others.

Notices

References in this publication to Tivoli Systems or IBM products, programs, or services do not imply that they will be available in all countries in which Tivoli Systems or IBM operates. Any reference to these products, programs, or services is not intended to imply that only Tivoli Systems or IBM products, programs, or services can be used. Subject to valid intellectual property or other legally protectable right of Tivoli Systems or IBM, any functionally equivalent product, program, or service can be used instead of the referenced product, program, or service. The evaluation and verification of operation in conjunction with other products, except those expressly designated by Tivoli Systems or IBM, are the responsibility of the user. Tivoli Systems or IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, New York 10504-1785, U.S.A.

IBM Tivoli Identity Manager Performance Tuning Guide Page 1

Contents Contents ...................................................................................................................................................1 About this guide ........................................................................................................................................3

Who should use this guide ...................................................................................................................3 What’s new ...........................................................................................................................................3

Forth edition .....................................................................................................................................3 Third edition .....................................................................................................................................3 Second edition .................................................................................................................................4

1 Introduction...........................................................................................................................................5 1.1 Vital tunings .................................................................................................................................5 1.2 Initial tunings................................................................................................................................5 1.3 Resource allocation .....................................................................................................................5

1.3.1 Memory....................................................................................................................................6 1.3.2 CPU .........................................................................................................................................6 1.3.3 Disk space...............................................................................................................................7

2 IBM WebSphere Application Server.....................................................................................................8 2.1 Java virtual machine (JVM) size..................................................................................................8 2.2 Java virtual machine (JVM) options – Sun Solaris ......................................................................9 2.3 Decreasing individual transaction times ....................................................................................11 2.4 IBM HTTP Server ......................................................................................................................12

2.4.1 Connections...........................................................................................................................12 2.5 IBM WebSphere MQ..................................................................................................................12

3 IBM Tivoli Identity Manager application..............................................................................................14 3.1 Failed/Warnings links.................................................................................................................14 3.2 List control options.....................................................................................................................14 3.3 Recycle bin ................................................................................................................................14 3.4 Reconciliations...........................................................................................................................15

3.4.1 Limiting attributes returned from the adapter ........................................................................15 3.4.2 Limiting the attributes evaluated............................................................................................15 3.4.3 Maximum duration.................................................................................................................16 3.4.4 Paged searches ....................................................................................................................17 3.4.5 enrole.reconciliation properties .............................................................................................17

3.5 Server-side sorting.....................................................................................................................17 3.6 ACI cache ..................................................................................................................................17

4 IBM Tivoli Identity Manager adapters.................................................................................................19 4.1 Microsoft Active Directory..........................................................................................................19

5 IBM Tivoli Directory Integrator ............................................................................................................20 5.1 DSML connector and chunked encoding...................................................................................20

6 Database servers ...............................................................................................................................21 6.1 IBM DB2 ....................................................................................................................................21

6.1.1 Quick guide for setting the IBM DB2 tuning parameters.......................................................21 6.1.2 Buffer pools ...........................................................................................................................21 6.1.3 Connections...........................................................................................................................22 6.1.4 Table spaces .........................................................................................................................22

6.1.4.1 Adding additional table space containers .....................................................................22 6.1.4.2 Enabling autoresize ......................................................................................................23

6.1.5 Transaction logs ....................................................................................................................23 6.1.6 Database application heaps ..................................................................................................24 6.1.7 Indexing .................................................................................................................................24 6.1.8 Runstats ................................................................................................................................25 6.1.9 Maximum open files ..............................................................................................................26 6.1.10 Lock tuning ............................................................................................................................26

6.1.10.1 Lock type ..................................................................................................................26 6.1.10.2 Lock list and maximum locks....................................................................................26 6.1.10.3 Lock timeout .............................................................................................................27

Page 2 IBM Tivoli Identity Manager Performance Tuning Guide

6.2 Oracle Database........................................................................................................................27 6.2.1 Server configuration ..............................................................................................................27 6.2.2 Table spaces .........................................................................................................................27

6.2.2.1 Spreading database data..............................................................................................27 6.2.2.2 Adding additional table space datafiles ........................................................................28

6.2.3 Indexing .................................................................................................................................28 6.2.4 Updating statistics .................................................................................................................28

7 Directory servers ................................................................................................................................30 7.1 IBM Tivoli Directory Server........................................................................................................30

7.1.1 Quick guide for setting the IBM Tivoli Directory Server tuning parameters ..........................30 7.1.2 Cache sizes ...........................................................................................................................30 7.1.3 Paging parameters ................................................................................................................31 7.1.4 Buffer pools ...........................................................................................................................32 7.1.5 Connections...........................................................................................................................33 7.1.6 Transaction logs ....................................................................................................................33 7.1.7 Database statement heap .....................................................................................................34 7.1.8 System limits .........................................................................................................................34 7.1.9 Indexing .................................................................................................................................35

7.1.9.1 Attribute indexes ...........................................................................................................35 7.1.9.2 DB2 indexes .................................................................................................................36

7.1.10 Runstats ................................................................................................................................36 7.2 Sun ONE Directory Server ........................................................................................................37

7.2.1 All IDs Threshold value .........................................................................................................37 7.2.2 Indexing .................................................................................................................................38 7.2.3 Cache sizes ...........................................................................................................................38

7.2.3.3 Sun ONE Directory Server V5.1 ...................................................................................38 7.2.3.4 Sun ONE Directory Server V5.2 ...................................................................................39

8 Operating Systems .............................................................................................................................40 8.1 AIX .............................................................................................................................................40 8.2 Red Hat......................................................................................................................................40

9 Best practices .....................................................................................................................................41 10 Regular maintenance .........................................................................................................................43 11 Troubleshooting..................................................................................................................................44

11.1 Log files .....................................................................................................................................44 11.2 Identifying bottlenecks ...............................................................................................................44 11.3 Specific problems ......................................................................................................................45

11.3.1 IBM Tivoli Identity Manager application ................................................................................45 11.3.2 IBM WebSphere MQ .............................................................................................................46 11.3.3 IBM Tivoli Directory Server....................................................................................................46 11.3.4 Sun ONE Directory Server ....................................................................................................47

12 IBM DB2 performance monitoring ......................................................................................................49 12.1 Enable monitoring......................................................................................................................49 12.2 Snapshots..................................................................................................................................49 12.3 Statement monitor .....................................................................................................................49

12.3.1 Set up the monitor .................................................................................................................49 12.3.2 Use the monitor .....................................................................................................................49

12.4 Buffer pool hit ratio.....................................................................................................................50 13 Other resources..................................................................................................................................51 14 Appendix A – Scripts and files............................................................................................................52


About this guide This guide identifies some ways to tune your IBM® Tivoli® Identity Manager system to improve performance.

Who should use this guide Use this guide if you are responsible for installing or maintaining an IBM Tivoli Identity Manager system. The following competencies are recommended:

• Familiarity with basic database and directory design principles.

• General knowledge of system resource management (memory, disk, and processor) including your operating system’s memory model. Without knowledge of other resource requirements on the system it is possible to negatively impact performance by over-tuning.

• Understanding of how to configure and administer your directory and database servers. You can have your local database administrator or directory administrator perform these tunings for you.

What’s new Fourth edition

New: Three new indexes for the DB2® database under IBM Tivoli Directory Server New index for IBM Tivoli Directory Server database to improve viewing completed requests Best practice for user attributes Best practice for erGlobalID values when doing an initial data load Best practice for using a SAN Best practice for running IBM Tivoli Directory Server on an LPAR Best practice for the number of accounts per person Guidance on setting the list control parameters Information on disabling the Failed/Warning links on the View Completed Requests page Using paged searches within IBM Tivoli Identity Manager to enable larger reconciliations Paging parameter details for IBM Tivoli Directory Server Guidance on enrole.reconciliation parameters within enRole.properties Information on the ACI cache tuning added in IF26 New section on IBM Tivoli Directory Integrator tuning

Updated: Additional attributes to index for IBM Tivoli Identity Manager Removed file listings in Appendix A, referring to the Tuning Guide scripts online instead Clarified setting Solaris JVM parameters and where they apply Added information on moving the DB2 transaction log to a different drive

Third edition

New: Database statement heap tunings for the IBM Tivoli Directory Server Additional troubleshooting information related to the statement heap tuning Minimum cardinality tunings for the IBM Tivoli Identity Manager IBM DB2 database Information on the IBM DB2 autoresize option available for table spaces Information about the IBM Tivoli Identity Manager recycle bin Information on improving IBM Tivoli Identity Manager reconciliations Additional section on IBM Tivoli Identity Manager adapters Section on regular maintenance activities

Updated: Clarified information regarding Java virtual machine (JVM) tuning parameters when using


the Sun JVM Clarified cache sizings for Sun ONE Directory Server V5.2 IBM DB2 monitoring information Information on DB2_RR_TO_RS locking for IBM DB2 V8 Best practices section Oracle section with additional tunings and recommendations

Second edition

New: Information on MAXFILOP tuning Information on IBM DB2 application heap tuning IBM Tivoli Directory Server cache size tuning Oracle indexes and table space distributions


1 Introduction

The IBM Tivoli Identity Manager product addresses the complex problem of identity management. Due to the complexity of the problem, it can be challenging to optimize the use of resources by IBM Tivoli Identity Manager – that is, to tune it. This tuning guide provides your system administrator with the information needed to tune the application for your environment. Other individuals (such as IBM DB2 or IBM Tivoli Directory Server administrators) in your organization might offer differing advice. In our experience, your system administrators know your environment better, and their advice can be more accurate for your environment than this tuning document.

The IBM Tivoli Identity Manager product can be divided into four major components: IBM WebSphere Application Server, the IBM Tivoli Identity Manager application, database servers, and directory servers. We will address each of these separately in this document.

The IBM Tivoli Identity Manager server can be installed as either a single server or as clustered servers. A clustered environment can be considered a group of single servers with regard to tuning.

This document is a working document. As more information is gathered, settings will be added, removed or changed in future editions. Check the IBM Web site for the most recent version at http://www-1.ibm.com/support/docview.wss?uid=swg27006281

IBM Tivoli Identity Manager configurations vary significantly by platform, operating system, middleware applications, and hardware being used. You must tailor your configuration settings to your deployment. Configurations and settings used in this document include AIX®, Windows®, and Sun Solaris in single server and cluster configurations.

1.1 Vital tunings There are several thousand different parameters that you can modify to tune WebSphere® Application Server, the IBM Tivoli Identity Manager product, directory servers, and database servers. This tuning guide discusses a small subset of these parameters that have proven effective during performance testing.

If you are setting up an acceptance or production environment, read each section and perform the applicable tunings for your systems. If you are setting up a test environment and want to get started as quickly as possible, focus on these areas:

• IBM WebSphere Application Server - Java virtual machine (JVM) size

• IBM DB2 - Buffer pools

• IBM Tivoli Directory Server – Indexing or Sun ONE Directory Server - Indexing

• IBM DB2 - Runstats or Oracle Database - Updating statistics

The database statistics tunings are a vital part of the IBM Tivoli Identity Manager product performance.

1.2 Initial tunings Most of these tunings can be implemented in a newly deployed environment or an environment that is already deployed. When tuning IBM DB2 in a newly deployed environment, it is important to prime your database statistics using the IBM DB2 runstats command and the corresponding manual cardinality tunings. See the IBM DB2 - Runstats section for more information. Failing to prime the database in this manner can result in poor performance or transaction rollbacks.

1.3 Resource allocation Tuning values are more complex to manage when more than one middleware component is running on a given system; for example, having the IBM Tivoli Identity Manager server, IBM DB2, and IBM Tivoli

http://www-1.ibm.com/support/docview.wss?uid=swg27006281


Directory Server all on the same server. Regardless of configuration, it is important to calibrate the following resources so that they are not over-allocated.

1.3.1 Memory All middleware components allow you to adjust how much memory they will use. When calculating how to allocate memory to middleware components, keep these considerations in mind:

• Configuring middleware memory settings too high can result in the operating system swapping memory out to disk if the physical memory is exceeded. This will result in extremely poor performance and should be avoided. After setting up or changing the memory values for the middleware, monitor the memory and swap space used to ensure that nothing is being swapped out to disk. If it is, adjust your memory settings to correct.

• 32-bit processes can only allocate 2 GB (on AIX, Windows, and some Linux® kernels) to 4 GB (on Solaris) of RAM. If a 32-bit process is configured to allocate more than the OS-specific limit on process memory, the application (such as IBM Tivoli Directory Server) could halt or unexpectedly fail.

• IBM Tivoli Directory Server has internal caches that contribute to the size of the IBM Tivoli Directory Server process. Ensure that the size of the IBM Tivoli Directory Server process does not exceed 2 GB on 32-bit platforms such as Windows. When this 2 GB limit is reached, the IBM Tivoli Directory Server will refuse new connections and fail. The entry cache size limit determines the number of entries in the cache, not the size of the cache. The size of each cache entry will vary based on the IBM Tivoli Identity Manager product configuration and any extensions to the base IBM Tivoli Directory Server schema. In rare cases the default cache values might exceed the 2 GB limit.

• Although the buffer pools account for a large amount of the memory used by IBM DB2, the application control heaps, the sort heaps, and the statement heaps also use memory. Do not overlook these sizes when computing how much memory to allocate to IBM DB2. See the IBM DB2 section.

• A large part of the WebSphere Application Server’s memory usage is the JVM size. However, the size of the JVM does not set an upper bound on the amount of memory that the WebSphere Application Server will use. See the IBM WebSphere Application Server section.

• Operating system limits can prevent processes from accessing all available memory. Confirm the appropriate ulimit values for your system to ensure they do not artificially limit the amount of memory available.

1.3.2 CPU All the components of the IBM Tivoli Identity Manager product (IBM Tivoli Identity Manager application, WebSphere Application Server, database server, and directory server) are CPU-intensive. This document groups WebSphere Application Server and the IBM Tivoli Identity Manager application together because it is difficult to isolate their CPU usage. Keep in mind the following considerations:

• In IBM Tivoli Identity Manager 4.6 role evaluation, policy enforcement and reconciliation have been enhanced by distributing work across multiple server threads and multiple Tivoli Identity Manager cluster members to complete these activities much faster than in previous Tivoli Identity Manager versions.

• Both IBM Tivoli Directory Server and IBM DB2 are multi-threaded (multi-process in the case of DB2) applications and will perform best on a multi-processor server.

• Even in a well-tuned environment the system bottlenecks might vary between the CPU, memory and disk on the IBM Tivoli Identity Manager server, the directory server, and the database server. For this reason, the IBM Tivoli Identity Manager server, the directory server, and the database server can benefit by being deployed on separate servers. If that is not possible, put the database


and directory server on a server with a high performance disk configuration using multiple disks and a high performance RAID configuration to provide fast read and write capacity.

1.3.3 Disk space Each of the middleware components uses different amounts of disk space for various purposes.

• WebSphere Application Server and the IBM Tivoli Identity Manager application use disk space beyond their installation size because of log files (such as the msg.log and trace.log files) and WebSphere MQ queues. Adjust the number of archives and size of the msg.log and trace.log files in the enRoleLogging.properties file. Make sure that WebSphere MQ has enough disk space for its processing logs (not error logs) to grow. The IBM Tivoli Identity Manager server pushes many entries onto the queues during large provisioning changes, causing the queues to grow.

• IBM Tivoli Directory Server uses disk space both from the IBM Tivoli Directory Server process (log files like ibmslapd.log) and the IBM DB2 database. IBM Tivoli Directory Server uses system-managed space (SMS) table spaces that allow the system to manage the amount of disk space used. SMS table spaces do not allow the specification of an upper bound so it is important to monitor the amount of disk space used to ensure that the drive does not become full.

• In addition to the table spaces for the database data, IBM DB2 uses disk space for the transaction logs. When configuring the transaction logs (discussed in the IBM Tivoli Directory Server – Transaction logs and IBM DB2 – Transaction logs sections), ensure that you have enough disk space for the log files.

• The IBM Tivoli Identity Manager DB2 database uses directory-managed space (DMS) table spaces that require pre-allocating disk space for the database to use. If the table space becomes full, IBM DB2 will be unable to continue and will return an error in the msg.log and trace.log files. If this occurs, add more table space containers or enable autoresize (available in IBM DB2 V8.2.2 and later). See the IBM DB2 – Table spaces section for additional information.

• The default maximum sizes of the IBM Tivoli Identity Manager Oracle datafiles are 1 GB. If the datafile, becomes full Oracle returns an error and all workflow activities fail. If this occurs, add more datafiles or increase the maximum size of the existing datafiles. See the Oracle Database – Table spaces section for additional information.

• As a general rule, for every 160,000 transactions that occur, 1 GB of table space is needed to archive the historical transaction information. A policy change that makes modifications to 5,000 users is 5,000 transactions from the perspective of IBM Tivoli Identity Manager.


2 IBM WebSphere Application Server

Regardless of the installation type (single server or cluster), the IBM Tivoli Identity Manager server can be thought of as two components: WebSphere Application Server (the J2EE application server running the application) and the IBM Tivoli Identity Manager application itself. Both components need to be tuned.

WebSphere Application Server allows you to use a variety of settings to tune your environment. This document discusses the JVM size, the maximum HTTP connections and Java™ Messaging Service (JMS) queue size.

2.1 Java virtual machine (JVM) size By default, WebSphere Application Server sets the maximum JVM size to 256 MB. This value is too small for the IBM Tivoli Identity Manager product to run beyond a basic concept test and should be increased to a minimum of 1 GB. If your server has adequate available RAM, increase this value to as much as 1.5 GB. For large reconciliations or role and policy evaluations, the default values will not provide enough memory to complete these tasks.

Setting the maximum memory value too high can cause memory allocation problems in Java when the 2 GB limit is reached for 32-bit processes. The maximum JVM size used in this value is not the actual maximum allocated size of the Java heap – as much as 15% is allocated to a portion of the heap for the system’s use. Do not use a value higher than 1.5 GB even if your system has the available memory.

Do not set the JVM heap size to be larger than the physical RAM. The WebSphere Application Server suffers significant performance degradation if the operating system swaps out the JVM to swap space. Consequences of this include very slow user interface (UI) performance, transaction rollbacks, timeouts, and high disk utilization.

The WebSphere Application Server sets the initial JVM size to 0 MB, use a value of 256 MB instead. While previous JVM versions have worked well with the initial JVM size set the same as the maximum JVM size, this is not recommended with version 1.4.2 of the JVM (the version embedded in WebSphere Application Server V5.1) as it can cause memory fragmentation, resulting in poor performance.

Determining the values

initial_jvm_heap_size – The initial size of the JVM heap in megabytes. Recommended value: 256 MB.

max_jvm_heap_size – The maximum size of the JVM heap in megabytes. Recommended value: 1 GB to 1.5 GB.

Setting the values

1) Open the WebSphere Application Server Administration Console.

2) Expand the Servers list in the navigation pane.

3) Select Application Servers in the navigation pane.

4) Select the server to manage.

5) Select Process Definition from the Additional Properties pane at the bottom.

6) Select Java Virtual Machine from the Additional Properties pane at the bottom.

7) Set the Initial Heap Size to initial_jvm_heap_size.

8) Set the Maximum Heap Size to max_jvm_heap_size.

9) Repeat this procedure for each IBM Tivoli Identity Manager server.


2.2 Java virtual machine (JVM) options – Sun Solaris Important: This section applies specifically to WebSphere Application Server on Sun Solaris (including later fix pack releases) that use the Sun garbage collection (GC) algorithm. This section does not apply to WebSphere Application Server for AIX, Windows, or Linux. The IBM Java and Sun Java implementations have two different GC algorithms and need to be tuned differently. If you are using WebSphere Application Server on AIX, Windows, or Linux, skip this section.

Before proceeding, determine what JVM version and implementation you are using by using the following commands:

<WebSphere Home Directory>/AppServer/java/bin/java –version <WebSphere Home Directory>/AppServer/java/bin/java –server –version

The output will look something like the following sample: # <WebSphere Home Directory>/AppServer/java/bin/java –version java version "1.4.2_01" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_01-b03) Java HotSpot(TM) Client VM (build 1.4.2_01-b03, mixed mode)

# <WebSphere Home Directory>/AppServer/java/bin/java –server –version java version "1.4.2_01" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_01-b03) Java HotSpot(TM) Server VM (build 1.4.2_01-b03, mixed mode)

Notice the Client VM and Server VM strings. If the output does not include these strings, skip this section, because it does not apply to your JVM.

The Sun 1.4.x and 1.5.x JVM on Sun Solaris supports two different JVM types: client and server. The client JVM is designed to have a faster startup time and is tuned for shorter-running applications than the server JVM. The server JVM is designed for longer-running applications and therefore sets different initial values to some JVM tunings. By default the WebSphere Application Server uses the client JVM. IBM Tivoli Identity Manager runs better under the server JVM than under the client JVM.

Sun’s 1.4.x and 1.5.x JVM uses a generational garbage collection algorithm. How this algorithm works is outside the scope of this document, but we will give a short summary. See the Other resources section at the end of this document for links to more information.

The generational garbage collection algorithm works in this way:

• The JVM divides the Java heap size into two major components: the young generation, which includes Eden and two Survivor pools, and the old generation (sometimes called the tenured generation). When objects are allocated inside IBM Tivoli Identity Manager, they are created inside Eden.

• After Eden becomes full, a partial garbage collection occurs and any objects that are still being used (live objects) are moved to the first Survivor pool. Eden is cleared and object allocation begins again inside Eden.

• After Eden becomes full again, any live objects inside Eden and the first Survivor pool are moved to the second Survivor pool. Eden is cleared and object allocation begins again inside Eden. This process repeats with the objects being switched between the two survivor pools until objects die or are eventually tenured to the old generation.

• When there is no more space inside Eden or the Survivor pools, all remaining live objects are moved to the old generation. Eden is cleared and object allocation begins again.

• When the old generation becomes full, a full garbage collection occurs. Every object within the heap is analyzed and heap compaction occurs within the old generation.

When partial or full garbage collections occur, everything within the JVM stops. This causes small pauses within IBM Tivoli Identity Manager during garbage collections. Partial garbage collections finish very quickly, usually in less than a half second. Full garbage collections take longer to compact the entire heap and can take 10 seconds or more to finish, depending on the size of the heap. Because of this, it is important to minimize the number of garbage collections, particularly full garbage collections. We do this by adjusting the size of the Java heap, the new generation, and the Survivor pools.

The following figure illustrating the JVM heap layout.

Eden Tenured GenerationSurvivor pools

max_new_sizemax_jvm_size

new_ratiosurvivor_ratio


jvm_type – The type of JVM to use, either -client or -server. Recommended value: -server

max_new_size – The maximum size of the new generation within the JVM in megabytes. Set this to 1/3 of your maximum heap size.

new_ratio – The ratio of the new generation to the old generation. Set this to 2, thereby setting the size of your new generation to 1/3 of your heap size.

survivor_ratio – The ratio of a single survivor pool to Eden. Set this to 3, thereby setting the size of each survivor pool to 1/5 of your new generation (note this is not 1/4 because there are two survivor pools).

target_survivor_ratio – A target for the amount of survivor space to be used after a garbage collection. This value is used by the JVM to calculate for how many iterations an object will stay in the survivor pools before being tenured to the old generation. Set this to 75 to allow more time for objects to die in the survivor pool before being tenured.

With a total JVM heap size of 512 MB, the recommendations above would yield the following heap configuration:

Eden Tenured GenerationSurvivorpools

170 MB

512 MB

102 MB 34 MB

342 MB

Calculating the values

argument_string = jvm_type -XX:MaxNewSize=max_new_sizem -XX:NewRatio=new_ratio -XX:SurvivorRatio=survivor_ratio -XX:TargetSurvivorRatio=target_survivor_ratio

Tip: Ensure the MaxNewSize value has a trailing ‘m’ to specify that the value is in megabytes.

Example: For a JVM size of 512 MB, use the following argument string:

argument_string = -server -XX:MaxNewSize=170m -XX:NewRatio=2 -XX:SurvivorRatio=3 -XX:TargetSurvivorRatio=75

Setting the values

1) Open the Administration Console.






5) Select Process Definition from the Additional Properties pane at the bottom.

6) Select Java Virtual Machine from the Additional Properties pane at the bottom.

7) Append argument_string to Generic JVM arguments Important: If the Generic JVM arguments field is not empty (for example, other control options like -Dclient.encoding.override=UTF-8 are already present), insert the jvm_type before the existing values in that field and append the new values at the end. In addition, verify that the new values being appended are not already set in the existing string. If they are already set, remove the existing duplicates.

8) Repeat this procedure for each IBM Tivoli Identity Manager server.

2.3 Decreasing individual transaction times By default, the WebSphere Application Server uses the Application Server Facilities (ASF) mode when determining which JMS message to process next. This mode can result in delays when processing JMS messages with different priorities. IBM Tivoli Identity Manager uses priorities to execute interactive tasks like password changes and account creations first over non-interactive traffic like creating a person. In ASF mode, transactions with higher priority are not spotted immediately and can be delayed for a few seconds while other lower-priority transactions are being worked on. ASF mode and non-ASF mode both result in the same transaction throughput, the difference is in the order that messages are processed.

The non-ASF mode within WebSphere Application Server results in higher priority messages always being accessed before lower-priority messages, which can increase the speed of some IBM Tivoli Identity Manager transactions such as the creation of a person that will result in an automatic account generation. With ASF mode, the account creation can be delayed if other person creates are occurring in the system. With non-ASF mode, the account creation will happen before the next person create is executed.

To enable non-ASF mode, set the NON.ASF.RECEIVE.TIMEOUT parameter to a value greater than 0. To re-enable ASF mode, set the value to 0 or delete the parameter.


non-asf-timeout – The time in milliseconds before timing out and running the cleanup function. Recommended value: 500.

Setting the values

1) Open the Administration Console.




5) Select Message Listener Service from the Additional Properties pane at the bottom.

6) Select Custom Properties from the Additional Properties pane at the bottom.

7) Click New to create a new property

8) Set the Name to NON.ASF.RECEIVE.TIMEOUT

9) Set the Value to non-asf-timeout

10) Click OK to save the changes

11) Repeat steps 3 through 10.

12) Stop and restart the application servers.


The Other resources section has links to additional information on non-ASF mode.

2.4 IBM HTTP Server The WebSphere Application Server uses the IBM HTTP Server as a front-end server in a single-server installation and as a load balancer between nodes in a cluster installation. The default configuration parameters for the IBM HTTP Server work well for small and medium configurations, but should be increased in environments with a large number of concurrent users.

2.4.1 Connections The IBM HTTP Server can be configured to limit the number of connections that it will accept at one time. This value might be too small if the servers experience a large number of concurrent users. Connections to the server are maintained until a user logs out (which does not always happen) or until after 10 minutes of inactivity, when the connection is automatically closed.

The IBM HTTP Server supports the HTTP/1.1 KeepAlive request that allows a client to make multiple HTTP requests through a single persistent connection. There is a limit on the number of KeepAlive requests that a single connection will accept. After this limit is reached, the connection is closed and another connection must be established. The default value might be too small for some external provisioning processes such as a Java Naming and Directory Interface (JNDI) feed.


ibmhttp_home – The home directory of the IBM HTTP Server, such as /usr/IBMHttpServer.

max_connections – The maximum number of connections that can be made to the HTTP server at one time. This should be set to the maximum number of concurrent users you expect on your system. Recommended value: 200. This can be increased to a maximum value of 600, if you have a large number of concurrent users. Exceeding a value of 600 requires more operating system resources.

Setting the values

Tip: The MaxClients parameter on Windows is called ThreadsPerChild.

Edit the ibmhttp_home/conf/httpd.conf file and update the following entries: MaxClients max_connections

Stop and restart the IBM HTTP Server for these changes to take effect.

2.5 IBM WebSphere MQ The WebSphere Application Server uses an embedded copy of the IBM MQSeries® messaging service to manage Java Message Service (JMS) messages. The IBM Tivoli Identity Manager server configures several persistent queues (the exact name and number can vary slightly from version to version). The IBM Tivoli Identity Manager server uses these queues during various provisioning actions. During large provisioning policies it is possible to reach the limit on the number of uncommitted messages, so this value should be increased.


queue_manager_name – The name of the queue manager, such as WAS_vanexel_jmsserver. You can find a list of valid queue manager names by running dspmq.

max_uncommitted_messages – The maximum number of uncommitted messages. Default value: 10000. Recommended value: 50000.

Setting the values

1) Start the MQ Script interpreter: runmqsc queue_manager_name


2) Alter the maximum uncommitted messages size: alter qmgr maxumsgs(max_uncommitted_messages)

3) Confirm the change: display qmgr maxumsgs

4) Exit the MQ Script interpreter: end


3 IBM Tivoli Identity Manager application

The IBM Tivoli Identity Manager application includes several configuration files that provide an area for tuning various parts of the application’s performance. These are in the data/ directory under the IBM Tivoli Identity Manager product home directory.

3.1 Failed/Warnings links IBM Tivoli Identity Manager FP25 introduced new links on the View Completed Requests page. These links allow users to view details for transactions that resulted in a Failed or Warning status. The performance of these links were improved in IF45 and can be improved further with the addition of another index (see IBM DB2 – Indexing for more information). These links can be disabled if you do not want them.


show_errorpopup – Enable the Failed/Warning links on the View Completed Requests page. Valid values: true or false.

Setting the values

Edit the ui.properties file and change or add the following property: enrole.auditing.errorpopup.enabled=show_errorpopup

3.2 List control options The ui.properties file has several parameters that control how many entries will be included in a list, such as viewing the people in an organizational unit, and how many pages will be available in that list. Setting these values too high can result in Java OutOfMemory errors due to heap fragmentation.


page_size – The number of entries to show on a page. Default value: 10.

page_link_max – The number of links to show at the bottom of a page. This is not the total number of pages that can be seen, but the number that is displayed before the Next button is shown for additional pages. Default value: 10.

max_search_results – The maximum number of results to return from a search. It is not recommended that this value be increased as doing so can result in heap fragmentation issues. This value should always be equal to or greater than (page_size * page_link_max). If (page_size * page_link_max) is larger than 1000, it is recommended that you decrease one of the two parameters until the product is less than 1000. Default value: 1000.

Setting the values

Edit the ui.properties file and change the following properties: enrole.ui.pageSize=page_size enrole.ui.pageLinkMax=page_link_max enrole.ui.maxSearchResults=max_search_results

This must be done on all nodes in a clustered environment. The IBM Tivoli Identity Manager application should be restarted for these values to take effect.

3.3 Recycle bin When objects such as people, accounts, roles, and provisioning policies are deleted from the IBM Tivoli


Identity Manager system using either the graphical user interface (GUI) or the application program interface (API), these objects are not removed from the underlying directory server but rather moved into the recycle bin. The recycle bin is implemented as the following LDAP container:

ou=recycleBin, ou=itim, ou=<tenant>, <suffix>

When LDAP entries are moved under this DN due to a deletion, the attribute erIsDeleted is set to the value Y to enable IBM Tivoli Identity Manager to identify these objects as entries it should neither display to the user nor act upon. Because of the filter that IBM Tivoli Identity Manager uses, having a large number of entries in the recycle bin can negatively impact performance. It is recommended that the size of the recycle bin be kept as small as possible for optimum performance.

There are several ways to remove entries from the recycle bin. IBM Tivoli Identity Manager includes a script that will delete entries in the recycle bin older than a specified age range. See the discussion of the recycle bin age limit in IBM Tivoli Identity Manager Server Installation and Configuration Guide for WebSphere Environments for more information.

An alternate method is to use an LDAP display tool to view the entries and delete them directly in the directory server. Be careful to delete only the entries themselves and not the ou=recycleBin container. Similarly, it is possible to use a combination of the ldapsearch and ldapdelete commands to delete entries. For example:

ldapsearch -h <host> -p <port> -D <user> -w <password> \ -b "ou=recycleBin,ou=itim,ou=<tenant>,<suffix>" -s sub "erisdeleted=Y" dn | \ ldapdelete -h <host> -p <port> -D <user> -w <password>

After deleting entries from the recycle bin when using an IBM Tivoli Directory Server, run runstats to make IBM DB2 pick up the changes. See the IBM Tivoli Directory Server – Runstats section for more information.

3.4 Reconciliations Reconciliations are resource-intensive operations and can take a while for services with a large account population. Limiting the number of attributes returned by the adapter and processed by IBM Tivoli Identity Manager can improve reconciliation performance. Large reconciliations can also exceed the default Max Duration and if so the value can be increased. Finally, larger reconciliations can benefit from using paged searches.

3.4.1 Limiting attributes returned from the adapter Some adapters (such as the adapter for Microsoft Active Directory) can limit the attributes that are returned to the IBM Tivoli Identity Manager server during reconciliations. This can reduce the amount of work required by the adapter to retrieve or calculate the values of the attributes as well as amount of data sent back to IBM Tivoli Identity Manager.

Consult the adapter documentation for information specific to that adapter. See also the IBM Tivoli Identity Manager adapters section.

3.4.2 Limiting the attributes evaluated During reconciliation, any attributes that were identified as having been changed are updated within the IBM Tivoli Identity Manager directory server. Before this update takes place, the new value is evaluated against the provisioning policy that governs the account to ensure that the change is allowed within the policy, and if not, a policy enforcement is triggered. Any change to the account will trigger the policy evaluation for that account regardless if the change would invalidate the policy.

To reduce the number of policy evaluations, limit the attributes that are evaluated during reconciliation. Some endpoints (such as Microsoft Active Directory) contain attributes that change frequently but are


seldom used to enforce policy, such as last logon time. If these attributes are required, consider setting up a second reconciliation to reconcile these attributes on a more infrequent schedule and remove them from the more frequently running reconciliations. If possible, reconcile only those attributes that are required for policy evaluation.


excluded_attributes – The list of attributes that are returned from the adapter to exclude from processing within IBM Tivoli Identity Manager. Ideally these would be all attributes except those that are required for policy evaluation.

Setting the values

1) Log into IBM Tivoli Identity Manager as a user with sufficient privileges to edit the service being reconciled.

2) Select the Provisioning tab.

3) Navigate to the service in the organizational tree.

4) Select the service to modify.

5) Select Reconciliation.

6) Select the reconciliation schedule to modify.

7) Select the Query tab.

8) Move all excluded_attributes from to the Excluded Attributes list.

9) Click Submit.

3.4.3 Maximum duration Large reconciliations can sometimes exceed the default maximum duration as specified in the reconciliation schedule. When this limit is reached, the reconciliation is halted. Increase the limit to allow longer-running reconciliations to complete.


max_duration – The maximum time in minutes that the reconciliation should run. To calculate this value, do an initial run with a very large duration and measure the time required. Consider setting the maximum duration to 10% above this time. Default value: 600.

Setting the values

1) Log into IBM Tivoli Identity Manager as a user with sufficient privileges to edit the service being reconciled.

2) Select the Provisioning tab.

3) Navigate to the service in the organizational tree.

4) Select the service to modify.

5) Select Reconciliation.

6) Select the reconciliation schedule to modify.

7) Set the Max Duration to max_duration.

8) Click Submit.


3.4.4 Paged searches IBM Tivoli Identity Manager IF23 included an important update to help alleviate Java OutOfMemory errors for large reconciliations. During a reconciliation, all of the existing accounts are pulled from the LDAP server to be compared with incoming accounts from the service being reconciled. If the number of accounts is large enough, an OutOfMemory error can occur in the JVM as the heap fills up with LDAP objects.

In IF23, these LDAP searches were changed to take advantage of paged searches if the underlying LDAP server supports it and the appropriate parameter is set in the enRole.properties file. Paged searching is disabled by default since it places an additional load onto the LDAP server and some LDAP servers have a limit on the number of concurrent paged searches. If enabling this parameter, ensure that the underlying LDAP server is configured to accept as least as many paged search requests as you have concurrent reconciliations.

Tip: A related parameter governs the enabling of server-side sorting. It is recommended that you do not enable server-side sorting. See also Server-side sorting.


paging_enabled – Enable LDAP paging for searches that support it. Valid values: true or false. Default value: false.

paging_size – Size of the paging request to the LDAP server. If this value is set too high, the LDAP server might ignore the paging request. It is not recommended that this value be set larger than 128. Default value: 128

Setting the values

Edit the enRole.properties file and change or add the following properties: enrole.search.paging.enable=paging_enabled enrole.search.paging.pagesize=paging_size

This must be done on all nodes in a clustered environment. Restart the IBM Tivoli Identity Manager application for these values to take effect.

3.4.5 enrole.reconciliation properties The enRole.properties file has two properties that impact reconciliation performance. Testing has shown the default values of these parameters to be optimal. You should not change them unless explicitly asked to do so by IBM support personnel. Increasing these values can decrease reconciliation performance.

enrole.reconciliation.accountcachesize=2000 enrole.reconciliation.threadcount=8

3.5 Server-side sorting When retrieving lists of objects from LDAP to show in the interface, IBM Tivoli Identity Manager sorts the results before presenting them to the user. When paged searches are enabled (see Paged searches for more information) IBM Tivoli Identity Manager also supports having the LDAP server sort the results. Enabling server-side sorting (via the enrole.search.sss.enabled property in enRole.properties) can have a large negative impact when viewing large organizational units and therefore it is recommended that this option remain disabled in most environments.

3.6 ACI cache In IBM Tivoli Identity Manager IF26 and later, three additional properties are available to control the ACI cache for improving performance or reducing memory requirements.



refresh_interval – The time in minutes between ACI cache refreshes. Increasing this value can result in better ACI performance but will result in ACI changes taking longer to be enforced. Default value: 5.

user_cache_size – The maximum number of ACI evaluation results to cache per user. Increasing this value can result in better performance for systems with many ACIs but will require more memory from the JVM. Default value: 50.

cache_size – The maximum size of the ACI cache. Increasing this value can result in better performance but will require more memory from the JVM. Default value: 1000.

Setting the values

Edit the enRole.properties file and change or add the following properties: enrole.accesscontrollist.refreshInterval=refresh_interval enrole.userACICache.maxSize=user_cache_size enrole.accesscontrollist.maxSize=cache_size

This must be done on all nodes in a clustered environment. The IBM Tivoli Identity Manager application should be restarted for these values to take effect.


4 IBM Tivoli Identity Manager adapters

It is sometimes necessary to tune the IBM Tivoli Identity Manager adapters when doing large provisioning changes or reconciliations. This section supplements, rather than superseding, the documentation included with the adapter.

4.1 Microsoft Active Directory The Microsoft Active Directory adapter returns attributes to IBM Tivoli Identity Manager that are not directly retrieved from Active Directory, but rather calculated from other Windows sources. Querying these external sources can slow down Active Directory reconciliations and can be disabled if these attributes are not needed.

The Home Directory Security and Mailbox Permissions are two such attributes. Retrieving this information requires looking up the appropriate access control entry, which is a costly operation. Setting ReconHomeDirSecurity and ReconMailboxPermissions to FALSE in the adapter registry will disable this overhead.

Working with Windows Terminal Services (WTS) attributes can slow down provisioning and reconciliation as well. There are two adapter registry keys that control access to these attributes:

• WtsEnabled – This key controls the adapters’ access to WTS attributes. If this key is enabled (set to TRUE) the adapter will have access to provision and reconcile WTS attributes. If this key is disabled (set to FALSE) the adapter will not provision WTS attributes if requested, nor will it return them during reconciliation. The default value for this key is FALSE.

• WtsDisableSearch – This key controls whether the adapter will return WTS attributes during a reconciliation (a “search” from the adapter’s perspective). If this key is enabled (set to TRUE), WTS attributes will not be returned in a reconciliation but the attributes will still be updated in account provisions. If this key is disabled (set to FALSE), WTS attributes will be returned in a reconciliation. This key only applies if the WtsEnabled key is set to TRUE. The default value for this key is TRUE.


5 IBM Tivoli Directory Integrator

IBM Tivoli Directory Integrator is often used in an IBM Tivoli Identity Manager environment for creating custom adapters.

5.1 DSML connector and chunked encoding The DSML connector is often used when creating custom agents for returning data back to IBM Tivoli Identity Manager. The DSML connector can return information either all at once or in smaller units using a mechanism called chunked encoding. Each methods has advantages and disadvantages. Chunked encoding applies to all responses to IBM Tivoli Identity Manager although it is most relevant for reconciliations.

Using chunked encoding prevents the DSML file from being created in-memory within IBM Tivoli Directory Integrator. Chunked encoding also allows IBM Tivoli Identity Manager to begin processing account reconciliations while the adapter is still streaming accounts back to the server. The implementation of the chunked encoding Sun API causes results in the entire DSML file being built within the IBM Tivoli Identity Manager JVM and can result in OutOfMemory errors for large reconciliations.

Without chunked encoding, the DSML file is created in-memory within the IBM Tivoli Directory Integrator and can result in OutOfMemory errors for large reconciliations.

In general it is recommended that chunked encoding not be used for custom adapters with over 10,000 accounts. Instead, increase the maximum JVM size within IBM Tivoli Directory Integrator to avoid OutOfMemory errors within that product. Decreasing the number of attributes being returned from the custom adapter will improve memory usage as well. Alternatively, change the adapter to be RMI-based, as these adapters do not suffer from this OutOfMemory problem.


6 Database servers

The IBM Tivoli Identity Manager product supports two different database systems: IBM DB2 and Oracle Database. Each database requires slightly different tunings.

Each database server should have at least one processor and 1 GB of RAM. It can reside on a single-processor server by itself or share a multi-processor server with other applications, but the requirements of 1 GB of RAM per one CPU is the minimum for the database server. Tuning the database is one of the most important tunings for the IBM Tivoli Identity Manager product.

6.1 IBM DB2 Tuning IBM DB2 to run with the IBM Tivoli Identity Manager product involves adjusting the buffer pools, modifying the number of connections, modifying internal database values, adding table space, adjusting logs, indexing, and running runstats.

6.1.1 Quick guide for setting the IBM DB2 tuning parameters This section includes the commands needed to tune the IBM DB2 parameters. This section assumes you have been through the tunings before, know what the desired values are, and need a reminder of how to set them.

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool enrolebp size enrolebp_npages db2 "ALTER TABLESPACE ENROLE_DATA ADD ( \ FILE '/database_home/database_name/NODE0000/SQL00001/container_name' num_pages) \ PREFETCHSIZE 16 OVERHEAD 24.1 TRANSFERRATE 0.9 BUFFERPOOL ENROLEBP" db2 "ALTER TABLESPACE ENROLE_INDEXES ADD ( \ FILE '/database_home/database_name/NODE0000/SQL00001/container_name' num_pages) \ PREFETCHSIZE 16 OVERHEAD 24.1 TRANSFERRATE 0.9 BUFFERPOOL ENROLEBP" db2 "ALTER TABLESPACE TEMP_DATA ADD ( \ FILE '/database_home/database_name/NODE0000/SQL00001/container_name' num_pages) \ PREFETCHSIZE 16 OVERHEAD 24.1 TRANSFERRATE 0.9 BUFFERPOOL ENROLEBP" db2 update db cfg for itim_database using logsecond logs_secondary db2 update db cfg for itim_database using logfilsiz logs_size db2set DB2_RR_TO_RS=YES

6.1.2 Buffer pools IBM DB2 buffer pools are the secondary buffer for IBM Tivoli Directory Server. These buffer pools should be large enough that most table searches can be read directly from memory instead of using the disk. This value can be measured by looking at the hit ratio for the buffer pools. See the IBM DB2 performance monitoring section for information about calculating the buffer pool hit ratio.

The IBM Tivoli Identity Manager database has two buffer pools: IBMDEFAULTBP and ENROLEBP. IBMDEFAULTBP is used as a buffer for table spaces with small extent sizes (4 KB) and ENROLEBP is used as a buffer for table spaces with large extent sizes (32 KB). Most of the tables within the IBM Tivoli Identity Manager database use the table space with a large extent size and thus will use the ENROLEBP buffer pool. Use a 1:3 memory ratio between IBMDEFAULTBP and ENROLEBP.


First, determine the following values for your system:

total_mem_avail – The total amount of physical memory in bytes.

mem_for_itimdb – The amount of memory in bytes to allocate to the IBM Tivoli Identity Manager database. This value should be small enough that it will reside in physical memory and not be swapped out to disk. Recommended value: 500000000 (500 MB) or greater.


Next, calculate the following values:

ibmdefaultbp_npages = (mem_for_itimdb / 4096) * 0.25 enrolebp_npages = (mem_for_itimdb / 32768) * 0.75

Setting the values

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool enrolebp size enrolebp_npages

Reviewing the values

To view the current buffer pool sizes, connect to the database as the database administrator and run the following command:

db2 "select bpname, npages, pagesize from syscat.bufferpools"

6.1.3 Connections The IBM Tivoli Identity Manager server uses JDBC connections from WebSphere Application Server to communicate with the database. The number of connections from the application server to the database depends on the needs of the application. Both application servers allow you to set maximum connection values. Performance testing has not shown any need to increase the maximum connection values from their default value of 50. The IBM DB2 variable MAXAPPLS must be set to a value greater than the maximum possible connections from all nodes. The default value for MAXAPPLS is AUTOMATIC, which is the recommended setting. If you need to set it lower, set MAXAPPLS to five more than the total maximum possible number of connections.


itim_database_name – The name of your IBM Tivoli Identity Manager database, such as itim.

num_connections – Maximum number of connections. See the IBM WebSphere Application Server section to calculate the maximum number of connections, and add five to that value.

Setting the values

As the database administrator, connect to the database and run the following command: db2 update db cfg for itim_database_name using maxappls num_connections

6.1.4 Table spaces The IBM Tivoli Identity Manager product uses a database managed space (DMS) table space to store its data. This type of table space gives better performance than system managed space (SMS) table spaces but requires you to preallocate disk space for the database to use. The default size of the IBM Tivoli Identity Manager table space is 512 MB for ENROLE_DATA, 256 MB for ENROLE_INDEXES, and 256 MB for TEMP_DATA. This is frequently not large enough and should be increased by either adding more containers, enabling the autoresize option for the table space, or both.

It is possible to add additional table space containers and to enable autoresize on the table spaces depending on your specific environment, disk restrictions, and table space layouts.

6.1.4.1 Adding additional table space containers

Add more table spaces by using the IBM DB2 alter tablespace command. If possible, add files that reside on another physical drive because IBM DB2 performs better if a table space has multiple containers on multiple drives. The creation and adoption of these altered table spaces is not immediate. Examine the output of the alter tablespace command as it executes and rerun the command if the database is busy altering a table space.



tablespace_name – The name of the table space onto which you want to add containers. The IBM Tivoli Identity Manager table space names are ENROLE_DATA, ENROLE_INDEXES, and TEMP_DATA.

database_home – The home directory of your database administrator, such as /home/db2inst1.

database_name – The name of the database such as db2inst1. This is a subdirectory under database_home.

container_name – The name of the file to create to hold the table space container, such as enrole_data2.

num_pages – The number of 32 KB pages to add to the table space. To calculate the number of pages from the amount of disk space, divide the size in megabytes by 0.032768. A 512 MB table space would be 15625 pages.

Setting the values

As the database administrator, connect to the database and run the following command for each table space:

db2 "ALTER TABLESPACE tablespace_name ADD ( \ FILE '/database_home/database_name/NODE0000/SQL00001/container_name' num_pages) \ PREFETCHSIZE 16 OVERHEAD 24.1 TRANSFERRATE 0.9 BUFFERPOOL ENROLEBP"

6.1.4.2 Enabling autoresize

An alternative to adding another table space is to enable the current table space to grow larger. This feature is available in IBM DB2 V8.2.2 (IBM DB2 V8.2 FP2 or IBM DB2 V8.1 FP9) and later. If a table space has autoresize enabled, any containers for that table space will automatically grow if the containers become full. This can decrease the administrative overhead for these DMS table spaces; however, care must be taken to ensure that the disks the containers are on do not become full.

The autoresize option can be enabled on both new and existing table spaces using the IBM DB2 alter tablespace command.


tablespace_name – The name of the table space on which you want to enable autoresize. The IBM Tivoli Identity Manager table space names are ENROLE_DATA, ENROLE_INDEXES, and TEMP_DATA.

Setting the values

As the database administrator, connect to the database and run the following command for each table space on which you want to enable autoresize:

db2 "ALTER TABLESPACE tablespace_name AUTORESIZE YES”

6.1.5 Transaction logs IBM DB2 keeps logs during transaction processing. During large transactions the default log number and sizes might be too small and cause transaction rollbacks. Increasing the size and number of log files available to IBM DB2 solves this issue.

IBM DB2 has two types of log files: primary and secondary. Primary logs are allocated when the database is started and remain allocated until the database is stopped. Secondary logs are allocated as needed after the primary logs are full and are released after they are no longer needed.

For best performance, move the transaction logs to a different physical drive than where the database is located. This might not be necessary with intelligent data storage devices.



Increase the number of secondary logs to be prepared when large transactions occur. The default size of the log files is 1000 4 KB pages, or 4 MB. Increase this value to 10000 4 KB pages, or 40 MB. Note that this will increase the size of both your primary and secondary log files.

itim_database – The name of the IBM Tivoli Identity Manager database, such as itim.

logs_secondary – The number of secondary logs. Recommended value: 12.

logs_size – The size of the primary and secondary logs in 4 KB pages. Recommended value: 10000.

log_path – The path where the transaction logs should be located.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for itim_database using logsecond logs_secondary db2 update db cfg for itim_database using logfilsiz logs_size db2 update db cfg for itim_database using newlogpath log_path

Stop and restart the database instance for these changes to take effect.

6.1.6 Database application heaps Some of the queries that the Tivoli Identity Manager application submits to the DB2 server result in complex SQL statements. The Tivoli Identity Manager installation program will increase the default application heap sizes to useable values. If you see transaction rollback errors in the itim.log file, increase the values of the heaps in increments of 256 until the errors stop.


IBM Tivoli Identity Manager adjusts the values of the following parameters from their default state. Further adjustment is required for proper tuning.

itim_database – The name of the Tivoli Identity Manager database, such as itim.

applheap_size – The value of applheapsz in 4 KB pages. Initial value: 2048.

appctl_size – The value of app_ctl_heap_sz in 4 KB pages. Initial value: 1024.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for itim_database using applheapsz applheap_size db2 update db cfg for itim_database using app_ctl_heap_sz appctl_size


6.1.7 Indexing Adding an index to a heavily used table can greatly increase performance. Without indexes, DB2 must scan every row of the table until it finds the specified data. With an index, it uses a more efficient search method. The following indexes have been found to increase database performance.

Important: It is very important to run runstats on the system after adding new indexes. See the IBM DB2 - Runstats section for more information.

Setting the values

As the database administrator, connect to the database and run the following commands.

Tip: Some of these indexes might already exist, possibly under a different name. If there are "unable to create index" errors, they can be ignored as duplicates.

db2 'create index PROCESS_SUB_X on ENROLE.PROCESS ("SUBMITTED" DESC, "PARENT_ID" ASC) \ MINPCTUSED 10 ALLOW REVERSE SCANS'


db2 'create index PROCESS_ID_ST on ENROLE.PROCESS ("ID" ASC, "STATE" ASC) MINPCTUSED 10 \ ALLOW REVERSE SCANS'

db2 'create index PROCESSDATA_ID_DEF on ENROLE.PROCESSDATA \ ("PROCESS_ID" ASC, "DEF_ID" ASC, "VALUE_LAST_MODIFIED" ASC) MINPCTUSED 10 \ ALLOW REVERSE SCANS'

db2 'create index SCH_MSG_SVR on ENROLE.SCHEDULED_MESSAGE ("SERVER" ASC) MINPCTUSED 10 \ ALLOW REVERSE SCANS'

The following index will improve performance when viewing completed requests sorted by time the activity was completed.

db2 'create index ENROLE.PROCESS_VPC2_X on ENROLE.PROCESS \ ("PARENT_ID" ASC, "COMPLETED" DESC, "STATE" ASC) MINPCTUSED 10 ALLOW REVERSE SCANS'

These indexes will improve performance when drilling down into completed or pending requests for more information. If the PROCESS_PID_TYPE_X index is added, the PROCESS_PID_X index can be dropped.

db2 'create index ENROLE.PROCESSLOG_PID_X on ENROLE.PROCESSLOG \ ("PROCESS_ID” ASC) MINPCTUSED 10 ALLOW REVERSE SCANS'

db2 'create index ENROLE.PROCESS_PID_TYPE_X on ENROLE.PROCESS \ ("PARENT_ID” ASC, “TYPE” ASC) MINPCTUSED 10 ALLOW REVERSE SCANS'

db2 'drop index ENROLE.PROCESS_PID on ENROLE.PROCESS'

The Warning/Failed links on the completed requests page that were introduced in IBM Tivoli Identity Manager FP25 can cause long waits when selected. The query was optimized in IF45 but further improvements can be seen from the following index.

db2 'create index ENROLE.ACT_PID_RS_TY on ENROLE.ACTIVITY \ ("PROCESS_ID" ASC, "RESULT_SUMMARY" ASC, "TYPE" ASC) MINPCTUSED 10 \ ALLOW REVERSE SCANS'

6.1.8 Runstats IBM DB2 requires statistics on the number of rows in the tables and what indexes are available to efficiently fulfill queries. It is important to update these table and index statistics after large Directory Server Markup Language (DSML) loads, HR feeds, and reconciliations. If you experience high CPU usage or poor IBM DB2 performance, run the runstats command on all of the tables in the database. IBM DB2 reorgchk does not update index statistics and is not a replacement for runstats. To update index statistics, run the runstats command on each table individually. Scripts (perftune_runstats.sh and perftune_runstats.bat) that detect the version of IBM DB2 and runs the runstats command against all tables for a given schema in a database are provided in Appendix A – Scripts and files.

If the runstats command is run in a working environment, allow the connected applications to continue to write to the database. Use the "allow write access" option to allow users to write to a database while runstats is running.

Use the runstats command on an idle or lightly-used database because it requires update locking on the system statistics table to update the database statistics. A database with a heavy load might experience transaction rollbacks due to the system acquiring locks on the tables that are used by the database optimizer to fulfill queries.

In addition to running runstats on all tables within the database, you must manually update the statistics table for the ACTIVITY, PROCESS, PROCESSDATA, and SCHEDULED_MESSAGE tables to ensure a minimum cardinality. Setting a minimum cardinality on these tables helps the IBM DB2 query optimizer and can decrease locking issues in the database.


Use the runstats command on every table in the ENROLE schema. As the database administrator, connect to the database and use the following command to get a listing of all tables in the schema:

db2 list tables for all | grep ENROLE

Setting the values

For each table in the ENROLE schema, run the following IBM DB2 runstats command:


db2 runstats on table ENROLE.table_name with distribution and \ detailed indexes all allow write access

Manually update the database statistics table for the workflow tables: db2 update sysstat.tables set card = 50000 where tabname = ‘ACTIVITY’ and card < 50000 db2 update sysstat.tables set card = 50000 where tabname = ‘PROCESS’ and card < 50000 db2 update sysstat.tables set card = 50000 where tabname = ‘PROCESSDATA’ and card < 50000 db2 update sysstat.tables set card = 50000 where tabname = ‘SCHEDULED_MESSAGE’ \ and card < 50000

6.1.9 Maximum open files In an attempt to work well with other applications running on the system, DB2 sets a limit on the number of files it keeps open through the maxfilop setting. After reaching this limit, DB2 will close a currently open file to open the new one. This can cause a performance loss on systems that do not need this restriction on the number of open files. The Tivoli Identity Manager installation program will raise the default value higher. If database snapshots show that database files have been closed, increase this value in increments of 64.


IBM Tivoli Identity Manager adjusts the values of the following parameters from their default state. Further adjustment is required for proper tuning.

max_files_open – The maximum number of files DB2 will have open at any one time. Initial value: 256.

Setting the values

As the database administrator, connect to the database and run the following command: db2 update db cfg for itim_database using maxfilop max_files_open

6.1.10 Lock tuning

6.1.10.1 Lock type

The IBM Tivoli Identity Manager application requires that RR_TO_RS locking be enabled. This setting allows select statements that involve rows that are deleted but not yet committed to be fulfilled without waiting for the transaction to complete.

Tip: This setting only applies to IBM DB2 V7 and earlier. The type 2 indexes introduced in IBM DB2 V8 make this setting unnecessary and will be ignored if set.

Setting the values

Run the following command as the IBM Tivoli Identity Manager database administrator: db2set DB2_RR_TO_RS=YES

Stop and restart the database instance for this change to take effect.

6.1.10.2 Lock list and maximum locks

The performance team has found that the default settings for the DB2 lock list (locklist) and maximum locks (maxlocks) are adequate for most installations. These should be increased if the local DB2 administrator recommends so for your environment.


6.1.10.3 Lock timeout

The IBM Tivoli Identity Manager application can generally be classified an online transaction processing (OLTP) application. As such, it is normally recommended that the lock timeout (locktimeout) value within IBM DB2 be decreased from the default of infinity, represented by the value -1. IBM Tivoli Identity Manager requires the lock timeout set to infinity to work correctly – do not modify this from the -1 value. Not all components of the application recover from a lock timeout and making such a change can result in transaction rollback errors.

6.2 Oracle Database

6.2.1 Server configuration The default Oracle configuration uses the “small” settings in the init.ora file for the database. Increasing the options in this file to use the “middle” or “large” values will provide faster performance. Consult with an Oracle DBA or administrator, or Oracle documentation for more information on tuning of the Oracle server.

6.2.2 Table spaces During database configuration, the IBM Tivoli Identity Manager application creates several small table spaces. The table spaces are large enough to evaluate the product and for small testing environments, but must be increased for a production environment. The initial size of the IBM Tivoli Identity Manager table space is 160 MB for ENROLE_DATA and 160 MB for ENROLE_INDEXES. Both table spaces are configured to autoextend up to 1 GB in size. The ITIML000_DATA table space, if present, has a maximum size of 80 MB. This is frequently not large enough and should be increased by either adding more containers, increasing the maximum size, or both.

Additionally, consider spreading the data across multiple physical disks either during the initial database configuration or afterwards by adding additional table space containers.

6.2.2.1 Spreading database data

Spreading the database files across multiple disks will decrease the I/O contention in the database and improve Oracle performance. When creating the Oracle database, as documented in the IBM Tivoli Identity Manager Server Installation and Configuration Guide, consider spreading the table space files across multiple disks. The following SQL statements illustrate the spreading for a UNIX® system that has four hard drives (disk1 through disk4 with Oracle on disk1):

Important: The SQL statements below are for illustrative purposes only. They are environment specific – not only the file systems, but also the sizes that you can allocate to each table space file. Consult your Oracle DBA and tailor the statements to your environment before you apply them.


disk# – The disk to spread the table spaces onto (in our example, disks 1 through 4).

tablespace_size – The size to make each table space (in our example between 1 GB and 3 GB).

Setting the values create tablespace RBS \ datafile '/disk2/oradata/rbs01.dbf' size 1000m; create temporary tablespace TEMP \ tempfile '/disk2/oradata/temp01.dbf' size 1000m reuse; create tablespace ENROLE_DATA \ datafile '/disk3/oradata/enrole_data_01.dbf' size 3000m; create tablespace ENROLE_INDEXES \ datafile '/disk4/oradata/enrole_indexes_01.dbf' size 1500m;


Tip: The itim_home/config/rdbms/oracle/create_rollbackSegment.sql script puts the rollback segment table space on ORACLE_HOME (because it only specifies the file name). Consider changing these to use a different disk if your environment allows.

6.2.2.2 Adding additional table space datafiles

The maximum size of the initial table spaces is unlikely to be large enough for a production system. Increase the amount of table space available by either increasing the maximum size of the existing datafile within a table space or add additional datafiles. When adding additional datafiles, consider adding them to different physical drives to decrease I/O contention.


tablespace_name – The name of the IBM Tivoli Identity Manager table space to alter, such as ENROLE_DATA, ENROLE_INDEXES, or ITIML000_DATA.

datafile_name – The name of the file to use when adding additional datafiles to a table space or modifying an existing datafile. Consider using a separate physical drive. Example value: /data/ou1/app/oracle/oradata/itimdb/enrole_data2.dbf.

initial_size – The initial size of the datafile. Example value: 512M.

maxsize_string – String used to set the maximum size of the datafile. Specify UNLIMITED to allow the datafile file to grow unbounded or maxsize <number> to limit it to a specific size. Example: maxsize 2048M.

Setting the values

To alter the maximum size of a table space, use the following SQL statement: alter database datafile 'datafile_name' autoextend on maxsize_string;

To add additional datafiles to a table space, use the following SQL statement: alter tablespace tablespace_name add datafile 'datafile_name' \ size initial_size autoextend on maxsize_string;

6.2.3 Indexing Adding an index to a heavily used table can greatly increase performance. Without indexes, Oracle must scan every row of the table until it finds the specified data. With an index, it uses a more efficient search method. The following indexes have been identified as increasing database performance.

Setting the values

As the database administrator, connect to the database and run the following SQL statements: create index PROCESS_SUB_X on ENROLE.PROCESS ("SUBMITTED" DESC, "PARENT_ID" ASC); create index PROCESS_ID_ST on ENROLE.PROCESS ("ID" ASC, "STATE" ASC); create index PROCESSDATA_ID_DEF on ENROLE.PROCESSDATA ("PROCESS_ID" ASC, \ "DEF_ID" ASC, "VALUE_LAST_MODIFIED" ASC); create index SCH_MSG_SVR on ENROLE.SCHEDULED_MESSAGE ("SERVER" ASC);

6.2.4 Updating statistics Gather and update database statistics at regular intervals, from one week to one month on a production IBM Tivoli Identity Manager system, or after a large amount of processing has been completed. These statistics are used by Oracle to make query decisions on locating information that have a dramatic impact on how fast Oracle can return requests. The DBMS_STAT package must be installed to issue the following DBMS_STAT commands.


database_instance – The name of the database instance, such as enrole.


Setting the values

Create a file named Oracle_dbms.stat_cmds.txt containing the following information: exec dbms_stats.gather_schema_stats(ownname => ‘database_instance’, cascade => true);

As the system user, execute sqlplus and run the following command: @ Oracle_dbms.stat_cmds.txt

Generating statistics can take from several minutes to several hours for a large database. Execution should be issued during off-peak times.


7 Directory servers

The IBM Tivoli Identity Manager product supports two different directory servers: IBM Tivoli Directory Server and Sun ONE Directory Server (formerly known as Netscape iPlanet Directory Server).

7.1 IBM Tivoli Directory Server With respect to tuning, IBM Tivoli Directory Server can be divided into two parts: the IBM Tivoli Directory Server process and IBM DB2.

In a well-tuned environment, the IBM Tivoli Directory Server process and the IBM DB2 processes consume approximately the same amount of CPU cycles. IBM DB2 can max out the CPU usage trying to fulfill queries in a poor manner.

Both IBM Tivoli Directory Server and IBM DB2 have caches that speed up data retrieval. Optimizing your available memory is the key to tuning IBM Tivoli Directory Server. When a read request comes in to IBM Tivoli Directory Server, it checks the filter cache to see if has seen that search filter before. If it has, the results are pulled from the cache, otherwise the query goes to IBM DB2. After the search filter is evaluated, IBM Tivoli Directory Server pulls the entries that match the search filter from the entry cache. If the values are not in the entry cache, it queries IBM DB2. For each request to IBM DB2, IBM DB2 checks to see if the data resides in a buffer pool. If not, it reads the value from the disk. Ideally, all requests to the directory server register an IBM Tivoli Directory Server cache hit or an IBM DB2 buffer pool hit for the quickest response. Queries that require disk access can be very slow.

If you encounter problems with the IBM Tivoli Directory Server, see the Troubleshooting – IBM Tivoli Directory Server section for help with common issues.

7.1.1 Quick guide for setting the IBM Tivoli Directory Server tuning parameters

This section includes the commands needed to tune the IBM Tivoli Directory Server parameters. This section assumes you have been through the tunings before, know what the desired values are, and need a reminder of how to set them.

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool ldapbp size ldapbp_npages db2 update db cfg for ldap_database using logsecond logs_secondary db2 update db cfg for ldap_database using logfilsiz logs_size db2 update db cfg for ldap_database using newlogpath log_path

Stop IBM Tivoli Directory Server, edit /etc/ibmslapd.conf and update the following configuration options: ibm-slapdDbConnections: ldap_db_connections ibm-slapdACLCache: acl_cache ibm-slapdACLCacheSize: acl_cache_size ibm-slapdFilterCacheSize: filter_cache_size ibm-slapdEntryCacheSize: entry_cache_size

Stop and restart IBM DB2 and the IBM Tivoli Directory Server server for these changes to take effect.

7.1.2 Cache sizes IBM Tivoli Directory Server has four caches: the access control list (ACL) cache, the filter cache, the entry cache, and the attribute cache. Because the Tivoli Identity Manager server binds as an authoritative user, the ACL cache is used only for internal processes and can be small to allow the memory to be used elsewhere. It has been found that a small value for the ACL cache is beneficial for IBM Tivoli Directory Server performance.


The filter cache is most helpful for programs that issue more read requests than write or update requests, because the entire filter cache is invalidated at every write. The Tivoli Identity Manager application frequently updates the directory server, so it is not beneficial to allocate a large filter cache. However, because Tivoli Identity Manager frequently sends identical queries sequentially to the server, it is useful to have a small filter cache for those queries.

IBM Tivoli Directory Server allows you to control how many entries the entry cache can store but does not restrict the size of the cache. The size of each entry in the cache is based on the number and the size of attributes that a given LDAP entry has. Typically, many entries are users and their accounts, which have a fairly constant size. When setting the value for the entry cache, calculate the size of the average entry and divide that into the amount of memory used by the IBM Tivoli Directory Server process. Users with few attributes can generate entry sizes that are approximately 4 KB where users with more attributes can generate entry sizes around 9k. The IBM Tivoli Directory Server Performance Tuning Guide discusses the procedure for determining the size of the average entry.


acl_cache – Specifies whether the ACL cache is used. Recommended value: TRUE (enabled).

acl_cache_size – The size of the ACL cache. Recommended value: 100.

filter_cache_size – The size of the filter cache. Recommended value: 100.

entry_cache_size – The size of the entry cache. Recommended value: max_users * (average_accounts + 1) For example, if you have 25,000 users with two accounts each, you would want 25,000 * (2+1) = 75,000. This value is bounded by the amount of memory allocated to the IBM Tivoli Directory Server process minus the size of the process itself (about 128 MB).

Attention: Do not set the entry cache size larger than you have available physical memory. If the IBM Tivoli Directory Server process size exceeds the amount of available memory, significant performance degradation will occur because of page swapping. Take care when increasing the size of the caches so the amount of memory required does not exceed the maximum amount a process can allocate such as the 2 GB for most 32-bit processes.

Tip: Performance metrics suggest that the attribute cache available in the IBM Tivoli Directory Server V5 and later does not provide a significant performance boost and the memory would be better allocated elsewhere.

Example: Given an average entry cache size of 9 KB, setting the entry cache size to 75,000 would require 675 MB (75,000 * 9 KB = 675,000 KB = 675 MB) of physical RAM for the entry cache alone, not including the 128 MB needed for the server process itself.

Setting the values

Stop IBM Tivoli Directory Server, edit /etc/ibmslapd.conf and update the following configuration options: ibm-slapdACLCache: acl_cache ibm-slapdACLCacheSize: acl_cache_size ibm-slapdFilterCacheSize: filter_cache_size ibm-slapdEntryCacheSize: entry_cache_size

Restart IBM Tivoli Directory Server for these changes to take effect.

Caches are only one part of tuning the IBM Tivoli Directory Server. Tuning the underlying IBM DB2 database, as mentioned in the sections below, has equal or perhaps greater performance impact than tuning the caches – do not skip the IBM DB2 tunings.

7.1.3 Paging parameters IBM Tivoli Directory Server supports returning search results in pages to allow the client more control over receiving the data. Because paged searches require more resources on the directory server, IBM Tivoli Directory Server has limits on whether paged searches can be performed by non-administrative users and how many concurrent paged searches can occur at one time.


If paged searches are enabled within IBM Tivoli Identity Manager (see Paged searches) it is important to ensure the paged search parameters are set correctly.


allow_non_admin – Specifies if non-administrative users are allowed to request paged searches. Recommended value: TRUE if IBM Tivoli Identity Manager is binding as a non-administrative user or FALSE otherwise. Default value: TRUE

concurrent_paged_searches – The number of concurrent paged searches allowed. This value should be 1 more than the total number of concurrent reconciliations configured to run at one time. Default value: 3

Attention: Paged searches require more resources be available to the directory server. When increasing the number of concurrent paged searches, monitor resource utilization on the directory server to ensure overall performance does not degrade.

Setting the values

Stop IBM Tivoli Directory Server, edit /etc/ibmslapd.conf and update the following configuration options: ibm-slapdPagedResAllowNonAdmin: allow_non_admin ibm-slapdPagedResLmt: concurrent_paged_searches

Restart IBM Tivoli Directory Server for these changes to take effect.

7.1.4 Buffer pools IBM DB2 buffer pools are the secondary buffer for IBM Tivoli Directory Server. These buffer pools should be large enough that most table searches can be read directly from memory instead of using the disk. This value can be measured by looking at the hit ratio for the buffer pools. See the IBM DB2 performance monitoring section for information about calculating the buffer pool hit ratio.

The IBM Tivoli Directory Server database (ldapdb2 by default) has two buffer pools: IBMDEFAULTBP and LDAPBP. IBMDEFAULTBP is used as a buffer for table spaces with small extent sizes (4 KB) and LDAPBP is used as a buffer for table spaces with large extent sizes (32 KB). Most of the tables in the ldapdb2 database use the table space with a small extent size and use IBMDEFAULTBP. Use a 3:1 memory ratio between IBMDEFAULTBP and LDAPBP.


Allocate enough memory to the IBM DB2 buffer pools so the buffer pool hit ratio is greater than 95% and allocate the rest of the memory to the IBM Tivoli Directory Server process and the caches. The following formula can help in determining the buffer pool values for your server.

First, determine the values for your system:

ldap_database – The name of the IBM Tivoli Directory Server database, such as ldapdb2.

mem_for_ldapdb2_bps – The amount of memory in bytes to allocate to the ldapdb2 buffer pools. This value should be small enough that it will reside in physical memory and not be swapped out to disk. Recommended value: 500000000 (500 MB) or greater.

Next, calculate the following values:

ibmdefaultbp_npages = (mem_for_ldapdb2_bps / 4096) * 0.75 ldapbp_npages = (mem_for_ldapdb2_bps / 32768) * 0.25

Setting the values

As the database administrator, connect to the database and run the following commands: db2 alter bufferpool ibmdefaultbp size ibmdefaultbp_npages db2 alter bufferpool ldapbp size ldapbp_npages


7.1.5 Connections There are two connection values to modify for IBM Tivoli Directory Server. The first value is the number of connections that IBM Tivoli Directory Server should make to the IBM DB2 back-end server. Set this to 30.

The second modification applies only to Windows systems. Windows allows up to 64 sockets in a select call for each thread. This limits the number of inbound connections to 64. To overcome this problem, set the SLAPD_OCHANDLERS environment variable to specify the number of connection threads, each one supporting 64 connections.


ldap_database_name – The name of your IBM Tivoli Directory Server database, such as ldapdb2.

num_ldap_connections – The number of connections from IBM Tivoli Directory Server to the IBM DB2 back-end server. Recommended value: 30.

For Windows systems, also determine the following values:

max_num_LDAP_connections – The maximum number of IBM Tivoli Directory Server connections (enrole.connectionpool.maxpoolsize) from the enRole.properties file. The IBM Tivoli Identity Manager application default value is 100.

itim_nodes – The number of IBM Tivoli Identity Manager nodes in your cluster.

oc_handlers – (max_num_LDAP_connections * itim_nodes) / 64 rounded up to the nearest integer.

Setting the values

All systems:

To set the number of connections the IBM Tivoli Directory Server server will make to the IBM DB2 server, stop IBM Tivoli Directory Server, edit ibmslapd.conf, and find the stanza:

dn: cn=Directory, cn=RDBM Backends, cn=IBM Directory, cn=Schemas, cn=Configuration

Change the following attribute: ibm-slapdDbConnections: num_ldap_connections

Windows systems:

Set the SLAPD_OCHANDLERS values. Stop IBM Tivoli Directory Server, edit ibmslapd.conf and find the stanza:

dn: cn=Front End, cn=Configuration

Add the following configuration option at the end of the stanza before the blank line: ibm-slapdsetenv: SLAPD_OCHANDLERS=oc_handlers

All systems:

Stop and start the IBM Tivoli Directory Server database and restart IBM Tivoli Directory Server for these changes to take effect.

7.1.6 Transaction logs IBM DB2 keeps logs during transaction processing. During large transactions, the default log number and sizes might be too small and cause transaction rollbacks. This is solved by increasing the size and number of log files available to IBM DB2.


IBM DB2 has two types of log files: primary and secondary. Primary logs are allocated when the database is started and remain allocated until the database is stopped. Secondary logs are allocated, as needed after the primary logs are full, and are released after they are no longer needed.

For best performance, move the transaction logs to a different physical drive than where the database is located. This may not be necessary with intelligent data storage devices, however.


Increase the number of secondary logs to be prepared when large transactions occur. The default size of the log files is 1000 4 KB pages, or 4 MB. Increase this to 10000 4 KB pages, or 40 MB. Note that this will increase the size of both your primary and secondary log files.


logs_secondary – The number of secondary logs. Recommended value: 12.

logs_size – The size of the primary and secondary logs in 4 KB pages. Recommended value: 10000.

log_path – The path where the transaction logs should be located.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using logsecond logs_secondary db2 update db cfg for ldap_database using logfilsiz logs_size db2 update db cfg for ldap_database using newlogpath log_path


7.1.7 Database statement heap The Tivoli Identity Manager application can submit very long LDAP queries to the IBM Tivoli Directory Server. If the query is particularly long, the query will not fit in the DB2 statement heap (stmtheap) and IBM Tivoli Directory Server will return an error to the Tivoli Identity Manager application.

Important: The statement heap is allocated per agent (connection) therefore increasing this value can dramatically increase the memory used by the DB2 server.



stmtheap_size – The value of stmtheap in 4 KB pages. Default value: 2048. Recommended value: 4096.

Setting the values

As the database administrator, connect to the database and run the following commands: db2 update db cfg for ldap_database using stmtheap stmtheap_size

Stop and restart the IBM Tivoli Directory Server database and IBM Tivoli Directory Server for these changes to take effect.

7.1.8 System limits System limits (also called ulimits) might prevent the IBM Tivoli Directory Server process from accessing enough real or virtual memory. Failure to set system limits high enough can cause the IBM Tivoli Directory Server process to core dump or stop without indication.


process_data_size – The maximum data segment size for the process. Minimum value: 256000 (256 MB). Recommended value: unlimited.


virtual_mem_size – The maximum virtual memory size for the process. Minimum value: 256000 (256 MB). Recommended value: unlimited.

Setting the values

As the user who starts the IBM Tivoli Directory Server process, run the following commands prior to starting ibmslapd:

Solaris: ulimit –d process_data_size ulimit –v virtual_mem_size

AIX: ulimit –d process_data_size ulimit –m virtual_mem_size

Optional: Put these commands into the shell startup files for the user.

Alternatively, on AIX you can edit the /etc/system/limits file and change the data and rss limits in the default stanza to process_data_size and virtual_mem_size values respectively or -1 for unlimited. For these limits to take effect, the user must log out of the current session and log back in.

For more information on setting the ulimit correctly for IBM Tivoli Directory Server, see http://www-1.ibm.com/support/docview.wss?uid=swg21206894

7.1.9 Indexing IBM Tivoli Directory Server allows attributes to be indexed for increased search performance. In addition, the underlying DB2 database can benefit from some additional indexes for some versions of IBM Tivoli Directory Server.

7.1.9.1 Attribute indexes

Indexing the attributes that applications search on increases IBM Tivoli Directory Server performance. IBM Tivoli Directory Server indexes are automatically translated into IBM DB2 indexes when you update the IBM Tivoli Directory Server schema for those attributes.

If you extend the LDAP schema in IBM Tivoli Directory Server to include additional attributes, index those attributes that you will search for. Any filter in the Tivoli Identity Manager application (such as with dynamic roles) is translated into a search string for the IBM Tivoli Directory Server.

Tip: IBM Tivoli Directory Server V6 FP2 and later will report messages in the ibmslapd.log file for attributes that are searched on and are not indexed. Consider indexing attributes that are searched on more than 100 times.

The Tivoli Identity Manager application frequently searches against the organization (o), organizational unit (ou), locality (l), and owner attributes that are included in the default schema. In addition, the following IBM Tivoli Identity Manager attributes should be indexed as well: erDraft, erGlobalId, erScope, erVersionID, and erSystemRoleCategory. Index these attributes either through the IBM Tivoli Directory Server Web Administration tool or by importing the itim_indexes_for_ids.ldif file using the ldapmodify command.

After updating the LDAP schema, run DB2 runstats on the database to update the statistics for the newly created indexes.


root_dn – The root DN of the IBM Tivoli Directory Server server.

root_password – The password for the root DN.



Setting the values

1) To use the LDAP Data Interchange Format (LDIF) file, create the itim_indexes_for_ids.ldif file from Appendix A – Scripts and files and place it on the computer.

2) Run the following command to import the LDIF into IBM Tivoli Directory Server: ldapmodify –D root_dn –w root_password –f itim_indexes_for_ids.ldif

7.1.9.2 DB2 indexes

The following indexes improve search performance for some queries and have been included in later versions of IBM Tivoli Directory Server.


schema_name – The schema that the IBM Tivoli Directory Server tables reside in.

Setting the values

As the database administrator, connect to the database and run the following commands.

Tip: Some of these indexes might already exist, possibly with different names. If there are "unable to create index" errors, they can be ignored as duplicates.

db2 'create index schema_name.LDAP_DESC_DEID on schema_name.LDAP_DESC \ (“AEID” ASC, “DEID” ASC) MINPCTUSED 10 ALLOW REVERSE SCANS'

db2 'create index schema_name.OBJECTCLASS1 on schema_name.OBJECTCLASS \ (“EID” ASC, “OBJECTCLASS” ASC) MINPCTUSED 10 ALLOW REVERSE SCANS'

db2 'create index schema_name.OBJECTCLASS2 on schema_name.OBJECTCLASS \ (“OBJECTCLASS” ASC, “EID” ASC) MINPCTUSED 10 ALLOW REVERSE SCANS'

7.1.10 Runstats IBM DB2 requires information on the number of rows in the tables and what indexes are available so that it can efficiently fulfill queries. Run the runstats command on all tables in the database before and after large DSML loads and reconciliations. If you experience high CPU usage or poor IBM DB2 performance, run the runstats command on all tables in the database. IBM DB2 reorgchk does not update index statistics and is not a replacement for runstats. To update index statistics, run the runstats command on each table individually. Scripts (perftune_runstats.sh and perftune_runstats.bat) that detect the version of IBM DB2 and runs the runstats command against all tables for a given schema in a database are provided in Appendix A – Scripts and files.

If the runstats command is run in a working environment, allow the connected applications to continue to write to the database. Use the "allow write access" option to allow users to write to a database while runstats is running.

Use the runstats command on an idle or lightly-used database because it requires update locking on system statistics table to update the database statistics. A database with a heavy load might experience transaction rollbacks due to the system acquiring locks on the tables that are used by the database optimizer to fulfill queries.

In addition to running runstats on all tables within the database, it is necessary to manually update the statistics table for the LDAP_DESC and LDAP_ENTRY tables. The choices IBM DB2 makes about when to use these tables for fulfilling queries for the IBM Tivoli Directory Server are not ideal for IBM Tivoli Identity Manager. Manually adjusting the statistics table helps IBM DB2 make better choices and use these tables at the end of the access plan instead of the beginning.


As the database administrator, connect to the database and use the following command to get a listing of all tables in the schema:

db2 list tables for all | grep LDAPDB2


Setting the values

For each table in the LDAPDB2 schema, run the following DB2 runstats command: db2 runstats on table ENROLE.table_name on all columns allow write access

Manually update the database statistics table for the LDAP_DESC and LDAP_ENTRY tables. db2 update sysstat.tables set card = 9E18 where tabname = 'LDAP_DESC' db2 update sysstat.tables set card = 9E18 where tabname = 'LDAP_ENTRY'

7.2 Sun ONE Directory Server The Sun ONE Directory Server (formerly known as Netscape iPlanet Directory Server) does not provide many parameters to adjust for better performance. The three areas that provide the best performance boost are indexing, the All IDs Threshold value, and the database cache size.

7.2.1 All IDs Threshold value The Sun ONE Directory Server sets an upper bound on how many index entries should be allowed for each attribute indexed. If this value is exceeded, the index is invalidated and is not used during searches. This behavior prevents the server from including all entries in an index and consuming excessive amounts of memory.

The Sun ONE Directory Server Installation and Tuning Guide recommends setting this value to 5% of the size of your directory. In order to properly support IBM Tivoli Identity Manager this value should be increased to the largest number of users, groups, services, or accounts the IBM Tivoli Identity Manager server has stored in the directory.

Changing this value is disruptive to a production system. Set the value based on the estimated size of your directory in its final form. If the current size of your directory is 20,000 but you expect the final size of your directory to be 100,000 entries, use the larger value to calculate the All IDs Threshold value.

See the Sun ONE Directory Server Installation and Tuning Guide for more information on adjusting the All IDs Threshold value.


directory_home – The Sun ONE Directory Server home directory, such as /usr/iplanet/servers/slapd-gso-2DS.

number_of_entries – The number of entries in your directory. A good way to estimate this number is to multiply the number of Tivoli Identity Manager users by the average number of accounts for each user.

suffix_name – The name of the root suffix, such as dc=com.

database_name – The name of the directory server database, such as itim5145.

temp_directory – The name of a temporary directory, such as /tmp. This directory should have enough free space to store an LDIF of your entire directory. Allow 1 KB per Tivoli Identity Manager user.

Setting the values

Attention: Setting this value requires the database to be rebuilt (the data exported and re-imported). If done incorrectly, this process has the potential to destroy information within your directory. It is highly recommended that you back up the information in your directory prior to setting this value.

Tip: Increasing the size of the All IDs Threshold will cause the size of the indexes to grow. Because the indexes are stored in the database cache, the cache size might need to be increased as well. See the See the Sun ONE Directory Server - Cache size section for information about increasing this value.

Edit directory_home/config/dse.ldif file and find the stanza: dn: cn=config,cn=ldbm database,cn=plugins,cn=config


Change the following attribute: nsslapd-allidsthreshold: number_of_entries

You must export and re-import all of your data for this parameter to take effect. This can be done with the db2ldif and ldif2db commands.

1) Export your data: db2ldif -n database_name -a temp_directory/export.ldif -s "suffix_name"

2) Stop the directory server.

3) Import your data: ldif2db -n database_name -i temp_directory/export.ldif

4) Start the directory server.

7.2.2 Indexing Indexing the attributes that applications search on increases Sun ONE Directory Server performance.

If you extend the LDAP schema to include additional attributes, index those attributes that you will search for. Any filter in the IBM Tivoli Identity Manager application (such as with dynamic roles) is translated into a search string for the LDAP server.

The IBM Tivoli Identity Manager application frequently searches against the organization (o), organizational unit (ou), locality (l), and owner attributes. Owner is indexed by default in Sun ONE Directory Server but ou, o, and l are not. Index the ou, o, and l attributes from the directory server configuration console.

Setting the values

1) Open the Directory Server Console.

2) Select the Configuration tab.

3) Expand the Data node in the navigation pane.

4) Expand the suffix of your database in the navigation pane and select the database.

5) Select the Indexes tab.

6) Click Add attribute button.

7) Find OU in the list box and click OK.

8) If the Equality and Presence options are not already selected for the OU attribute, select them.

9) Click Save. A progress dialog displays while the index is added.

10) Click Close after the index creation completes.

11) Repeat steps 7 through 10 for the O and L attributes as well

7.2.3 Cache sizes The caches used by Sun ONE Directory Server V5.1 and V5.2 are slightly different. In V5.2 there are two distinct caches that need to be tuned: the database cache that stores database-level entries and indexes, and the entry cache that stores formatted entries. In V5.1 there is only one cache, the database cache that stores index information, filter requests, and entries.

7.2.3.3 Sun ONE Directory Server V5.1

The directory server uses a database cache of 10 MB by default. This value is too small for almost any installation.



cache_size – The size of the database cache. Recommended value: 256000000 (256 MB). If the entry cache hit ratio on the Status tab for the database is below 95%, increase this value in increments of 128 MB until the hit ratio is above 95% or your system is out of physical memory.

Important: Setting this value greater than the amount of physical RAM available will cause performance degradation as the system swaps the pages out to disk.

Setting the values





5) Select the Database Settings tab.

6) Set the Memory available for cache to cache_size.

7) Click Save.

7.2.3.4 Sun ONE Directory Server V5.2

As previously mentioned, V5.2 has two types of caches: database and entry. The entry cache is used only for base-level searches and thus provides no benefit for subtree or one-level searches. The database cache is used for both subtree and one-level searches as well as base-level searches if the results are not found in the entry cache.

A typical IBM Tivoli Identity Manager does about 50% base searches and 50% one-level and subtree searches. For this reason, we recommend allocating no more than 50% of your total cache memory to the entry cache. For larger directories, you might see better performance benefit by using significantly more memory in the database cache than in the entry cache.


db_cache_size – The size of the database cache. Recommended value: 512000000 (512 MB).

entry_cache_size – The size of the entry cache. Recommended value: 512000000 (512 MB).

Important: Setting these values greater than the amount of physical RAM available will cause performance degradation as the system swaps the pages out to disk.

Important: The nsslapd-dbcachesize (db_cache_size above) and nsslapd-cachememsize (entry_cache_size above) parameters have platform-specific limits. Consult the Sun ONE documentation on these parameters before increasing them above the recommended values identified here.

Setting the values





5) Select the Database Settings tab.

6) Set the Maximum Cache size to db_cache_size.

7) Set the Memory Available for Cache to entry_cache_size.

8) Click Save.


8 Operating Systems

Performance can be improved on some systems by making some operating system-specific changes. The sections below are guidelines only. Consult the documentation for your middleware and apply any operating system tunings they require.

8.1 AIX Performance improvements might be obtained by tuning the virtual memory-management (VMM) settings such as minperm and maxperm. Consult the documentation for your release of AIX for more information.

8.2 Red Hat Update the msgmni kernel variable to 1026.


msgmni_value – 1026. Default value: 16.

Setting the values

As the root user, run the following command: sysctl -w kernel.msgmni=msgmni_value

Update the open file limit to 8000.


ulimit_value – 8000. Default value: 2000.

Setting the values

As the root user, run the following command: ulimit -n ulimit_value

Change the value of nofiles in /etc/security/limits to 8000.


9 Best practices

The IBM Tivoli Identity Manager product can be set up and configured in many ways. The following best practices can help guide you in setting up your environment.

• Each agent modifies the LDAP schema by adding new attributes to support a new service. These attributes are created without indexes, and for services that service thousand of users, a large benefit can be achieved by adding indexes to attributes with many members.

• In general, network latency is not a major performance bottleneck, but it is still a good idea to have as few hops as possible between components. This includes the IBM Tivoli Identity Manager server components, the directory server, database server, agents, and agent endpoints. Ideally all components should be on the same subnet or no more than one hop away and on a 100-megabit or faster network.

• To minimize disk bottlenecks, it is frequently better to have several disks in the system rather than one large drive. IBM DB2 has the ability to use multiple disks but must be required to do so. The IBM Tivoli Identity Manager product itself can benefit from having the log files on a different disk than the messaging queues. Keep in mind that high-end I/O backplanes or other advanced storage systems can overcome this by balancing the load across multiple disks automatically.

• Database and directory activity can be very CPU- and memory-intensive. It is recommended that each application have at least one processor and 2 GB of RAM each. More processors are generally better. To obtain optimal performance, it is not recommended to have all IBM Tivoli Identity Manager components on the same system unless it is a very powerful unit such as an 8 x 1.4 GHz server with 12 GB RAM.

• Complicated provisioning policies can result in complicated directory and database queries with poor performance. Policies with small numbers of roles and services will perform best.

• Dynamic roles affect people in a given scope, either one-level or subtree. When a person object within that scope is modified or added, that role must be re-evaluated. This is true for every dynamic role in the system. For instance, if there are three dynamic roles with subtree scope and a person object within that scope is updated, all three dynamic roles will be re-evaluated. For this reason, it is recommended that you limit the number of dynamic roles, either by number or by scope, that affect people that are modified frequently. It does not matter if the dynamic role ends up enrolling the person or not, the evaluation itself is the performance-impacting overhead.

• Only store person information needed for policy evaluation and account management within IBM Tivoli Identity Manager to reduce attribute updates that are not used for policy enforcement. When a person object is updated either manually or via some automated method such as an HR feed or JNDI update, all provisioning policies that the user is a member of must be re-evaluated to see if the update would change a provisioning action. For this reason it is recommended to minimize person updates when possible.

• Limiting the scope (via placement within the organizational tree) and number of ACIs will increase performance by requiring fewer evaluations. When doing a person search via the APIs, be sure to limit the scope of your search to be as narrow as possible to avoid the system evaluating more ACIs than necessary.

• When enabling WebSphere global security, do not enable Java 2 security unless it is required for your environment. Enabling WebSphere global security automatically enables Java 2 security unless it is explicitly disabled. Having Java 2 security enabled can cause a significant performance degradation to IBM Tivoli Identity Manager.

• When loading users into IBM Tivoli Identity Manager, particularly in bulk when using DSML file or through an IDI Feed, do not include O, OU, or L attributes on the person records. IBM Tivoli Identity Manager includes searches on these attributes for the organizational chart, which can


slow down if large numbers of users have these attributes as well. This is particularly important for larger user populations.

• When loading objects such as people or accounts directly into the IBM Tivoli Identity Manager LDAP server, such as during an initial data load via LDIF, use all numeric values for the erGlobalID rather than an alphanumeric value. Updates were added in FP23 to support larger recons but are only effective if the erGlobalID of the account does not contain non-numeric characters.

• When putting data on a SAN in a failover environment, ensure that separate physical devices in the SAN are used for each failover’s underlying datastore. If, for example, the database for each LDAP server is on the same physical devices you will likely introduce I/O performance problems.

• When working with LPARs, the following items can improve performance; the ones that improve performance the most are listed first:

o SMT should be disabled for IBM Tivoli Identity Manager nodes.

o Have the latest update of the WebSphere Application Server JVM (SR6 improves performance on LPARs).

o Give at least 1 physical CPU to each LPAR.

o Use dedicated instead of virtual processors.

• As a recommendation, each person should have no more than 1200 accounts. Often administrative accounts on target systems are mapped to a single person object resulting in one person with possibly thousands of accounts, which can degrade performance or result in Java OutOfMemory errors. The ability to scale beyond this number will depend on the specifics of the system configuration and hardware.


10 Regular maintenance

Regular maintenance is required to maintain optimal performance for your IBM Tivoli Identity Manager environment.

Regularly empty the IBM Tivoli Identity Manager recycle bin. As the number of objects in the recycle bin increase LDAP performance can degrade. The frequency at which the recycle bin is emptied depends on how frequently deletes occur in the system. Ideally, the recycle bin should contain fewer than 100 items. See IBM Tivoli Identity Manager application – Recycle bin for more information on how to empty the recycle bin.

Update database statistics after a large number of updates. Updating database statistics in the underlying databases can significantly improve performance and should be done on a weekly basis for most environments. This applies to the IBM DB2 database if using IBM Tivoli Directory Server as well as the underlying IBM DB2 or Oracle Database of the IBM Tivoli Identity Manager application. See the appropriate database sections on how to update database statistics.


11 Troubleshooting

A large number of middleware dependencies can complicate the task of finding performance problems with IBM Tivoli Identity Manager. A problem such as a slow DSML feed with account provisioning could be the result of a slow directory server, database locking issues, or an inadequate number of IBM Tivoli Identity Manager workflow threads. This section is designed to assist you in identifying problem areas and provide some pointers on fixing them.

Information in this section is provided with the assumption that you have read and applied the tunings recommended in the previous sections. If this is not the case, stop and review those sections now.

11.1 Log files When something seems to stop working, the first place to check is the msg.log and trace.log files. They often contain useful information about what is currently being processed and any errors that might have occurred. If you suspect that something is not working correctly and the msg.log and trace.log files do not contain enough information, increase the logging in the enRoleLogging.properties file. Edit the file and either uncomment or add one or more of the following lines:

logger.trace.com.ibm.itim.workflow.level=DEBUG_MAX logger.trace.com.ibm.itim.remoteservices.level=DEBUG_MAX

The workflow class is used during all workflow processes. Setting this to DEBUG_MAX can help track down workflow problems.

The remoteservices class communicates with external agents (such as a Windows NT service and a Tivoli Access Manager service but not the ITIM service) during provisioning. Set this to DEBUG_MAX to help diagnose problems with agents.

Changes to the enRoleLogging.properties file are picked up every 10 minutes or whenever the IBM Tivoli Identity Manager application is restarted.

Also, be sure to check the log file for the appropriate middleware components:

• IBM WebSphere Application Server – /usr/WebSphere/AppServer/logs/server_name/

• IBM Tivoli Directory Server – /var/ldap/ibmslapd.log

• Sun ONE Directory Server – sun_one_home/logs/access

• IBM DB2 – instance_home/sqllib/db2dump/db2diag.log

• Oracle Database – *.trc files

11.2 Identifying bottlenecks Start by monitoring the processor and disk usage of every server (IBM Tivoli Identity Manager nodes, directory, database) to see which server is most heavily utilized. Based on this information, review the monitoring and tuning steps specific to that component.

The IBM Tivoli Identity Manager application makes intense usage of both the database and the directory server it is configured into. The database is used as an information staging area and audit trail for provisioning actions. The directory server is used as a permanent storage location and can be heavily queried when evaluating provisioning policies. Because of this usage, it is possible for either the database or the directory server to be a bottleneck during heavy usage or large provisioning changes.

During an action that causes a large provisioning policy to be evaluated, such as adding a new policy or


modifying an existing policy that impacts a large number of users, the affected users must be queried from the directory server. The directory server evaluates the query and returns the matching users. If the directory server is not tuned correctly it can become the bottleneck as the IBM Tivoli Identity Manager server waits for the result set before starting the required provisioning action. Make sue that the directory server is fulfilling the requested queries as quickly and efficiently as possible to minimize this behavior. See the appropriate directory server tuning section for more information.

After the result set is returned, the IBM Tivoli Identity Manager server will begin to enforce the provisioning policy for each user. This process is multi-threaded and benefits from multiple processors (the workflow cluster member in a functional cluster). If it seems that the processors on the server are not being fully utilized, check to see if there is a bottleneck on the LDAP or database server.

Any enforcement action required for a user (account addition, modification, or deletion) is stored as a workflow item in the database. When a thread becomes available, it queries the database for the next workflow item that needs processing and proceeds to act upon that item. This usage can result in access contention and thus locking in the database. Appropriate indexes and access plan statistics will help minimize the number of locks needed to fulfill these requests. See the appropriate database tuning section for more information.

11.3 Specific problems

11.3.1 IBM Tivoli Identity Manager application Problem: Transaction rollback errors in the trace.log file. Solution: Transaction rollbacks can occur for several different reasons, most of them database-related. There should be an error message within the trace.log file giving more information about what went wrong. Some areas to check when you get a transaction rollback:

• Database storage space: If the database runs out of storage space for the table spaces, a transaction rollback error can occur. Increase the amount of disk space allocated to the table spaces to resolve this problem.

• Database locking: If the database encounters extreme locking issues it is possible to get a transaction rollback error. Confirm that the locks are tuned appropriately and update the table statistics.

• Database memory: If there is not enough memory available to database structures to fulfill the requested query, a transaction rollback error might occur. The JNDI error in the trace.log file should indicate which database heap should be increased. Increase the appropriate heap using the recommendations provided in the section for that middleware.

Problem: Java OutOfMemory errors in the trace.log file. Solution: The message is from WebSphere Application Server. The Java virtual machine (JVM) ran out of heap size. Increase the maximum heap size if possible and restart the application server. If the heap size is already at the limit, then you should break up transactions; for example, use fewer services or roles in a provisioning policy.

If the OutOfMemory error occurred during a reconciliation, see the Reconciliations - Paged searches and IBM Tivoli Directory Integrator - DSML connector and chunked encoding sections for some additional suggestions.

Problem: “Error searching for governing policies” found in the trace.log file. Solution: This error is seen because the statement heap for the IBM Tivoli Directory Server database is too small and large LDAP queries are failing. Increase the statement heap as specified in the Database statement heap section under IBM Tivoli Directory Server.


11.3.2 IBM WebSphere MQ Problem: Nothing seems to be going through workflow, even after restarting the application servers, and new items are piling up on the queue. Solution: A message on the message queue might be corrupt and blocking new transactions. Clearing the queues can help.

Attention: Clearing the queues can result in an inconsistent system state. Do not clear your queues unless instructed to do so by a support technician; use the following procedure at that time.

1) Stop the IBM Tivoli Identity Manager application. You can not clear the queues while the IBM Tivoli Identity Manager application is running.

2) Find the name of the queue manager by listing all queues on the system: dspmq

3) Start the MQ Script interpreter: runmqsc WAS_servername_jmsserver

4) List IBM Tivoli Identity Manager queues: display queue('WQ_itim*')

5) For each queue, clear it: clear qlocal('WQ_itim_name')

Tip: If you receive a message that the queue is in use, stop the IBM Tivoli Identity Manager server and try again.

6) Exit the MQ Script interpreter: end

11.3.3 IBM Tivoli Directory Server Problem: Poor search performance. Solution: Slow queries or high CPU utilization with the IBM Tivoli Directory Server can be caused by several factors. Some areas to consider:

1) Check for long-running queries that need indexes. The IBM Tivoli Directory Server uses IBM DB2 to process LDAP queries. By checking IBM DB2 for long-running queries it is possible to find what attributes need indexing. To find how long each query takes, turn on statement cache monitoring within IBM DB2: db2 update dbm cfg using DFT_MON_STMT ON Stop the directory server, restart the database, and restart the directory server for this to take effect. After the monitoring is turned on, duplicate the suspected action within IBM Tivoli Identity Manager. After the action has completed, get a snapshot of the statement cache: db2 get snapshot for dynamic sql on database_name The output will contain stanzas like the following: Number of executions = 12 Number of compilations = 1 Worst preparation time (ms) = 3 Best preparation time (ms) = 3 Internal rows deleted = 0 Internal rows inserted = 0 Rows read = 10024 Internal rows updated = 0 Rows written = 0 Statement sorts = 0 Total execution time (sec.ms) = 136.000663


Total user cpu time (sec.ms) = 62.010000 Total system cpu time (sec.ms) = 10.000000 Statement text = SELECT distinct E.EID FROM LDAPDB2.LDAP_ENTRY AS E, LDAPDB2.LDAP_ENTRY as pchild WHERE E.EID=pchild.EID AND pchild.PEID=? AND E.EID IN (SELECT EID FROM LDAPDB2.OU WHERE OU = ?)

Calculate the average execution time per query by dividing the total execution time by the number of executions: total execution time / number of executions. In the example above: 136 / 12 = 11.33 seconds per execution. Most queries should take a second or less to complete. Queries that take longer might be searching on columns that are not indexed by IBM DB2, and hence are not indexed within the IBM Tivoli Directory Server. Another symptom of this problem is a high average number of rows read, calculated by dividing the rows read by the number of executions. In the above example, the column OU is likely not indexed. Correct this by indexing the OU attribute within the IBM Tivoli Directory Server. See the IBM Tivoli Directory Server – Indexing section for more information. The IBM Tivoli Identity Manager tuning scripts contain a script (perfanalyze_dynamicsql.pl) that will calculate the time per execution for all stanzas and sort the results. See Appendix A – Scripts and files for information on how to get the tuning scripts.

2) Check for low buffer pool hit ratio. The IBM Tivoli Directory Server uses IBM DB2 to process LDAP queries. The IBM Tivoli Directory Server database should have a high (>95%) hit ratio. If the buffer pools are not large enough IBM DB2 must read more information from the disk resulting in high I/O wait. See the IBM DB2 performance monitoring – Buffer pool hit ratio section for information on calculating the buffer pool hit ratio. The IBM Tivoli Identity Manager tuning scripts contain a script (perfanalyze_bufferpools.pl) that will calculate the hit ratio for all buffer pools for you. See Appendix A – Scripts and files for information on how to get the tuning scripts.

Problem: The directory server crashes or hangs for no apparent reason. Solution: Check the size of your cache sizes. If your cache sizes cause the IBM Tivoli Directory Server process to grow beyond what is supported by your operating system’s memory model (in general around 2 GB on 32-bit operating systems) it can crash. See the IBM Tivoli Directory Server Tuning Guide (in Other resources) for more information. Alternatively the ibmslapd process might be hitting an artificial system limit, such as a ulimit. See IBM Tivoli Directory Server – System limits for more information on increasing system limits.

11.3.4 Sun ONE Directory Server Problem: Poor search performance. Solution: Slow queries or high CPU utilization with the Sun ONE Directory Server can be caused by several factors. Some areas to consider:

1) Check for long-running queries that need indexes. Most queries should take less than one second to complete. Queries taking longer than a second are generally searching on attributes that are not indexed. To find queries taking longer than a second, search the Sun ONE Directory server access logs for etime=X where X is an integer greater than 1, X representing the number of seconds the query took to complete. We are searching for queries running 2 seconds or longer. For example: [26/Mar/2004:12:54:25] conn=4236 op=1 msgId=2 - RESULT err=0 tag=101 nentries=1 etime=2

Now search for which query the search time is associated with by searching the same file for the matching connection and operation number, which in the above example is conn=4236 op=1.


The query for the above example was: [26/Mar/2004:12:54:24] conn=4236 op=1 msgId=2 – SRCH base="ou=ibm,dc=com" scope=2 filter="(&(!(erIsDeleted=y))(&(erEnabled=true)(erPolicyMembership=2;*))(objectClass=erProvisioningPolicy))" attrs=ALL

Examine the LDAP query string and identify the attributes being searched on. In the above example, they are erIsDeleted, erEnabled, erPolicyMembership, and objectClass. Ensure that all attributes being searched on are correctly indexed. See the Sun ONE Directory Server – Indexing section for information on how to index attributes.

2) See if the All IDs Threshold value needs to be increased. If the All IDs Threshold value is too small, Sun ONE Directory Server might not be using the indexes for certain attributes. Determine if queries are searching on these attributes by searching for notes=U in the Sun ONE Directory Server access logs. The presence of this flag for a query indicates that one or more attributes being searched on is over the All IDs Threshold and should probably be increased. See the Sun ONE Directory Server – All IDs Threshold value section for more information on how to increase this value.

3) See if the database cache hit ratio is low. If the Sun ONE Directory Server database cache is too small, the server must access the disk for the information and will result in a low database cache hit ratio. Identify the hit ratio for the database by viewing the Monitor tab for that database. The hit ratio should be 95% or higher. If it is lower than 95%, increase the size of your database cache. See the Sun ONE Directory Server – Cache size section for information on increasing the size.


12 IBM DB2 performance monitoring

IBM DB2 provides several tools for troubleshooting and analyzing performance problems. This section contains a primer for how to get started monitoring with IBM DB2. For more extensive information, see IBM DB2 Administration Guide: Performance.

12.1 Enable monitoring To gather performance information, turn on the IBM DB2 monitoring flags. To turn on monitoring, run the following commands as the database administrator. Do this for each database.

db2 update dbm cfg using DFT_MON_STMT ON db2 update dbm cfg using DFT_MON_BUFPOOL ON db2 update dbm cfg using DFT_MON_LOCK ON db2 update dbm cfg using DFT_MON_SORT ON db2 update dbm cfg using DFT_MON_TIMESTAMP ON db2 update dbm cfg using DFT_MON_UOW ON

Stop and restart the database instance for the monitoring to take effect.

Tip: Do not enable the table monitor. It has a slight performance impact when enabled and is not needed for IBM Tivoli Identity Manager.

12.2 Snapshots Snapshots allow you to view the internal state of various IBM DB2 components. The following commands access important IBM DB2 snapshots:

db2 get snapshot for database on database_name db2 get snapshot for dynamic sql on database_name db2 get snapshot for bufferpools on database_name db2 get snapshot for tables on database_name db2 get snapshot for locks on database_name

Alternatively, you can gather all snapshots at one time using the following command: db2 get snapshot for all on database_name

12.3 Statement monitor The statement monitor is useful for examining what is occurring for each request sent to the database. While the statement monitor is turned on, each SQL request is recorded and the results of the queries can be examined looking for missing indexes, execution time, preparation time, database scans, and index scans. The monitor collects a great deal of information and should only be activated for a short time to gather requests.

12.3.1 Set up the monitor If this is the first time you’ve done any monitoring or generated an explain plan on this database you must set up the explain tables:

db2 -tf sqllib/misc/EXPLAIN.DDL db2 "create event monitor dstatement for statements write to file '/tmp/dstatements'" mkdir /tmp/dstatements

12.3.2 Use the monitor To monitor the database statements, connect to the database, clear out any previous data, and turn on the monitor:


db2 connect to ldapdb2 rm -f /tmp/dtatements/* db2 "set event monitor dstatement state 1"

Run the query or the action that you want to monitor and turn off the monitor: db2 "set event monitor dstatement state 0"

Convert the data to a human-readable format: db2evmon -path /tmp/dstatements >/tmp/dstate.out

The do_statistics_monitoring.sh script in the Tuning Guide scripts package automates this process. You can use the explainSQL.sh script to then have IBM DB2 explain how the optimizer will process a particular query, including any use of indexes.

12.4 Buffer pool hit ratio The buffer pool hit ratio gives a good indication of how many data reads are coming from the buffer pool and how many are coming from the disk. The larger the hit ratio, the less disk I/O used. Calculate the buffer pool hit ratio by enabling buffer pool monitoring and taking a database snapshot. Use the following formula:

P = buffer pool data physical reads + buffer pool index physical reads L = buffer pool data logical reads + buffer pool index logical reads Hit ratio = (1-(P/L)) * 100%


13 Other resources

You will find the following resources useful for further tuning of IBM Tivoli Identity Manager:

IBM DB2 Administration Guide: Performance

http://www.ibm.com/cgi-bin/db2www/data/db2/udb/winos2unix/support/v8pubs.d2w/en_main

IBM DB2 Linux Tuning

http://publib.boulder.ibm.com/infocenter/db2help/?topic=/com.ibm.db2.udb.doc/start/t0008238.htm

IBM DB2 Solaris Tuning


IBM Tivoli Identity Manager Server Installation and Configuration Guide for WebSphere Environments

http://publib.boulder.ibm.com/tividd/td/ITIM/SC32-1750-01/en_US/PDF/im460_ins_ws_entprise.pdf

IBM Tivoli Directory Server Tuning Guide

http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer5.2.html (IBM Tivoli Directory Server V5.2) http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer6.0.html (IBM Tivoli Directory Server V6.0)

WebSphere Application Server Monitoring and Tuning Performance

http://www.ibm.com/software/webservers/appserv/infocenter.html

WebSphere Application Server – Tuning Operating Systems

http://publib.boulder.ibm.com/infocenter/wasinfo/v5r1/?topic=/com.ibm.websphere.base.doc/info/aes/ae/tprf_tuneopsys.html

Sun ONE Directory Server Documentation

http://docs.sun.com/app/docs/coll/S1_DirectoryServer_52 (Sun ONE Directory Server V5.2) http://developers.sun.com/prodtech/dirserver/reference/docs/ (Sun ONE Directory Server V5.1)

Sun Java Garbage Collection References

http://java.sun.com/docs/hotspot/gc1.4.2 http://java.sun.com/docs/hotspot/VMOptions.html

IBM Java Garbage Collection References

http://tesch.dfw.ibm.com/java/gc_faq.html http://www.ibm.com/developerworks/ibm/library/i-garbage1

developerWorks articled discussing non-ASF mode

http://www.ibm.com/developerworks/websphere/library/techarticles/0611_titheridge/0611_titheridge.html

http://www.ibm.com/cgi-bin/db2www/data/db2/udb/winos2unix/support/v8pubs.d2w/en_main

http://publib.boulder.ibm.com/infocenter/db2help/index.jsp?topic=/com.ibm.db2.udb.doc/start/t0008238.htm


http://publib.boulder.ibm.com/tividd/td/ITIM/SC32-1750-01/en_US/PDF/im460_ins_ws_entprise.pdf

http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer5.2.html

http://publib.boulder.ibm.com/tividd/td/IBMDirectoryServer6.0.html

http://www.ibm.com/software/webservers/appserv/infocenter.html



http://docs.sun.com/app/docs/coll/S1_DirectoryServer_52

http://developers.sun.com/prodtech/dirserver/reference/docs/

http://java.sun.com/docs/hotspot/gc1.4.2/

http://java.sun.com/docs/hotspot/VMOptions.html

http://tesch.dfw.ibm.com/java/gc_faq.html

http://www.ibm.com/developerworks/ibm/library/i-garbage1




14 Appendix A – Scripts and files

The most recent version of the tuning scripts can be obtained at http://www-1.ibm.com/support/docview.wss?uid=swg27006281


Date post:	05-May-2018
Category:	Documents
Upload:	phunghanh
View:	236 times
Download:	1 times

IBM Tivoli Identity Manager Version 4.6 Performance Tuning...

Documents