IBM SmartCloud Analytics - Log Analysis

PVT performance tuning andconfiguration recommendationVersion 1.2

���

IBM SmartCloud Analytics - Log Analysis

PVT performance tuning andconfiguration recommendationVersion 1.2

���

NoteBefore using this information and the product it supports, read the information in “Notices,” on page 15.

Edition notice

This edition applies to IBM SmartCloud Analytics - Log Analysis and to all subsequent releases and modificationsuntil otherwise indicated in new editions.

References in content to IBM products, software, programs, services or associated technologies do not imply thatthey will be available in all countries in which IBM operates. Content, including any plans contained in content,may change at any time at IBM's sole discretion, based on market opportunities or other factors, and is notintended to be a commitment to future content, including product or feature availability, in any way. Statementsregarding IBM's future direction or intent are subject to change or withdrawal without notice and represent goalsand objectives only. Please refer to the developerWorks terms of use for more information.

© Copyright IBM Corporation 2014.US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contractwith IBM Corp.

Contents

Chapter 1. About this publication . . . . 1Audience . . . . . . . . . . . . . . . 1Publications . . . . . . . . . . . . . . 1

Accessing terminology online . . . . . . . . 1Accessibility . . . . . . . . . . . . . . 1Tivoli technical training. . . . . . . . . . . 1Providing feedback . . . . . . . . . . . . 2Conventions used in this publication . . . . . . 2

Typeface conventions . . . . . . . . . . 2

Chapter 2. Server tuning . . . . . . . 3Enabling hyper threading . . . . . . . . . . 3

Chapter 3. Tuning the operation system 5Increasing the default process and thread count forLinux . . . . . . . . . . . . . . . . . 5

Reducing the TIME WAIT parameter for socketconnections . . . . . . . . . . . . . . 5

Chapter 4. Tuning IBM WebSphereApplication Server . . . . . . . . . . 7Tuning Java Virtual Machine (JVM) options . . . . 7

Chapter 5. Tuning IBM SmartCloudAnalytics - Log Analysis. . . . . . . . 9

Chapter 6. Tuning and configuringMozilla Firefox . . . . . . . . . . . 11Creating dedicated Firefox profiles. . . . . . . 11Configuring Firefox for extensive result sets andcharts . . . . . . . . . . . . . . . . 11

Chapter 7. Hardware sizingconsiderations . . . . . . . . . . . 13

Appendix. Notices . . . . . . . . . . 15Trademarks . . . . . . . . . . . . . . 16

© Copyright IBM Corp. 2014 iii

iv IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 1. About this publication

This guide contains information about how to use IBM SmartCloud Analytics - LogAnalysis.

AudienceThis publication is for users of the IBM SmartCloud Analytics - Log Analysisproduct.

PublicationsThis section provides information about the IBM SmartCloud Analytics - LogAnalysis publications. It describes how to access and order publications.

Accessing terminology onlineThe IBM Terminology Web site consolidates the terminology from IBM productlibraries in one convenient location. You can access the Terminology Web site at thefollowing Web address:

http://www.ibm.com/software/globalization/terminology.

AccessibilityAccessibility features help users with a physical disability, such as restrictedmobility or limited vision, to use software products successfully. In this release, theIBM SmartCloud Analytics - Log Analysis user interface does not meet allaccessibility requirements.

Accessibility features

This information center, and its related publications, are accessibility-enabled. Tomeet this requirement the user documentation in this information center isprovided in HTML and PDF format and descriptive text is provided for alldocumentation images.

Related accessibility information

You can view the publications for IBM SmartCloud Analytics - Log Analysis inAdobe Portable Document Format (PDF) using the Adobe Reader.

IBM and accessibility

For more information about the commitment that IBM® has to accessibility, see theIBM Human Ability and Accessibility Center. The IBM Human Ability andAccessibility Center is at the following web address: http://www.ibm.com/able(opens in a new browser window or tab)

Tivoli technical trainingFor Tivoli® technical training information, refer to the following IBM TivoliEducation Web site at http://www.ibm.com/software/tivoli/education.

© Copyright IBM Corp. 2014 1

Providing feedbackWe appreciate your comments and ask you to submit your feedback to the IBMSmartCloud Analytics - Log Analysis community.

Conventions used in this publicationThis publication uses several conventions for special terms and actions, operatingsystem-dependent commands and paths, and margin graphics.

Typeface conventionsThis publication uses the following typeface conventions:

Bold

v Lowercase commands and mixed case commands that are otherwisedifficult to distinguish from surrounding text

v Interface controls (check boxes, push buttons, radio buttons, spinbuttons, fields, folders, icons, list boxes, items inside list boxes,multicolumn lists, containers, menu choices, menu names, tabs, propertysheets), labels (such as Tip:, and Operating system considerations:)

v Keywords and parameters in text

Italic

v Citations (examples: titles of publications, diskettes, and CDsv Words defined in text (example: a nonswitched line is called a

point-to-point line)v Emphasis of words and letters (words as words example: "Use the word

that to introduce a restrictive clause."; letters as letters example: "TheLUN address must start with the letter L.")

v New terms in text (except in a definition list): a view is a frame in aworkspace that contains data.

v Variables and values you must provide: ... where myname represents....

Monospace

v Examples and code examplesv File names, programming keywords, and other elements that are difficult

to distinguish from surrounding textv Message text and prompts addressed to the userv Text that the user must typev Values for arguments or command options

2 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 2. Server tuning

Read the following section to find out how to tune your server.

Enabling hyper threadingIBM SmartCloud Analytics - Log Analysis uses multi threading and hyperthreading for much of the processing. Therefore, it is important to enable hyperthreading on your server.

Before you begin

Ensure that you have KVM access for the server that you are configuring.

About this task

This topic describes how to enable hyper threading on the IBM XSeries hardwareonly. If you do not use this hardware, refer to the manual for your specifichardware for instructions about how to enable hyper threading on your server.

Procedure1. Restart the server.2. When the launch screen displays, press F1.

3. The System Configuration and Boot Management screen displays as below. Toenable hyper-threading, select System settings > Processors > Hyper-threading> Enable.

© Copyright IBM Corp. 2014 3

4. To return to the main menu, press Esc twice. Select Save Settings.5. Select Exit and restart the server.

4 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 3. Tuning the operation system

Read this section to learn how to optimize your operating system.

Increasing the default process and thread count for LinuxThe maximum number of threads and processes that a non-root user can use islimited in Linux. This can cause performance issues in basic installations whereIBM SmartCloud Analytics - Log Analysis and all the required components areinstalled on a single server.

About this task

The most recent versions of the Red Hat Enterprise Linux (RHEL) and SUSE Linuxoperating systems limit this setting to 1024. To maximize the performance of IBMSmartCloud Analytics - Log Analysis, change this setting.

Procedure1. Use the default IBM SmartCloud Analytics - Log Analysis user to log in to the

backend of IBM SmartCloud Analytics - Log Analysis.2. Run the following command:

ulimit –u

If the value that is returned is 1024 or less, proceed to the next step.3. Use the root user ID to log in to the IBM SmartCloud Analytics - Log Analysis

server.4. Open the limits.conf file that is in the /etc/security directory and edit the

following rows:scala hard nproc 4096scala soft nproc 4096

Save the file.5. Use the default IBM SmartCloud Analytics - Log Analysis user ID to log in to

the IBM SmartCloud Analytics - Log Analysis server.6. To confirm that 4096 is the new value for the max users processes parameter,

run the following command:ulimit –u

7. Repeat the procedure for each instance of IBM InfoSphere Data Explorer.8. Start the IBM SmartCloud Analytics - Log Analysis and IBM InfoSphere Data

Explorer servers.

Reducing the TIME WAIT parameter for socket connectionsTCP socket connections in Linux operating systems is set to 60 seconds by default.You can increase this to optimize performance.

About this task

To check the current TIME WAIT value, run the following command:$ cat /proc/sys/net/ipv4/tcp_fin_timeout

© Copyright IBM Corp. 2014 5

The default value is 60 seconds.

Procedure1. Use the root user ID to log in to the IBM SmartCloud Analytics - Log Analysis

server.2. To temporarily change the value of the TIME WAIT parameter, run the

following command:# echo 15 > /proc/sys/net/ipv4/tcp_fin_timeout

3. To permanently change the value of the /proc/sys/net/ipv4/tcp_fin_timeoutparameter to 15 seconds, run the following command:net.ipv4.tcp_fin_timeout=15

4. To restart the networking service, run the following command:# service network restart

6 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 4. Tuning IBM WebSphere Application Server

To optimize IBM SmartCloud Analytics - Log Analysis, you may want to tune IBMWebSphere Application Server.

For more information about how to do this, see Tuning the Liberty profile.

You can also change the stack size in IBM SmartCloud Analytics - Log Analysis.For more information about how to do this, see Performance considerations.

Tuning Java Virtual Machine (JVM) optionsTo optimize performance, change the JVM settings.

About this task

As a minimum, we recommend that you complete steps 1 and 2.

Procedure1. Stop the IBM SmartCloud Analytics - Log Analysis server.2. Open the jvm.options file.3. Ensure that the minimum and maximum heap sizes are the same. For example:

-Xms4g-Xmx4g

4. For a IBM SmartCloud Analytics - Log Analysis reference server with 16 ormore physical cores, set the number of garbage collection threads to 6:-Xgcthreads6

5. Start the IBM SmartCloud Analytics - Log Analysis server.

© Copyright IBM Corp. 2014 7

8 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 5. Tuning IBM SmartCloud Analytics - Log Analysis

Read the following section to find out how to optimize IBM SmartCloud Analytics- Log Analysis.

Data ingestion

To optimize performance, it is important that you use the correct method of dataingestion. Each type of data ingestion is optimized for different scenarios.

If you want to ingest historical log data, use the data collector. If you want toingest log files in real time, use the IBM Tivoli Monitoring Log File Agent or theEIF.

For more information, see Batch loading data into IBM SmartCloud Analytics - LogAnalysis

© Copyright IBM Corp. 2014 9

10 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 6. Tuning and configuring Mozilla Firefox

IBM SmartCloud Analytics - Log Analysis supports Mozilla Firefox EnterpriseService Editions (ESR) versions 17 and 24.

Creating dedicated Firefox profilesTo further optimize the performance of IBM SmartCloud Analytics - Log Analysis,you can create a dedicated Firefox profile that is used only for IBM SmartCloudAnalytics - Log Analysis.

Before you begin

Ensure that you have the required permissions within your organization to createnew Firefox profiles.

About this task

In most cases, the default Firefox profile is adequate for IBM SmartCloud Analytics- Log Analysis. However, in some cases, the performance of Firefox can bediminished. This diminished performance can reduce the overall performance ofIBM SmartCloud Analytics - Log Analysis. Firefox performance can diminish forvarious reasons, such as third-party extensions or plug-ins and customizing of thedefault profile.

Procedure1. Open a command-line window and navigate to the location of the Firefox

executable file.2. To open the Firefox Profile Manager, run the following command:

firefox.exe -p

3. To start the Create Profile wizard, click Create Profile > Next.4. Enter a new profile name, for example SCALA. You can use the default location

for settings and preferences.5. To start Firefox, click Finish.6. When Firefox opens, it prompts the user to select the profile that is used for the

new session. Select the profile that you created in the previous steps.

Results

For more information about Firefox profiles, see Use the Profile Manager to createand remove Firefox profiles.

Configuring Firefox for extensive result sets and chartsIf you want to use IBM SmartCloud Analytics - Log Analysis to process manyfacets or very detailed charts, you can configure Firefox to reduce the number ofJava Script timeout messages that are displayed on the Firefox UI.

© Copyright IBM Corp. 2014 11

About this task

If you use IBM SmartCloud Analytics - Log Analysis to display many facets or avery detailed and intricate chart, you may notice that Firefox generates a numberof error messages about a Java Script timeout. To continue working, the IBMSmartCloud Analytics - Log Analysis user must confirm this message.

You can change the default settings in Firefox so that the number of thesemessages is reduced and overall performance is optimized.

Procedure1. Log in to Firefox. If you created a dedicated Firefox profile for IBM SmartCloud

Analytics - Log Analysis, use it.2. Open a new tab. Enter about:config in the Search bar and press enter. To

continue, click the I'll be careful, I promise button.3. Enter dom.max script run time in the Search bar. Wait for the setting to

display in the Preferences table. The default value is 10 seconds.4. To change the default setting 10 - 60 seconds, double-click on dom.max script

run time and change the value 10 - 60 seconds.

12 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Chapter 7. Hardware sizing considerations

To ensure optimal performance of medium and large scale deployments of IBMSmartCloud Analytics - Log Analysis, we recommend the following hardwareconfiguration.v Use non virtualized, bare metal infrastructurev Enable hyper-threading in both the IBM SmartCloud Analytics - Log Analysis

and the IBM InfoSphere Data Explorer servers.v The underlying storage system should consist of:

– At least 8 300GB hard disks with at least 10,000 RPMs.– RAID 5 is recommended as the minimum level required for the IBM

InfoSphere Data Explorer file system. RAID 10 is recommended as theoptimal configuration.

v For the IBM InfoSphere Data Explorer file system, we recommend that you useRAID 10 as the RAID configuration. RAID 5 is also supported. However, youmust use one of these, IBM InfoSphere Data Explorer does not support any ofthe other RAID levels.

v For the IBM SmartCloud Analytics - Log Analysis server, we recommend thatyou use RAID 1 or higher. This assumes that you install IBM InfoSphere DataExplorer on a separate server.

v Network-attached storage is not supported or recommended.v Remotely shared file systems are not recommend for either server.v We recommend that all external based file systems are SAN only.v The underlying IP network must support a minimum of 1GB per second transfer

rates.v We recommend periodic auditing and analysis of the installation to ensure that

performance remains optimal and that future requirements are identified quickly.

If you do not follow our recommendations and use a virtual environment, werecommend that the CPU, memory, and disk IO are each dedicated and not shared.

© Copyright IBM Corp. 2014 13

14 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Appendix. Notices

This information was developed for products and services offered in the U.S.A.IBM may not offer the products, services, or features discussed in this document inother countries. Consult your local IBM representative for information on theproducts and services currently available in your area. Any reference to an IBMproduct, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product,program, or service that does not infringe any IBM intellectual property right maybe used instead. However, it is the user's responsibility to evaluate and verify theoperation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matterdescribed in this document. The furnishing of this document does not give youany license to these patents. You can send license inquiries, in writing, to:

IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785 U.S.A.

For license inquiries regarding double-byte (DBCS) information, contact the IBMIntellectual Property Department in your country or send inquiries, in writing, to:

Intellectual Property LicensingLegal and Intellectual Property LawIBM Japan, Ltd.1623-14, Shimotsuruma, Yamato-shiKanagawa 242-8502 Japan

The following paragraph does not apply to the United Kingdom or any othercountry where such provisions are inconsistent with local law:

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THISPUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHEREXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESSFOR A PARTICULAR PURPOSE.

Some states do not allow disclaimer of express or implied warranties in certaintransactions, therefore, this statement might not apply to you.

This information could include technical inaccuracies or typographical errors.Changes are periodically made to the information herein; these changes will beincorporated in new editions of the publication. IBM may make improvementsand/or changes in the product(s) and/or the program(s) described in thispublication at any time without notice.

Any references in this information to non-IBM Web sites are provided forconvenience only and do not in any manner serve as an endorsement of those Websites. The materials at those Web sites are not part of the materials for this IBMproduct and use of those Web sites is at your own risk.

© Copyright IBM Corp. 2014 15

IBM may use or distribute any of the information you supply in any way itbelieves appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purposeof enabling: (i) the exchange of information between independently createdprograms and other programs (including this one) and (ii) the mutual use of theinformation which has been exchanged, should contact:

IBM Corporation2Z4A/10111400 Burnet RoadAustin, TX 78758 U.S.A.

Such information may be available, subject to appropriate terms and conditions,including in some cases payment of a fee.

The licensed program described in this document and all licensed materialavailable for it are provided by IBM under terms of the IBM Customer Agreement,IBM International Program License Agreement or any equivalent agreementbetween us.

Any performance data contained herein was determined in a controlledenvironment. Therefore, the results obtained in other operating environments mayvary significantly. Some measurements may have been made on development-levelsystems and there is no guarantee that these measurements will be the same ongenerally available systems. Furthermore, some measurement may have beenestimated through extrapolation. Actual results may vary. Users of this documentshould verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers ofthose products, their published announcements or other publicly available sources.IBM has not tested those products and cannot confirm the accuracy ofperformance, compatibility or any other claims related to non-IBM products.Questions on the capabilities of non-IBM products should be addressed to thesuppliers of those products.

All statements regarding IBM's future direction or intent are subject to change orwithdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily businessoperations. To illustrate them as completely as possible, the examples include thenames of individuals, companies, brands, and products. All of these names arefictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

If you are viewing this information in softcopy form, the photographs and colorillustrations might not be displayed.

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks ofInternational Business Machines Corp., registered in many jurisdictions worldwide.Other product and service names might be trademarks of IBM or other companies.A current list of IBM trademarks is available on the Web at “Copyright andtrademark information” at www.ibm.com/legal/copytrade.shtml.

16 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

Adobe, Acrobat, PostScript and all Adobe-based trademarks are either registeredtrademarks or trademarks of Adobe Systems Incorporated in the United States,other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registeredtrademarks of Oracle and/or its affiliates.

Linux is a trademark of Linus Torvalds in the United States, other countries, orboth.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks ofMicrosoft Corporation in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and othercountries.

Other product and service names might be trademarks of IBM or other companies.

Appendix. Notices 17

18 IBM SmartCloud Analytics - Log Analysis: PVT performance tuning and configuration recommendation

����

Product Number:

Printed in USA