A Performance Boost for your

IBM Notes Client

Christoph Adler

panagenda

16.11.2015 1

Thanks to our sponsors!

7th Sutol Conference, November 2015

16.11.2015 2

@cadler80

#SUTOL

A Performance Boost for your IBM Notes Client

SUTOL Technical Conference 2015

@cadler80

#SUTOL

panagenda contributes tothe best collaboration landscapesall around the world

@cadler80

#SUTOL

IBM (Lotus) Notes / Domino since 2001

Consulting for projects

• Administration

• Migration / Consolidation

• Client Management

• Application Management

Since 2012 TAM at panagenda with core competence in

• Notes Client Management

• ICS Infrastructure Analysis and Optimizations

• Cloud onboarding

Good to know…

• Frequent traveler

• Speaks (very) fast

• Beer or Wine? Beer!

Christoph Adler Technical Account Manager (Senior Consultant)

5

@cadler80

#SUTOL

Before we start ...

6

I love Notes.

@cadler80

#SUTOL

What makes your Notes Client START slow

@cadler80

#SUTOL

What makes your Notes Client START slow

12

If you‘re not on at least 9.0.1 yet 9.0.1 is a significant improvement

• Pre-loading the client might also be recommended

EOM of IBM Lotus products

• http://www-01.ibm.com/software/lotus/support/lifecycle/

@cadler80

#SUTOL

What makes your Notes Client START slow (Continued)

13

~50% of the entire data volume in the Data directory changes on a DAILY basis in most setups

• 5 - 10% in Data\workspace ( up to 300 files)

• DISABLE Antivirus scanning for at least*.ns**.ndk*.xml*.properties

in Data Directory.

@cadler80

#SUTOL

What makes your Notes Client START slow (Continued)

14

Client crashes/database fixups

• especially bad for network based data directories clients

Outdated Hardware

• You need >1-2 GB memory if you want to use Notes AND something else

• Old fixed disks with fragmentation

• Fun Exercise: Buy a Solid State Disk and make Notes start off of that ;-)

Tons of startup programs/scripts launching during the “boot storm”

• if the disk is unable to keep up, it‘s not Notes' fault

• Again, see http://bit.ly/ACzO6Z

@cadler80

#SUTOL

What makes your Notes Client START slow (Continued)

15

Outdated Data directories

• Virtually 100% of all NTFs (Laptops: 90%) can be removed by

• Cleaning up old Notes 4+5+6 templates

• using ini:SharedDataDirectory= (also with Single User install!)

Operating System

• Windows 7 boots ~40% faster than XP

• Windows 8.1 ~55% faster than XP

• ~20% faster than Windows 7, supposedly

@cadler80

#SUTOL

What makes your Notes Client START slow - ODS

16

ODS = On Disk Structure

• ODS 16 = Notes 2

• ODS 17 = Notes 3

• ODS 20 = Notes 4 (or Templates)

• ODS 41 = Notes 5

• ODS 43 = Notes 6 & 7

• ODS 48 = Notes 8

• ODS 51 = Notes 8.5 & 9

• ODS 52 = Notes 9.0.1

The difference between ODS 43 and 52 = up to 80% LESS FILE I/OThis helps with ANY client AND server hardware

@cadler80

#SUTOL

What makes your Notes Client START slow – ODS (Continued)

17

CAUTION: The ODS is not automatically updated when upgrading clients

• Except for names.nsf, bookmark.nsf, and desktop*.dsk/ndk – IF – you upgrade properly(think ini:Setup/TemplateSetup=)

• ODS 51 requires Notes >= 8.5 and CREATE_R85_DATABASES=1 in notes.ini for NEW db's

• ODS 52 requires Notes >= 9.0.1 and CREATE_R9_DATABASES=1 in notes.ini for NEW db's

Good News for Notes > 8.5

• Use NSF_UpdateODS=1 with CREATE_R85/9_DATABASES=1 for a one-time upgrade of most local databases

– NOTE: USE WITH EXTREME CARE IF YOUR DATA DIRECTORIES ARE ON A NETWORK DRIVE!

– NOTE: End users can't access databases during compact

@cadler80

#SUTOL

What makes your Notes Client START slow – ODS (Continued)

18

Impact examples

• Startup time of a Notes 8.5.3 client with 3 ODS 20 apps in Notes data = 10 seconds

• After ODS upgrade: 2 seconds

• Reduced File I/O of your disks/SAN/NAS after ODS 41 to 52 upgrade by 60%

• Removing 70% of all old files in Data directories on SAN/NAS = 45% less managed storage (backup)

The bad news: There is no way to tell what is “out there”

• 3rd party tools or random inspections are highly recommended

@cadler80

#SUTOL

IBM helps!

19

@cadler80

#SUTOL

What makes your Notes Client START fast

20

Eclipse Framework Performance Improvement

• Speeds up starting the IBM Notes client up to 65 %

• ONLY usable in virtual environments (e. g. Citrix XenApp, etc.)

• ONLY usable with a IBM Notes Roaming Solution

@cadler80

#SUTOL

What makes your Notes Client START fast

21

Example: Configuration of an IBM Notes 9.0.1 FP3 Client

1. With a non-admin user (OS + Notes) run an initial setup of the Notes client.

2. Start the Notes client 3 times then wait for 2-3minutes each

3. Shutdown the Notes client

4. Create the following directory c:\ProgramData\IBM\Notes\Data\Common

5. Copy the workspace directory from %LocalAppData%\IBM\Notes\Data toc:\ProgramData\IBM\Notes\Data\Common

6. Delete the following files/directories out of c:\ProgramData\IBM\Notes\Data\Common\workspace

a. UDM

b. Logs all but .prov2install

c. cache if available

d. .metadata\.plugins\com.ibm.collaboration.*

e. .metadata\.plugins\com.ibm.rcp.personality.framework

f. .metadata\.plugins\com.ibm.rcp.security.auth

7. Change the parameter “jvm.shareclasses.loc=“ in “c:\Program Files x86)\IBM\Notes\framework\rcp\deploy\jvm.properties”

a. from “${rcp.data}/.config/org.eclipse.osgi“

b. to “c:/temp/xpdplat”

@cadler80

#SUTOL

LIVE

@cadler80

#SUTOL

What makes your Notes Client PERFORM slow

@cadler80

#SUTOL

What makes your Notes Client PERFORM slow

24

ODS of databases (again)

Hardware (again)

Notes.ini

• EXTMGR_ADDINS = (not just) local virus scanners

• Disable Notes “plugins“ like virus scanners etc.

– they increase Network traffic by up to 200 – 250%!

Data directory on the network vs. fixed disk

• Disk performance is the #1 bottleneck (think SSD vs. normal HDD ; 7.200rpm vs. 5.400rpm network drive vs. local fixed disk = latency/bandwidth limitation)

@cadler80

#SUTOL

What makes your Notes Client PERFORM slow (Continued)

25

Port settings are included in the notes.ini

• Type / Active / Encryption / Compression / Timeout

Bad idea

• LAN0 / COMx / DisabledPorts should be removed

Good idea

• Use TCPIP according to the client type

PORTS = TCPIP

TCPIP = TCP,0,15,0,,12320, Desktops

TCPIP = TCP,0,15,0,,45088, Laptops

TCPIP = TCP,0,15,0,,12288, Citrix / VDI

Riverbed !?

Don‘t forget your Servers!

@cadler80

#SUTOL

What makes your Notes Client PERFORM slow (Continued)

26

Install types: Basic vs. Standard

• Basic client starts up fast, Standard client, well … not so fast

IBM Roaming Users

• Try replicating 80MB each time you logon to Notes before use!

Connection documents

• Pointing to outdated server names or IP addresses

• Low priority connection documents, which can cause connectivity failures

Location documents

• IP addresses for mailserver breaks Policies

• Outdated mail and home server information

• Outdated mail file path

• Outdated Catalog server

@cadler80

#SUTOL

Performance Tips and Tricks

@cadler80

#SUTOL

Make Notes faster = 1

28

Search for the file jvm.properties underneath [LotusNotesProgramDir]\framework\rcp (might be in subdirectory ...\deploy)

• Open it in a text editor

• First line to look for is: vmarg.Xmx=-Xmx256m

• Change it to e.g. vmarg.Xmx=-Xmx1024m (Maximum is ½ of physical memory; do NOT change it to more than 1024m regardless of how much memory you have)

Change vmarg.Xms=-Xms48m to ¼ of what you changed Xmx to;

e.g. if you changed vmarg.Xmx to -Xmx1024m then use vmarg.Xms=-Xms256m

NOT recommended for production / normal end users; for admins & developers only!

• But in reality, who else counts ;)

@cadler80

#SUTOL

Performance Measuring - Client Clocking

29

NOTE: All of this should only be done on “ONE” client – slows down client, creates huge log files!

Use the following to see and measure client performance

CLIENT_CLOCK=1

CONSOLE_LOG_ENABLED=1

Debug_Outfile=c:\debug.txt

Debug_Console=1

DEBUG_TCP_ALL=1 (use only if needed)

Fileserver or local disk Performance Debugging is virtually impossible

If on >= 8.5 try adding DEBUGGINGWCTENABLED=16386 in Notes

@cadler80

#SUTOL

Performance Measuring - Client Clocking (Continued)

30

Client_Clock=1 leads to a debug file of the following format:

@cadler80

#SUTOL

Performance Measuring - Client Clocking (Continued)

31

OpenNTF – NRPC Parser

Link:

http://www.openntf.org/internal/home.nsf/project.xsp?action=openDocument&name=Notes%20RPC%20Parser

Developed by:

Jamie Magee, MartinScott Consulting LLC, US

Andrew Magerman, Magerman Collaborative Software GmbH, Switzerland

@cadler80

#SUTOL

Performance Measuring - Client Clocking (Continued)

32

OpenNTF – NRPC Parser

@cadler80

#SUTOL

Performance Tuning – Understanding cache.ndk

33

We've all been “advised to”/guilty of deleting the cache.ndk in an effort to cure Notes performance problems/symptoms

• Some companies even delete cache.ndk on a scheduled basis

HOWEVER, let us explain the impact of that on Notes performance ...

• Increases traffic between client and server by 4,000% (forty times more traffic, yes)(10,000+% if the data directory is on a network drive)

• The more databases end users use, the worse it gets

• There is always a root cause – deleting cache.ndk only fixes symptoms(since 2001 I've had to delete my cache.ndk only once(!))

• Let's look at some detailed traffic analysis for just ONE application (the mail file) of ONE user ...

@cadler80

#SUTOL

Performance Tuning – Understanding cache.ndk (Continued)

34

100 users =

150 MB vs. 3,7

MB

1000 users =

1.5 GB vs. 37

MB

Excluding

Fileservers!

After deleting cache.ndk Keeping / NOT deleting cache.ndk

@cadler80

#SUTOL

Summary

@cadler80

#SUTOL

Summary - A Performance Boost for your IBM Notes Client

36

Update/clean out your ODS, databases and templates and hardware

Standardize whenever and however possible

• This will save you hugely on support time/cost

Keep your users version/fix packs current

• Each release brings performance improvements but also resource demands

Work around your inability to see clients

• Write an inventory agent, login script, get a 3rd party tool

Any challenge can be overcome with Knowledge and Tools

• All client issues are solvable

@cadler80

#SUTOL

The end is near ...

37

Spending 45 minutes on Performance Improvements can be compared toa walk on the tip of the iceberg.

@cadler80

#SUTOL

Thank you !

Headquarter, Austria:

panagenda GmbH (Ltd.)

Schreyvogelgasse 3/10

AT 1010 Vienna

Phone: +43 1 89 012 89

Fax: +43 1 89 012 89-15

E-Mail: info@panagenda.com

Headquarter, Germany:

panagenda GmbH (Ltd.)

Lahnstraße 17

DE 64646 Heppenheim

Phone: +49 6252 67 939-00

Fax: +49 6252 67 939-16

E-Mail: info@panagenda.com

USA:

panagenda Inc.

60 State Street, Suite 700

MA 02109 Boston

Phone: +1 617 855 5961

Fax: +1 415 449 5940

E-Mail: info@panagenda.com

Germany:

panagenda Consulting GmbH (Ltd.)

Donnersbergstraße 1

DE 64646 Heppenheim

Phone: +49 6252 67 939-86

Fax: +49 6252 67 939-16

E-Mail: info@panagenda.com

The Netherlands:

Trust Factory B.V.

11th Floor,

Koningin Julianplein 10

NL 2595 AA The Hague

Phone: +31 70 80 801 96

E-Mail: info@trust-factory.com

© 2007-2015 panagenda