Notes Browser Clients, Client Upgrades und beste Startzeiten!DNUG – Dortmund, Juni 2015

2

3

4

Christoph AdlerTechnical Account Manager – panagenda

IBM (Lotus) Notes / Domino since 2001

Consulting for projects Administration Migration / Consolidation Client Management Application Management

Since 2012 TAM at panagenda with core competence in Notes Client Management ICS Infrastructure Analysis and Optimizations

Good to know… Frequent traveler Projects with many different companies in

different countries Beer or Wine? Beer!

5

ChallengeUpgrading to IBM Notes Browser Plugin-in

6

Challenge: Upgrading to IBM Notes Browser Plugin-in

General• Different OS versions in 32Bit &| 64Bit• Different Directories: Program &| Data folders• Single &| MultiUser installations (again Directories, Templates in Data Directory, etc.)• Desktop / Laptop / Citrix or other virtual environments• Hardware• Network drives• Harddisk capacity• (Intelligent) package deployment• Design updates + ODS-Upgrade• Notes.ini

7

Challenge: Upgrading to IBM Notes Browser Plug-in - continued

IBM Notes Browser Plugin-in specific• Usage of “InstallShield Tuner for Lotus Notes” not supported & not available

• stub notes.ini• additional files• modified shortcuts• […]

• Activate Browser Plug-in in Mozilla Firefox / Internet Explorer

8

Challenge: Upgrading to IBM Notes Browser Plug-in - continued

Current configuration• IBM Notes 9.0.1 Standard + FP2• MultiUser

Target configuration• IBM Notes 9.0.1 Browser Plug-in only + FP3 IF4• MultiUser installation• + pre defined stub notes.ini• + pre activated browser plug-in (addon) Mozilla Firefox

Upgrade process• < 4 minutes (complete uninstall + cleanup + install + configuration)

9

Live

10

What makes your initial IBM Notes Client START fast

11

What makes your initial IBM Notes Client START fast

Speeds up starting the IBM Notes client up to 65 %

MOSTLY useful in virtual environments (e. g. Citrix XenApp, vmware, Windows Terminal Server, VDI, etc.)

ONLY usable with an (IBM Notes) Roaming Solution

Eclipse Framework Performance Improvement

12

What makes your Notes Client START fast - Continued

Example: Configuration of an IBM Notes 9.0.1 FP3 Client1. With a non-admin user (OS + Notes) run an initial setup of the Notes client.2. Start the Notes client 3 times then wait for 2-3minutes each3. Shutdown the Notes client4. Create the following directory c:\ProgramData\IBM\Notes\Data\Common5. Copy the workspace directory from %LocalAppData%\IBM\Notes\Data to

c:\ProgramData\IBM\Notes\Data\Common6. Delete the following files/directories out of c:\ProgramData\IBM\Notes\Data\Common\workspace

a) UDMb) Logs all but .prov2installc) cache if availabled) .metadata\.plugins\com.ibm.collaboration.*e) .metadata\.plugins\com.ibm.rcp.personality.frameworkf) .metadata\.plugins\com.ibm.rcp.security.auth

7. Change the parameter “jvm.shareclasses.loc=“ in c:\Program Files (x86)\IBM\Notes\framework\rcp\deploy\jvm.propertiesa) from “${rcp.data}/.config/org.eclipse.osgi“b) to “c:/temp“

Eclipse Framework Performance Improvement

13

Live

14

IBM Notes Client Performance boost

15

Before we start ...

I love Notes.

16

Your IBM Notes Client: Version 9.0.1 FP3

> 20.000 41.000 files (Notes 6/7 | 8/9 Basic used to have ~550) = +7.400% 17.000 21.800 files in Notes Program\framework in 1.800 2.150 subdirectories

7.500 8.800 (largely undocumented) .properties files 1.500 1.600 jar files 1.200 1.300 HTML files

3.000 1.182 files in Notes Data\workspace in 2.400subdirectories, out of which 1,850 are empty 700 subdirectories 900 180 (largely undocumented) XML files 100+ 80 (largely undocumented) .properties files

Which is 20.000 41.000 files in 4.600 4.860 subdirectories 8.500+ 9.000+ largely undocumented files Many (many many) subdirectories are plain empty, especially in Data\workspace

... and around 100 NSFs/NTFs (=0.5 0.2 %) ... and the good old notes.ini file

17

Your IBM Notes Client: Version 9.0.1 FP3 - Continued

Add over 80 preference/dialog boxes• Each dialog has ~15 settings at average

For a total of 1,200+ settings sprayed across• address book (very few)• notes.ini (few)• and XML files (lots)

Plus any plugins, widgets,add-ons, etc. “you“ or the usermay have added to the Notesclient install

18

Your IBM Notes Client: Version 9.0.1 FP3 - Continued

Search

Sidebar

Shortcut

ButtonsToolbars

Bookmarkbar

19

Your IBM Notes Client: Version 9.0.1 FP3 - Continued

Workspace

(nsf)

Physical nsfs 1, 2, 3, … n

20

What makes your Notes Client START slow

If you‘re not on at least 9.0.1 yet 9.0.1 is a significant improvement Pre-loading the client might also be recommended

EOM of IBM Lotus products http://www-01.ibm.com/software/lotus/support/lifecycle/

21

What makes your Notes Client START slow - Continued

~50% of the entire data volume in the Data directory changes on a DAILY basis in most setups 5 - 10% in Data\workspace ( up to 300

files) DISABLE Antivirus scanning for at least

*.ns**.ndk*.xml*.properties

in Data Directory.

22

What makes your Notes Client START slow - Continued

Client crashes/database fixups especially bad for network based data directories clients

Outdated Hardware You need >1-2 GB memory if you want to use Notes AND something else Old fixed disks with fragmentation

Fun Exercise: Buy a small Solid State Disk and make Notes boot off of that ;-)

Tons of startup programs/scripts launching during the “boot storm” if the disk is unable to keep up, it‘s not Notes' fault Again, see http://bit.ly/ACzO6Z

23

What makes your Notes Client START slow - Continued

Outdated Data directories Virtually 100% of all NTFs (Laptops: 90%) can be removed by

Cleaning up old Notes 4+5+6 templates using ini:SharedDataDirectory= (also with Single User install!)

Operating System Windows 7 boots ~40% faster than XP Windows 8 ~55% faster than XP

~20% faster than Windows 7, supposedly

24

What makes your Notes Client START slow - ODS

ODS = On Disk Structure ODS 16 = Notes 2 ODS 17 = Notes 3 ODS 20 = Notes 4 (oder Templates) ODS 41 = Notes 5 ODS 43 = Notes 6 & 7 ODS 48 = Notes 8 ODS 51 = Notes 8.5 & 9 ODS 52 = Notes 9.0.1

The difference between ODS 43 and 51 = up to 80% LESS FILE I/OThis helps with ANY client AND server hardware

25

What makes your Notes Client START slow – ODS - Continued

CAUTION: The ODS is not automatically updated when upgrading clients Except for names.nsf, bookmark.nsf, and desktop*.dsk/ndk – IF – you upgrade properly

(think ini:Setup/TemplateSetup=) ODS 48 requires Notes >= 8 and CREATE_R8_DATABASES=1 in notes.ini for NEW db's ODS 52 requires Notes >= 9.0.1 and CREATE_R9_DATABASES=1 in notes.ini for NEW db's

Good News for Notes > 8.5 Use NSF_UpdateODS=1 with CREATE_R85/9_DATABASES=1 for a one-time upgrade of most local

databases NOTE: USE WITH EXTREME CARE IF YOUR DATA DIRECTORIES ARE ON A NETWORK DRIVE! NOTE: End users can't access databases during compact

26

What makes your Notes Client START slow – ODS - Continued

Impact examples Startup time of a Notes 8.5.2 client with 3 ODS 20 apps in Notes data = 10 seconds After ODS upgrade: 2 seconds Reduced File I/O of your disks/SAN/NAS after ODS 41 to 51 upgrade by 60% Removing 70% of all old files in Data directories on SAN/NAS = 45% less managed storage (backup)

The bad news: There is no way to tell what is “out there” 3rd party tools or random inspections are highly recommended

27

IBM helps!

28

What makes your Notes Client PERFORM slow

ODS of databases (again)Hardware (again)Notes.ini EXTMGR_ADDINS = (not just) local virus scanners Disable Notes “plugins“ like virus scanners etc.

they increase Network traffic by up to 200 – 250%!

Data directory on the network vs.. fixed disk Disk performance is the #1 bottleneck (think SSD vs. normal HDD ; 7.200rpm vs. 5.400rpm ;

network drive vs. local fixed disk = latency/bandwidth limitation)

29

What makes your Notes Client PERFORM slow – Port settings

Port settings are included in the notes.ini Type / Active / Encryption / Compression / Timeout

Bad idea LAN0 / COMx / DisabledPorts should be removed

Good idea Use TCPIP according to the client type

PORTS = TCPIP TCPIP = TCP,0,15,0,,12320, Desktops TCPIP = TCP,0,15,0,,45088, Laptops TCPIP = TCP,0,15,0,,12288, Citrix / VDI Riverbed !?

Don‘t forget your Servers!

30

What makes your Notes Client PERFORM slow - Continued

Install types: Basic vs. Standard Basic client starts up fast, Standard client, well … not so fastRoaming Users Try replicating 80MB each time you logon to Notes before use!Connection documents Pointing to outdated server names or IP addresses Low priority connection documents, which can cause connectivity failuresLocation documents IP addresses for mailserver breaks Policies Outdated mail and home server information Outdated mail file path Outdated Catalog server In general: catalog.nsf see next slide

31

What makes your Notes Client PERFORM slow – The catalog.nsf problem

User in Hamburg

Sends Doc|DB-Link to user in Frankfurt

Unfortunately, the user in Frankfurt does not open the database from a local serverbut from the one in Hamburg

... or even in Augsburg (Alphatbeth!)

32

Make Notes faster = 1

Search for the file jvm.properties underneath [LotusNotesProgramDir]\framework\rcp (mightbe in subdirectory ...\deploy) Open it in a text editor First line to look for is: vmarg.Xmx=-Xmx256m Change it to e.g. vmarg.Xmx=-Xmx1024m (Maximum is ½ of physical memory; do NOT change it to

more than 1024m regardless of how much memory you have)

Change vmarg.Xms=-Xms48m to ¼ of what you changed Xmx to;e.g. if you changed vmarg.Xmx to -Xmx1024m then use vmarg.Xms=-Xms256m

NOT recommended for production / normal end users; for admins & developers only! But in reality, who else counts ;)

33

Performance Measuring - Client Clocking

NOTE: All of this should only be done on “ONE” client – slows down client, creates huge log

files!

Use the following to see and measure client performanceCLIENT_CLOCK=1CONSOLE_LOG_ENABLED=1Debug_Outfile=c:\debug.txtDebug_Console=1DEBUG_TCP_ALL=1 (use only if needed)

Fileserver or local disk Performance Debugging is virtually impossibleIf on >8.5 try adding DEBUGGINGWCTENABLED=16386 in Notes >= 8.5

34

Performance Measuring - Client Clocking - ContinuedClient_Clock=1 leads to a debug file of the following format:

35

Performance Measuring - Client Clocking - Continued

Link:http://

www.openntf.org/internal/home.nsf/project.xsp?action=openDocument&name=Notes%20RPC%20Parser

Developed by:Jamie Magee, MartinScott Consulting LLC, USAndrew Magerman, Magerman Collaborative Software GmbH, Switzerland

OpenNTF – NRPC Parser

36

Performance Measuring - Network Latency

Network Latency has a major impact on perceived client performance for users that (try to)

work on server-side databases

For example• Number of NRPC calls x

(Latency Up + Latency Down) =how long it takes for a client to“talk something through”with the server

• 30 Calls for opening the mailfile x(60 ms download + 60 ms upload)= 3.600 ms to talk through what'snecessary to open the mailfile

37

Performance Tuning – Understanding cache.ndk

We've all been “advised to”/guilty of deleting the cache.ndk in an effort to cure Notesperformance problems/symptoms Some companies even delete cache.ndk on a scheduled basis

HOWEVER, let us explain the impact of that on Notes performance ... Increases traffic between client and server by 4,000% (forty times more traffic, yes)

(10,000+% if the data directory is on a network drive) The more databases end users use, the worse it gets There is always a root cause – deleting cache.ndk only fixes symptoms

(since 1992 Florian Vogler had to delete his cache.ndk only once(!)) Let's look at some detailed traffic analysis for just ONE application (the mail file) of ONE user ...

38

Performance Tuning – Understanding cache.ndk - Continued

100 users =150 MB vs. 3,7 MB

1000 users =1.5 GB vs. 37 MB

Excluding Fileservers!

After deleting cache.ndk Keeping / NOT deleting cache.ndk

39

Summary: A Performance Boost for your IBM Notes Client

Update/clean out your ODS, databases and templates and hardware

Standardize whenever and however possible This will save you hugely on support time/cost

Keep your users version/fix packs current Each release brings performance improvements but also resource demands

Work around your inability to see clients Write an inventory agent, login script, get a 3rd party tool

Any challenge can be overcome with Knowledge and Tools All client issues are solvable Be careful with thinking that removing the client/going to the cloud solves everything

40

The end is near ...

Spending 25 minutes on Performance Improvements can be compared toa walk on the tip of the iceberg.

41

panagenda MarvelClient helps!

• Client analysis and optimization• Reduce helpdesk calls by up to 80% • Achieve client startup times < 45 seconds • Roaming, including Citrix, WTS, VDI and

Connections Cloud• Seamless client setups, upgrades and updates• Seamless cloud onboarding

" panagenda MarvelClient significantlyreduces our cost to manage ourIBM Notes client base, including Citrixand the Notes browser plugin."

MarvelClient Platforms:IBM Notes Basic Client, IBM Notes Standard Client, IBM Notes Browser Plug-in, Microsoft Windows, Linux, Mac OS, Citrix, Microsoft Windows Terminal Server

42

Q & A

43

Thank you !

44

Christoph AdlerTechnical Account Manager

panagenda GmbH – Solutions for Infrastructure Analysis and Optimization

● Donnersbergstr. 1 ● 64646 Heppenheim (Germany)● Cell: +49 172 494 4141 ● Phone: +49 6252 679 39-52 ● Fax: +49 6252 679 39-16● E-Mail: christoph.adler@panagenda.com

Contact