IBM Connect 2016 - Logging Wars: A Cross Product Tech Clash Between Experts - 1470A

MakeEvery

MomentCount

2016ConnectThe Premier Social Business and Digital Experience Conference

#ibmconnect

Logging Wars: A Cross-Product Tech Clash Between ExpertsBen MenesiChris Miller


Ma

ke Every Mom

ent Coun

t

Acknowledgements and DisclaimersCopyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, BrassRing®, Connections™, Domino®, Global Business Services®, Global Technology Services®, SmartCloud®, Social Business®, Kenexa®, Notes®, PartnerWorld®, Prove It!®, PureSystems®, Sametime®, Verse™, Watson™, WebSphere®, Worklight®, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.


Ma

ke Every Mom

ent Coun

t

Speaker• Ben Menesi

§ Head of Product at Ytria§ IBM Notes Administration & Development§ Certified Domino Admin, Dev and Adv.

Security Professional v7+ § Speaker at ICS / ESS tech conferences

around the globe § IBM Champion ‘14 – ‘16


Ma

ke Every Mom

ent Coun

t

Speaker• Chris Miller

§ Director, Collaboration at Connectria§ @IdoNotes on everything§ IBM Champion ’11 – ‘16


Ma

ke Every Mom

ent Coun

t

Questions to ask ourselves

• Does Domino always display what is logged?

• Does Domino always log what it displays?


Ma

ke Every Mom

ent Coun

t

Agenda

• Mail Logging • User Activity Logging• Replication Logging• Web Server Logging• DDM vs. Other methods• Logging Agents


Ma

ke Every Mom

ent Coun

t

Mail Logging• Daily logging is different than troubleshooting logging

§ The goal is reduce logged data and utilize built in tools

• MTC can be your friend across a domain for daily logging issues


Ma

ke Every Mom

ent Coun

t

Mail Rule issues • Mail rules are frequent troublemakers for Administrators

§ Let’s understand their architecture a bit better

§ Tip: Create and save a new Mail Rule document to resync all rules§ More info: http://techlab.ytria.com/5243/ibm-notes-secrets/mail-

rules-architecture-sort-scanez/

MailRule Conditions Actions $FilterFormula

CalendarProfile $FilterFormula_x


Ma

ke Every Mom

ent Coun

t

Mail Tools• Mail File Digest - summary report for mailfiles and mail-in

databases from OpenNTF§ http://idonot.es/mailfiledigesttool

• Downside: Server add-on, requires downtime to get done, plus only supports 801, 852 and 9


Ma

ke Every Mom

ent Coun

t

User Activity• Who’s accessing what?

§ Increasingly important question whether you’re:• Upgrading / consolidating your environment

• Considering going to the cloud

• In the process of migrating

§ What data to use and HOW to use it• Frequently used databases and apps that aren’t used at all?


Ma

ke Every Mom

ent Coun

t

User Activity• Possible Data Sources

Activity Trends

Database User Activity

Log.nsfUser Activity

3rd Party Solutions

§ Valuable high level reports

§ Not very customizable§ Lack of customization

leads to inaccuracies

§ Very detailed logs § Really crucial info

buried inside one text field

§ Unless through 3rd

party tools: NONO

§ Great amount of detail § Only available per

database§ To be discussed later

§ Amazing products out there

§ Some installed on server side

§ Some client side§ To be discussed later


Ma

ke Every Mom

ent Coun

t

User Activity• Activity Trends

§ Offers high level reports on various aspects of application usage

• Inactive databases and users

• % of space used, growth data

§ Cons: • Considers EVERYTHING

– Sometimes you want to be able to dig deeper. And filter who you want to look at.


Ma

ke Every Mom

ent Coun

t

User Activity• Log.nsf Usage Session Activity

§ What do we want to know? • ForEach(database) what did our users do? (read / written/ bytes etc..)

§ What do we have? • ForEach(session) Σread, Σwritten, Σbytes read etc…)

§ See the issue?


Ma

ke Every Mom

ent Coun

t

User Activity• Recorded User Activity

§ User Activity can be a *VERY* rich source of data § Tip: No_Force_Activity_Logging=0 (Default value) allows on ALL

dbs§ Tip: Set confidential to prevent users with lower than Designer

access


Ma

ke Every Mom

ent Coun

t

User Activity• Prior to ODS 48

§ Entry size allocated 44 bytes, object size 61600 bytes (max 1400)

• ODS 48+ (undocumented improvement) § Entry size 92 bytes. Object size 128800 bytes (same max 1400)

§ Curious: only two additional counters added, yet ODS48+ entry size is over 2x the size of ODS48 – why?

Date & Time Reads Writes User

Date & Time Reads Writes Updates Deletes User


Ma

ke Every Mom

ent Coun

t

User Activity• Does Notes only capture what it displays?

§ Nope

§ Read more: http://techlab.ytria.com/5448/lotus-notes-articles/interpret-user-activity-traces-portrait-global-usage/

Reads Adds Updates Deletes

Data

NonData

Σ

Data

NonData

Σ

Data

NonData

Σ

Data

NonData

Σ


Ma

ke Every Mom

ent Coun

t

User Activity • Cons

§ Only available per database • You have free tools available to facilitate working with this data

across multiple databases: http://www.agecom.com.au/useractivity– Or implement via W32_NSFDbGetUserActivity

– Or get Ytria’s databaseEZ

§ Makes no difference between users and servers • Filtering this info is easy in tools like databaseEZ

§ Only maintains 1400 entries• Funny you should say that…


Ma

ke Every Mom

ent Coun

t

User Activity• 3rd Party products for usage analysis

§ Domino Doublecheck• http://goo.gl/maL0fo

§ Panagenda iDNA• https://www.panagenda.com/products/idna

§ Ytria databaseEZ & consoleEZ• http://ytria.com/databaseez

• http://ytria.com/consoleEZ


Ma

ke Every Mom

ent Coun

t

Replication Logging• Replication is a subject that raises various questions such as

§ How do I make sure replication runs smoothly? § What do I do when problems occur (and problems do occur)


Ma

ke Every Mom

ent Coun

t

Replication Logging• Various Replication diagnostic tools are available such as the

Log_Replication notes.ini parameter. § The log file and log_replication is pretty useless§ What if we want to know what exactly happens during replication?

• Set Log_Replication=3 to get information about each replicated note:


Ma

ke Every Mom

ent Coun

t

Web Server Logging• Filtering and excluding


Ma

ke Every Mom

ent Coun

t

Web Server Logging• Consolidating HTTP & Miscellaneous Logs

§ Use the HTTPLogUnauthorized=1 Notes.ini Parameter • Logs 401 HTTP Errors in console AND Misc. log events

• HTTP Users attempting to access resources they aren’t entitled to access + user authentication requests.


Ma

ke Every Mom

ent Coun

t

DDM vs. Other methods• DDM is about building proper triggers for logged and non-

logged data§ DDM takes effort to configure properly§ Domino exposes all errors but does not technically log them


Ma

ke Every Mom

ent Coun

t

DDM vs. Other methods• How to extract more from Log.nsf?

§ There is MUCH more data in your logs than you know§ Extract more data from your log documents using formulas such

as: • # of events, unique events

• Highest severity event encountered

• # of various severity events

• Total # of messages transferred per each Mail Routing document

§ More info: http://www.slideshare.net/BenedekMenesi/ibm-connected-2015


Ma

ke Every Mom

ent Coun

t

DDM and Stat Tools• Andy Pedisich statrep.nsf customization

§ http://www.andypedisich.com/blogs/andysblog.nsf/dx/resources.htm


Ma

ke Every Mom

ent Coun

t

Agent Logging• Debugging Agents through Print statements is BAD

§ How can we log and control our agents better?

• OpenLog on OpenNTF§ https://www.openntf.org/main.nsf/project.xsp?r=project/OpenLog

• Thomas Lindberg Agent Controller (OpenNTF)§ https://www.openntf.org/main.nsf/project.xsp?r=project/AgentCon

troller

• AgentEZ§ http://www.ytria.com/agentEZ


Ma

ke Every Mom

ent Coun

t

Thank you• Thank you for attending!

§ Feel free to get in touch: • Ben Menesi (@BenMenesi): [email protected]

• Chris Miller (@Idonotes): [email protected]

Date post:	15-Apr-2017
Category:	Technology
Upload:	chris-miller
View:	2,169 times
Download:	1 times