Date post: | 16-Apr-2017 |
Category: |
Technology |
Upload: | keith-brooks |
View: | 1,613 times |
Download: | 0 times |
DDM, Letting Admins Sleep Later and Stay at Pubs Longer
Since 2005
Keith BrooksRent This Space
Thanks to our sponsors!
7th Sutol Conference, November 2015
7th Sutol Conference, November 2015
Keith Brooks
• Twitter: @lotusevangelist• Skype: lotusevangelist• Email: [email protected]• Blog: http://blog.vanessabrooks.com• Linkedin: http://
linkedin.com/in/keithbrooks• Slides: http://Slideshare.net/kbmsg
3
4
7th Sutol Conference, November 2015
5
7th Sutol Conference, November 2015
Today’s Topics:History of DDMProbes and Listeners inside of DDMConsole Commands for DDMMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
6
7th Sutol Conference, November 2015
Why Domino Domain Monitoring (DDM)
• At the time there was no tool that can provide a single view of the overall status of multiple servers across one or more DOMINO Domains
• Administrators can spend too much time to find out the cause when a server problem occurs
• The analysis and fix of the problems depends on the experience and skill of the Administrator
7
7th Sutol Conference, November 2015Background of DDM
• Started with R7 although its history goes back to Stats and Events
• DDM (ddm.nsf) is a reporting database• It reports based on details that the Monitoring
Configuration (events4.nsf) database has defined• Events (events4.nsf) is the real information
wrapped around by the ddm.nsf• It is installed with every copy of Domino yet many
never use it … why?• If you use it, you probably set it up once and
never looked at it again ... right?
8
7th Sutol Conference, November 2015Sound Familiar?
Your server is crying out in pain and you don’t hear it, or worse, ignore it … at your peril.
Daily, hourly it suffers in silence because you do not let it talk to you.
You know it can talk, right? You turn DDM on and it talks….non-
stop! So you shut DDM off again.
9
7th Sutol Conference, November 2015
DDM — Making Everyone Accountable Inside DDM you can:
See if everything is okay or not See who closed which items and issues Delegate problems to someone else Provide management with a dashboard Change severity of error messages Monitor ACLs, security, SMTP, and on and on as
long as it’s a Domino item* * There are ways to do this for non-Domino too, sort of, hang on, will get to that soon.
10
7th Sutol Conference, November 2015Best Practices
Tip #57: Domino Domain monitor keeps you safe Enable DDM Have DDM tell you when anonymous gets
added to a database over a certain level Have DDM tell you not only that there is a
problem, but suggests how to fix it! Have DDM bring you directly to the problem Monitoring software makes your life easier
(From LS10 Paul Mooney’s Session)
11
7th Sutol Conference, November 2015
History of DDMProbes and Listeners inside DDMConsole Commands for DDMMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
12
7th Sutol Conference, November 2015Default DDM Probes
Three flavors of probes (some probes have more than one):• Scheduled probes - run according to a
configurable schedule – defaults are supplied
• Embedded probes - watch the feature area and catch problems/issues as they occur
• Listening probes - run when particular error codes are logged
13
7th Sutol Conference, November 2015More Key Benefits
• Active monitoring capability: Probes with highly-configurable schedules, content and targets
• Data aggregation with collection hierarchies• Automatic problem determination and
determination of probable cause in multiple feature areas, and provide possible solution.
• Delegation of problems• Automation of corrective actions• Auto-closing for resolved problems
14
7th Sutol Conference, November 2015
Configuration: Enable default DDM ProbesTypes expand into subtypes ...
15
7th Sutol Conference, November 2015Common Probes
• Monitor for any errors encountered when compacting database on Domino servers. (Database/Database Compact)
• Monitor the configured database(s) to ensure that replication occurs on the target servers within the configured time interval. (Replication/Scheduled Replication Checks)
• Monitor which documents are not replicating in the Domino Directory. (Replication/Replication Errors)
16
7th Sutol Conference, November 2015Common Probes
• Monitor if name lookups are taking too long. (Directory/Name Lookup/Search Response)
• Monitor CPU (55%: fatal) /Disk (30%: fatal) /Memory (98%) Utilization on your Domino Servers. (Operating System/CPU; Memory)
• Monitor the availability of all directories on the Domino servers. (Directory/Directory Availability)
• Monitor and fix a corrupt database by the corrective action code.
17
7th Sutol Conference, November 2015Application Probes
• Monitor whether an agent (or Web agents) run by Agent Manager took longer than X minutes to complete. (Application Code/Long running agents)
• Monitor database(s) for unused space, generate an event when the percent of 30% unused space exceeds, then automatically compact the database(s) if an event is generated. (Database/Scheduled Checks)
• Monitor a specific Notes DB with no access by any user for one month.
18
7th Sutol Conference, November 2015
• Monitor mail routing to Domino and/or non-Domino mail systems.(Messaging/Mail Reflector)
• Monitors the amount of mail on a Domino server and whether the number of mail messages in the MAIL.BOX file exceeds the number of mail messages that the router can process. (Messaging/Mail Flow Statistic)
• Monitor whether your SMTP servers are listening on the appropriate ports. (Messaging/SMTP process state)
Monitor What?
19
7th Sutol Conference, November 2015
• Monitor whether NRPC mail routing is occurring on the Mail Servers. (Messaging/NRPC Routing Status)
• Monitor which AdminP requests are being generated, and whether they are being processed. (Server/Administration)
• Monitor any ACL changes for a specific database.
Monitor What?
20
7th Sutol Conference, November 2015
Last Minute Developer/PM Info• Why is there no task probe for Traveler, for that matter why
not for Sametime? – Traveler and Sametime are not with Domino out of the
box and when DDM was created it was focused on only out of the box tasks. The Traveler task can be monitored by the Administrator client
• Can we get DDM to Tweet?– DDM only creates events, so it can directly tweet.
However, event can have event handlers act upon them including run a program. So in theory you could setup an event handler to do this, but DDM doesn't do this out of the box.
21
7th Sutol Conference, November 2015
Last Minute Developer/PM Info #2• Can DDM probe for HTTP when it is not port 80
and if so, how do we tell it to use port 8088 as an example? – DDM uses what's configured for HTTP. So either they
both use 8080 or 80. it would be looking in the server doc for the configuration.
• Can DDM probe that a website is up and running? Seems we can open a db, but not a web app. Is this true? – The HTTP probe can just probe the port or GET a URL,
but it will only check for the successful fetching of that URL, the HTTP return code.
22
7th Sutol Conference, November 2015
• I have not seen how I configure DDM to verify an HTTP website, it seems we can only verify if HTTP is running as a service. – Correct, the DDM HTTP only probes the
service, but the Event Generator -> TCP Server probe has the web page option
Last Minute Developer/PM Info #3
23
7th Sutol Conference, November 2015
Someone asked about DifferencesSee Ben Menesi at Ytria for more details
24
7th Sutol Conference, November 2015
History of DDMProbes and Listeners inside DDMConsole Commands for DDMMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
25
7th Sutol Conference, November 2015Console Commands
• View DDM probe schedule– show sch –ddm– Shows a table of enabled probes and next
scheduled run time Probe• View enabled probes
– tell event dumpprobes– Shows a list of enabled DDM probes by note ID
• Run probes manually– tell event runprobe ?????– Run a probe with note ID ????? immediately
(designed to be used with the dumpprobes command)
26
7th Sutol Conference, November 2015
History of DDMProbes and Listeners inside DDMConsole Commands for DDMTime Out for Crazy MessagesMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
Attack of the Ambiguous Error Messages
• Cannot process Full Text Index (FTI)• Admin Process: Gone Fishing/Org does not appear in
design elements of any database designating Server/Demo as their Administration Server
• SMTP Failed to Connect to us.ibm.com• Gone Fishing/Org attempted to login to Server/Demo• SSL Error: Keyring File access error• Unable to replicate or route mail• Calendar profile error• Events database missing documents or duplicated• Unable to compact compact no such file exists
(program doc added compact twice?)
27
Attack of the Ambiguous Error Message — Troubleshooting
• So now what? The great DOMINO has spoken, you must resolve the problem, but how? DDM does not always provide any details Check your server console
The answer may be staring you in the face, if not, search your log.nsf file for the text of the error
Check that you are running enough debug level in your log to show these errors
Dig deeper within the system from the Administrator monitoring and analysis tools to the new DCT
Google, Planetlotus.org, Bloggers, IBM Support 28
29
7th Sutol Conference, November 2015
History of DDMProbes and Listeners inside DDMConsole Commands for DDMMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
30
7th Sutol Conference, November 2015
DDM Making Admins Lives Easier
…. While Teaching Exchange
Admins What a Real Server Can Do
7th Sutol Conference, November 20154 Steps
1. Open events4.nsf2. Open DDM Configuration->DDM
Probes Section-> By Type3. Click on New DDM Probe ->
Messaging4. Add configuration details and Save
7th Sutol Conference, November 2015Steps 3 & 4 Details
Click on New DDM Probe -> Messaging– Probe Subtype = Transfer Queue Check– Probe Description = Whatever you Want it to Say– Which Servers to run on? Special Target
Servers-> SMTP Servers– Destination? Check All Destinations or Specify
that special one – Services NRPC and/or SMTP– Set Event Severity (not too low i.e group emails)– Save
7th Sutol Conference, November 2015
Over? Nothing is Over Until…
The Probe will be enabled on SaveSo how do you get notified?Glad you asked….
7th Sutol Conference, November 2015Notification
1. DDM Configuration->Event Handlers Section-> By Author
2. Click on New Event Handler from the Db menu3. Event Tab (can be any type)
– Events must be one of these severities: FATAL! And/or Failure, Send them all of them
– Events can have ANY message4. Action Tab
– Method = Mail– Mailing Address = Domino Email or Internet
Email of the MS ADMINS!– Enablement = Enable this notification
Not you! Send it to the MS Admins
7th Sutol Conference, November 2015
Keep in Mind
1. If Exchange is down, how would they get this? That’s why we use personal emails
2. Include the CIO/CEO or whomever needs to know Exchange fails
3. Set up a Mail Rule in your mail file to collect them all and publish a monthly outage report
36
7th Sutol Conference, November 2015
History of DDMProbes and Listeners inside DDMConsole Commands for DDMMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
37
7th Sutol Conference, November 2015
38
7th Sutol Conference, November 2015
We want Twitter updates!We are not Developers (Well I’m Not)This works for me https://uptimerobot.com/
Monitors ports, websites, etc..Notifies you via email, Twitter, SMSThey have their own APIsIntegrate into:
Slack and HipChat and PushOver
One Possible Solution
39
7th Sutol Conference, November 2015
Hi,The monitor VBI-SMTP (http://www.vanessabrooks.com) is currently DOWN (Port Is Not Listening).
Uptime Robot will alert you when it is back up.
Sincerely,
Uptime Robothttp://uptimerobot.com, http://twitter.com/uptimerobot, http://facebook.com/uptimerobot
The monitor C_Q (http://quickr.clientsite.com) is back UP (HTTP 200 - OK) (It was down for 44 minutes and 6 seconds).
40
7th Sutol Conference, November 2015
History of DDMProbes and Listeners inside DDMConsole Commands for DDMMonitoring Non-Domino ItemsDDM Can Tweet Now!References and Review
41
7th Sutol Conference, November 2015
DDM is free and running ... use itYou decide what to monitor and you should
be monitoring!Be Proactive, not just ReactiveEdit common issues that are unique to your
organization so the next admin has it easierCareful not to flood your inbox with
warningsManagement can have a dashboard with
colors, give it to them!Ambiguous error messages are a thing of
the past
7 Key Points to Take Home
42
7th Sutol Conference, November 2015
• Planetlotus.org – 315+ Lotus Bloggers especially, Andy Pedisich who gives a much more detailed session on DDM
• DDM: If You Only Knew What Your Server Was Saying by Keith Brooks• IBM Wiki Domino 9.0.1
http://www-01.ibm.com/support/knowledgecenter/SSKTMJ_9.0.1/admin/admn_dominodomainmonitoring_c.dita
Philip Monson, Thomas Gumz, Frank Nostrame, Leah Busque, “Lotus Domino Domain Monitoring” (IBM Redbook, 2005).
http://www.redbooks.ibm.com/abstracts/redp4089.html
Resources
43
7th Sutol Conference, November 2015
• This workshop introduces you to DDM ...Lotus Education on Demand: Domino Domain Monitoring http://www.ibm.com/support/docview.wss?uid=swg27007060
• Notes/Domino Best Practices: Domino Domain Monitoring• http://
www.ibm.com/support/docview.wss?uid=swg27009312• IBM Open Mic on DDM – For Administrators pdf• Add Server Console to a Notes Client http://
www.preemptive.com.au/node/44
More Resources