Muhammad Siddiqi
Event Source API &
Semantic Logging
@SiddiqiMuhammad
http://shujaat.net
MARQUEE SPONSOR
PLATINUM SPONSOR
PLATINUM SPONSOR
GOLD SPONSORS
SILVER SPONSORS
Ask the Logs!!!
176 [main] INFO examples.Sort - Populating an array of 2 elements in reverse order.225 [main] INFO examples.SortAlgo - Entered the sort method.262 [main] DEBUG SortAlgo.OUTER i=1 - Outer loop.276 [main] DEBUG SortAlgo.SWAP i=1 j=0 - Swapping intArray[0] = 1 and intArray[1] = 0290 [main] DEBUG SortAlgo.OUTER i=0 - Outer loop.304 [main] INFO SortAlgo.DUMP - Dump of integer array:317 [main] INFO SortAlgo.DUMP - Element [0] = 0331 [main] INFO SortAlgo.DUMP - Element [1] = 1343 [main] INFO examples.Sort - The next log statement should be an error message.346 [main] ERROR SortAlgo.DUMP - Tried to dump an uninitialized array.467 [main] INFO examples.Sort - Exiting main method.
When should we deploy?
When should we schedule maintenance?
What is the effect of our recent email advertising campaign?Should we support IE or Windows XP
anymore?
Why does application response gets slower between 6 – 11 PM?
Is the application flow as we expect? (I see a lot of items getting added to the cart but no orders, what is going on?)
We need…
Meaningful structured logging
Minimum impact on performance
Session Agenda?
-What is Semantic??
- Event Source API
- Semantic Logging Application Block
- Event Source & ETW Tools
- Event Source & Transfer Events
- Event Source & Windows Event Logs
Semantic [Webster]“Of or relating to meaning, especially meaning in language.”
-------Log What you mean!!! ------ What, Where & How??
Semantic Web { Tim Berners Lee – Scientific American 2001 }[ converting the unstructured / semi-structured data on web to structured one ]
[ a web of data that can be processed by machines ]
--- Creating logs which can be processed by machines ---
Other Names In the Industry- Structured Logging- Strongly typed events
Semantic Logging is to create logs with machine’s consumption in mind
Log4Net Exam
pleLog File
EventSource APIProvides the ability to create events for event tracing for Windows (ETW).
.net framework 4.5
EventSourceProvides the ability to create events for event tracing for Windows (ETW).
Helps with…
• Allows to separate the event authoring request from writing the event. + Simpler Logging site.
• Make it easier to work with ETW.
• Takes care of • generating the manifest for your events• converting your method parameters to event data.• XCOPY deployment now possible.
automatic
Performance – ETW based Structured Payload Cross-application
Benefits
EventSource & EventListener
Logging Call site
RFC4122
Listener ConsumerController
In-Proc Usage
Controller (Attachment with EventSource)
Consumer [ OnEventWritten() ]
EventSource’s [Event] MethodsLevel Keyword
Task & Opcodes Versioning
Max level of events to enable
VIWECL Very Intelligent & Wise Elfs Can Learn
Power of 2
Task oriented grouping of events
Task
OpcodesUsed in conjunction
Integer Value
Operation within a taskTask Opcodes
Versions cannot co-exist…duplicate Event Ids not allowed
Demo EventSource with attributes
Event Source Proxy
ETW : Event Tracing for Windows
Event Provider
Event Payload
.net framework ETW
Event Type
Method Parameters
Event Method
Event Source
Event Controller
Event Consumer
Event Listener
EventSource & ETW Mapping
Semantic Logging Application Block [SLAB]
- Provides- Customized Listener- Sinks- Unit Testing
Destination specific
Console
Flat File
Rolling Flat File
Database
Windows Azure Table Storage
- Can be consumed- In-Proc- Out-Proc
Semantic Logging Service
SLAB is about making your log talk!!! EventSource : Authoring LogsSLAB: Example for Logs Consumption
SLAB & Rx
ObservableEventListener : IObservable<EventEntry>
ConsoleLog
FlatFileLog
RollingFlatFileLog
SqlDatabaseLog
WindowsAzureTableLog
XLog
LogToX (extension methods
- creates subscription)
ConsoleSink
FlatFileSink
RollingFlatFileSink
SqlDatabaseSink
WindowsAzureTableSink
XSink
IObs
erve
r<T>
SLAB Nuget Packages
Semantic Logging Service Separate Download
Unit Testing Event Source
Why? EventSource.EventMethod[Parameters]
ETW Event Type
EventSourceAnalyzer
EventSourceInspect
• Error when the given EventSource is enabled by an EventListener?• Can event schema be requested from the EventSource?• Can all Event based methods be invoked in the specified EventSource
class?
Checks Performed
Event Text Formatting
- Flat File Sink- Rolling Flat File Sink- Console Sink
Supported Sinks
+ Custom Event Text Formatter
Semantic Logging Service
Can run as Console Application
Windows Service
Out-Proc Event Listener
Must run on the same machine as source
Xml based configurationsDynamically updated
No restart required
Available as a separate download
Increased fault tolerance for Application crashes
Can monitor Event Providers from multiple processes
SLAB Customizations
Custom Text formatters
Custom Sinks
Rx In-Proc filtering
Console color and level mapping
Application Buffer1
ETW Sessions
Semantic Logging Service
S3
S3 Sink
SLAB Custom Sinks
Demo Using Semantic Logging Service
with Windows Azure Table Storage Sink &
Sql Database Sink
ETW Tools
ETW Manifests
Installed[Static Registration]
Inline [EventSource][Dynamic Registration] Manifest based Provider
Classic Provider
TraceEvent Package
EventSource & ETW Tools
ETW Tools must incorporate support using Trace Event API
WPR
XPerf
PerfView
…..…..…..…..
ETL FilePerf Monitor
LogMan
WPA TraceRpt
Performance Monitor [Perfmon.exe]
PerfMonitor 2.01Not a next version of PerfMon but a different utility.Also called command line version of PerfViewBased on Trace Event Library
PerfMonitor listSources MyApp.exe PerfMonitor monitor MyApp.exe <args>
Single file deployment
Perfmonitor listSessions
PerfView
Based on Trace Event Library
Can run using Command Line
Shows Event Data on User Interface
Vance Morrison’s for WPA support
Windows Performance Analyzer
WPR
XPerf
PerfView
…..…..…..…..
ETL File
Collectors
Log Manager [LogMan]
TraceRpt
- Processing binary ETL files into human readable formats including XML / CSV
- Xml (default) or Html based report for generated events data
- Command line utility
Activity Id for Events [Transfer Event]Grouping related events from more than one component
Helps in understanding End-to-End scenarios
Supported in both Manifest based and Classic ETW provider
EventSource
Additional WriteEvent like methods
EventWrittenEventArgs EventListener.OnEventWritten()
.net framework 4.5.1
Correlation between calls and tiers. Can span applications
Semantic Logging Service
Application 1
Application 2
Application 3
ETW Infrastructure
Event Store
ActivityId = X, RelatedActivityId = A
ActivityId = X, RelatedActivityId = C
ActivityId = X, RelatedActivityId = B
EventSource & Windows Event Log
EventSource doesn’t support Windows Event Log by default
No Channel support!
Microsoft.Diagnostics.Tracing.EventSource (Standalone New definition)Provides
Compile time checks
Supports .net framework 4.0
Naming Event Source – Hyphenated Friendly names Consult IT
Consult IT[Channel]
Step I : Implementing EventSource
Step II : Compile the Project
Step III : Registering Provider
Using the Package…
Unified Enterprise Logging
Questions ???
http://shujaat.net
@SiddiqiMuhammad