Copyright © 2014 Splunk Inc.
Brian Wooden Senior Engineering Manager, Splunk
Jack Coates Senior Product Manager, Splunk
Building a Common InformaEon Model (CIM) Compliant Technical Add-‐on (TA)
Disclaimer
2
During the course of this presentaEon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauEon you that such statements reflect our current expectaEons and
esEmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements,
please review our filings with the SEC. The forward-‐looking statements made in the this presentaEon are being made as of the Eme and date of its live presentaEon. If reviewed aTer its live presentaEon, this presentaEon may not contain current or accurate informaEon. We do not assume any obligaEon to update any forward-‐looking statements we may make. In addiEon, any informaEon about our roadmap outlines our general product direcEon and is subject to change at any Eme without noEce. It is for informaEonal purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligaEon either to develop the features or funcEonality described or to
include any such feature or funcEonality in a future release.
About Us
3
Brian Wooden ! Sr Engineering Manager for
Add-‐ons and FoundaEons ! 4.5 years at Splunk ! [email protected] ! @BrianWooden on IRC or Twi[er
Jack Coates ! Sr Product Manager for
Add-‐ons and FoundaEons ! 3.5 years at Splunk ! [email protected] ! @puercomal on IRC or Twi[er
Different Phases Of Splunk Use
4
! The search bar, custom development, bespoke soluEons – Users must know data inEmately, but can produce exciEng results
! Splunk App for Product – Silo-‐bound apps provide visibility for their intended product
! Splunk App for Role – Mission-‐specific apps translate product-‐specific knowledge for users
Late Binding Schema Rewards Time Invested
5
Late Binding Schema Rewards Time Invested
6
CHANGE leads to Zeno’s Paradox… always halfway to done, never done!
7
All Data is Relevant = Big Data
Servers
Service Desk
Storage
Desktops Email Web
Call Records
Network Flows
DHCP/ DNS
Hypervisor Custom Apps
Industrial Control
Badges
Databases
Mobile Intrusion DetecEon
Firewall
Data Loss PrevenEon
AnE-‐Malware
Vulnerability Scans
AuthenEcaEon
8
All Data is Relevant = Big Data
Servers
Service Desk
Storage
Desktops Email Web
Call Records
Network Flows
DHCP/ DNS
Hypervisor Custom Apps
Industrial Control
Badges
Databases
Mobile
I don’t know how to ask four hundred
systems if something changed!
NormalizaEon: Not Just a Dirty Word
9
(tag=malware tag=a[ack acEon=allowed)
(sourcetype=SYMC “Delete failed”) OR (product=“VirusScan Enterprise” acEon=would*) OR (SourceName=“Trend Micro OfficeScan Server” “AcEon: * cannot *”)
>
! Normalizing at index Eme is pre[y lame ! Normalizing the data before it’s stored is VERY lame ! Normalizing with tags and fields at search Eme is very AWESOME
VOICE: {from TV} And That's Why Come Splunk is so Awesome
10
STRONG SAD: Oh! I see now.
The Value of NormalizaEon
Makes things easier for a Phase 1 user Phase 2 Apps can play nicely together Phase 3 Apps become far more useful
11
(why) Isn’t this Just DMTF CIM?
12
! DMTF is hierarchical, complex, and bri[le
! Splunk CIM is narrowly focused on the least common denominator
! We shiT work from the model developer to the applicaEon developer because it allows that developer greater flexibility
Datamodel Details
13
• Alerts • ApplicaEon State • AuthenEcaEon • Change Analysis • Databases • Email • Interprocess Messaging • Intrusion DetecEon/PrevenEon • Inventory
• Java Virtual Machines • Malware • Network Sessions • Network Traffic • Performance • Splunk Audit Logs • VulnerabiliEes • Web
Architecture Machines à Data à InformaEon à Users
Bits & Bytes ExtracEons & Tags
Schema & AcceleraEon
Reports & Alerts
14
RAW DATA TECHNOLOGY ADD-‐ONS
DATA MODELS SEARCH
A Deeper Dive
15
Inputs.conf
Props.conf Transforms.conf Even[ypes.conf
Tags.conf
Models.conf Constraints, evals
savedsearches.conf Indexes.conf
Outputs.conf
The constraints pick up data, and the evals make sure it
makes sense
Tags, lookups and regular expressions are why we’re here
Apps then go on to do whatever they are going to do
Data arrives in Splunk
Add Data Models at Will
16
Raw data in Splunk
Data model for X Data model for Y
Use Case X Use Case Y Use Case Z
A Data model is a extracEon or view of the data, not a hierarchical building block. Overlap is cool. Specific apps oTen include datamodels that go beyond the CIM. The CIM is a minimalist subset.
DEMO TIME!
17
Review
18
Inputs.conf
Props.conf Transforms.conf Even[ypes.conf
Tags.conf
Models.conf Constraints, evals
savedsearches.conf Indexes.conf
Outputs.conf
The constraints pick up data, and the evals make sure it
makes sense
Tags, lookups and regular expressions are why we’re here
Apps then go on to do whatever they are going to do
Data arrives in Splunk
THANK YOU Thank you!
19
