dlux splunk>live! 2012 Beginners Session

David LutzSr. Sales Engineer

Technical WorkshopsGetting Started User Training

Copyright © 2011, Splunk Inc. Listen to your data.2

Agenda

Getting Started with splunk>

Deployment and Integration(time permitting)

Getting Started With splunk>

Copyright © 2011, Splunk Inc. Listen to your data.

Download splunk>


Install splunk>

Start Splunk• WIN: \Program Files\Splunk\bin\splunk.exe start• Other: /opt/splunk/bin/splunk start

www.splunk.com/download• 32 or 64 bit?• Indexer or Universal Forwarder?

Splunk Home•WIN: \Program Files\Splunk•Other: /opt/splunk (Applications/splunk)

http://www.splunk.com/download


splunk> Web Basics

Browser Support• Firefox 2, 3.x, 4.0 (3.5 with splunk> version 4.0.6 and higher)• Internet Explorer 6, 7, 8 and 9• Safari 3, 4, 5 (and up)• Chrome 9 (and up)• be sure to install Flash (versions prior to 4.3.x)

Default on install is http://localhost:8000

http://localhost:8000/


Take A splunk> Tutorial

splunk> Tutorials

• splunk> Home >> Getting Started App

• splunk.com >> Documentation >> User Manual >> Tutorial


key : value

Universal Indexing Indexes Unstructured Data

It’s that simple.


Every wordEvery valueEvery character … even punctuation … even white spaces

Universal Indexing Indexes Unstructured Data


splunk> Web Basics

Index some data


SearchingSearch > *Select Time Range• Historical, custom, or real-timeUsing the timeline• Click events and zoom in and out• Click and drag over events for a specific range


Searching (continued)Search for any keyword

Search > error

Use Boolean expressions

Search > error OR failed NOT application

Spaces are implied AND, Operators need to be in caps

Search > Audit Failure = Audit AND Failure

Use quotes to search for a specific string

Search > “Audit Failure”

Use wildcards

Search > 46* OR 49*


Searching (continued)

Search > error | head 1Search results are “piped” to the commandCommands for:• Manipulating fields• Formatting• Handling results• ReportingSave a search


Search AssistantContextual Help

• advanced typeahead

History• search• commands

Search Reference• short/long description• examples


Tagging and Event TypingEventtypes for more human-readable reports

• categorize and make sense of mountains of data• punctuation helps find events with similar patternsSearch > eventtype=failed_login instead ofSearch > “failed login” OR “FAILED LOGIN” OR “Authentication failure” OR “Failed to authenticate user”

Tags are labels• apply ad hoc knowledge• create logical divisions or groups• tag hosts, sources, fields, even eventtypesSearch > tag=web_servers instead ofSearch > host=“apache1.splunk.com” OR host=“apache2.splunk.com” OR host=“apache3.splunk.com”


Over 100 Commands!

splunk.com > Documentation > Search Reference

abstract accum addcoltotals addinfo addtotals af analyzefields anomalies anomalousvalue append appendcols ar associate audit autoregress bin bucket chart cluster collect common contingency convert correlate counttable crawl ctable dbinspect dedup delete

delta diff discretize erex eval eventcount eventstats excerpt extract file fillnull folderize format gentimes head highlight iconify

input inputcsv inputlookup iplocation join kmeans kv kvform loadjob localize localop lookup macro makecontinuous makemv maketable map metadata multikv mvcombine mvexpand nomv

outlier outlierfilter outputcsv outputlookup outputtext overlap rangemap rare regex relevancy rename replace reverse run savedsearch savedsplunk script scrub selfjoin sendemail set sichart sirare sistats sitimechart sitop slc stash strcat streamstats sumindex summaryindex tail test timechart top

transaction transam trendline typeahead typelearner typer uniq untable xmlkv xmlunescape xpath xyserieshttp://www.splunk.com/base/Documentation/latest/SearchReference/SearchCheatsheet


FieldsDefault fields• host, source, sourcetype, _time, _raw, etc.• View on left panel in search results or all in field picker

Where do fields come from?• Pre-defined by sourcetypes• Automatically extracted key-value pairs• User-defined


Extract FieldsInteractive Field Extractor

• generate PCRE (perl compatible regular expression)

• editable regex• preview/save

props.conf

[mysourcetype]REPORT-myclass = myFields

transforms.conf

[myFields]REGEX = ^(\w+)\sFORMAT = myFieldLabel::$1

Configuration File•manual field extraction

•delim-based extractions

Rex Search Command... | rex field=_raw "From: (?<from>.*) To: (?<to>.*)"


Saved Searches and AlertingFind Something Interesting?

OR


Alerting (continued)

Searches run on a schedule and fire an alert • Example: Run a search for “Failed password” every 15 min

over the last 15 min and alert if the number of events is greater than 10

Searches are running in real-time and fire an alert• Example: Run a search for “Failed password user=john.doe”

in a 1 minute window and alert if an event is found


Alerting Actions• Send email• RSS• Execute a script• Track in Alert Manager


ReportingBuild reports from the results of any search

Select type of report (Values over time, Top Values, Rare Values) and on which fields to report or perform statistics Choose the type of chart (line, area, column, etc) and

other formatting options


Reporting Examples

• Use wizard or reporting commands (timechart, top, etc)• Build real-time reports with real-time searches• Save reports for use on dashboards


DashboardsCreate dashboards from search results


Dashboard Examples


splunk> Appssplunk> Apps

• Apps create different contexts for your data out of sets of views, dashboards, and configurations

• Splunk Home -> Getting Started App• Splunk Home -> Find more apps• Install an App• You can create your own!

Search is an App!• Summary will show everything you have indexed • Updated in real-time• Click on any source, sourcetype, or host to look at events


splunk> Supports Diverse Apps/Solutions

Security

IronPort WSA

CDR


Searches can be managed as asynchronous processes

Jobs can be • Scheduled• Moved to background tasks• Paused, stopped, resumed, finalized• Managed• Archived

Job Management


splunk> ManagerNow Manage All of that Cool Stuff You Just Created (and more!)

• Permissions• Saved Searches/Reports• Custom Views• Distributed Splunk• Deployment Server• License Usage….


splunk> LicensesFree Download Limits Indexing to 500MB/day

• Enterprise Features expire after 60 days• Reverts to Basic Free License

Features Disabled in Free License• Multiple user accounts and role-based access controls• Distributed search• Forwarding to non-Splunk Instances• Deployment management• Scheduled saved searches and alerting• Summary indexing

Add a License


Support Through the splunk> Community

Browse and share Apps from Splunk, Partners and the

Community

splunkbase.splunk.com

Splunkbase

Community-driven knowledge

exchange and Q&A

answers.splunk.com

5 tracks, more than 40 sessions, the smartest Splunk users together

www.splunk.com/goto/conference


Where to Go for HelpDocumentation– http://www.splunk.com/base/Documentation

Technical Support – http://www.splunk.com/support

Videos– http://www.splunk.com/videos

Education– http://www.splunk.com/goto/education

Professional Services

http://www.splunk.com/base/Documentation


http://www.splunk.com/support


http://www.splunk.com/videos


http://www.splunk.com/goto/education



The IT Search Company

Your Guide David Lutz

email: [email protected]: @dlux_at_splunk

skype: dluxatsplunkdotcom

mailto:[email protected]

Deployment and Integration


A splunk> Installation Has 4 Functions Searching and Reporting (Search Head)

Indexing and Search Services (Indexer)

Data Collection and Forwarding (Forwarder)

Local and Distributed Management (Deployment Server)

35

A splunk> installation can be one or all of these …


Getting Data Into splunk>Agent and Agent-less Approach for Flexibility

perf

shellcode

Mounted File Systems\\hostname\mount

syslogTCP/UDP

WMIEvent Logs Performance

Active Directory

syslog compatible hostsand network devices

Unix, Linux and Windows hosts

Windows hosts Custom apps and scripted API connections

Local File Monitoringlog files, config files

dumps and trace files

Windows InputsEvent Logs

performance countersregistry monitoring

Active Directory monitoring

virtualhost

Windows hosts

Scripted Inputsshell scripts custom

parsers batch loading

Agent-less Data Input Splunk Forwarder


Understanding the Universal ForwarderForward data without negatively impacting production performance.

Scripts

Universal Forwarder Deployment

Logs ConfigurationsMessages Metrics

Central Deployment Management

Monitor files, changes and the system registry; capture metrics and status.

Universal Forwarder

Regular (Heavy) Forwarder

Monitor All Supported Inputs

✔ ✔

Routing, Filtering, Cloning

✔ ✔

Splunk Web ✔

Python Libraries ✔

Event Based Routing ✔

Scripted Inputs ✔


Horizontal ScalingLoad balanced search and indexing for massive, linear scale out.

Forwarder Auto Load Balancing

Distributed Search


Multiple Datacenters

Headquarters

London Hong Kong Tokyo New York

Distributed Search

Index and store locally. Distribute searches to datacenters, networks & geographies.


Data Redundancy

40

Clone data to multiple index servers to eliminate a single point of failure.

Data CloningForwarding to DR site

Active Hot Standby

---------


High Availability

41

Combine auto load balancing and cloning for HA at every Splunk tier.

Clone Group 1 : Complete Dataset

Data Cloning & Auto Load Balancing

Distributed Search Distributed Search

Clone Group 2 : Complete Dataset

Shared Storage


Service Desk

Event Console

SIEM

Send Data to Other SystemsRoute raw data in real time or send alerts based on searches.


Integrate External Data

LDAP, AD Watch Lists

CRM/ERPCMDB

Correlate IP addresses with locations, accounts with regions

Extend search with lookups to external data sources.


Integrate Users and Roles

Problem Investigation Problem Investigation Problem Investigation

Save Searches

Share Searches

LDAP, AD Users and Groups

Splunk Flexible Roles

Manage Users

Manage Indexes

Capabilities & Filters

NOT tag=PCI

App=ERP…

Map LDAP & AD groups to flexible Splunk roles. Define any search as a filter.

Integrate authentication with LDAP and Active Directory.


Centralized Licensing Management

Problem Investigation

Groups, Stacks, and Pools for Enterprise Deployments


Deployment Monitoring

46

Keep Tabs On Your Splunk Enterprise Deployment

ForwardersIndexersSourcetypesLicenses

Support and Community


Support Through the Splunk Community

48

Browse and share Apps from Splunk, Partners and the

Community

splunkbase.splunk.com

Splunkbase

Community-driven knowledge

exchange and Q&A

answers.splunk.com

5 tracks, more than 40 sessions, the smartest Splunk users together

www.splunk.com/goto/conference


Where to Go for HelpDocumentation– http://www.splunk.com/base/Documentation

Technical Support – http://www.splunk.com/support

Videos– http://www.splunk.com/videos

Education– http://www.splunk.com/goto/education

Professional Services

49






The IT Search Company

Your Guide David Lutz

email: [email protected]: @dlux_at_splunk

skype: dluxatsplunkdotcom

mailto:[email protected]

Date post:	10-Nov-2014
Category:	Technology
Upload:	david-lutz
View:	1,989 times
Download:	0 times

dlux splunk>live! 2012 Beginners Session

Technology