ManySolutions,OneGoal.

SplunkAppforStreamDavidShpritz,ApluraLLC.BaltimoreAreaUserGroup

3/21/2016

ManySolutions,OneGoal.

Agenda

• WhatisSplunkAppforStream?• WhyuseSteam?• WheretouseStream?• DeployingStream• Questions

ManySolutions,OneGoal.

WhatIsSplunkAppforStream?

ManySolutions,OneGoal.

Somehistory

• SplunkacquiresCloudmeter,December2013• RenamedSplunkAppforStream• ReleasedwithSplunk6.0(August,2014)• Nowatversion6.4.3(January,2016)

ManySolutions,OneGoal.

PurposeofStream

• Rapiddeployment• Rapidconfiguration• Capturewiredata• Interpretwiredata• Summarize/filter/aggregate• Index• KindoflikeBro,butmoreSplunky,andGUI

ManySolutions,OneGoal.

Sowhatcanwecapture?

• Well,wearen’treallycapturingandindexingpackets• Forwarderscapturepackets,analyzetheprotocols• Whatprotocols(alot):• TCP/UDP• Applicationprotocols(HTTP,databases,email,filesharing,chat)• About30differentprotocolscurrently• http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/Whattypeofdatadoesthisappcollect

ManySolutions,OneGoal.

WhytouseSplunkStream

ManySolutions,OneGoal.

Nologs

• Noownership• Novisibility• Noforwarders(asendpoints)• Nologgingoptions

ManySolutions,OneGoal.

Poorlogs

• Loggingishighoverhead• Logsmakenosense• Keyeventsarenotlogged

ManySolutions,OneGoal.

Cloud

• Manycloudservicesdon’tofferlogsonthings• Nochokepoints

ManySolutions,OneGoal.

VS.BroIDS

• LowerCPUusage• LowerRAMusage• MoreOSsupport(Linux,Windows,OSX)But• Hightrafficrequiresnetworkpacketbrokers(Gigamon,Ixia,etc.)• Can’twriteyourworkinterpreters• NoSnortrules

ManySolutions,OneGoal.

Otherfeatures

• Filtering• Aggregation• EphemeralStreams(shortterm)• SSLdecrypt• Centralizedmanagement• IntegrationwithES• StartastreamafterNotableevent• Protocolanalysisdashboards

ManySolutions,OneGoal.

DataEstimation

• “WhatifIturnthison?”• Tellsyouhowmuchdatayouwouldbeindexing

ManySolutions,OneGoal.

Granularcontrolofthedata

• Notjustwhichsystems,butalsowhatdata,whichfields

ManySolutions,OneGoal.

GlobalFilters

• Filteroutnoisefromtheenterprise• Thingslikevulnerabilityscanners

ManySolutions,OneGoal.

DistributedForwarderManagement• Setupgroupsforcapture• Usesregexforgroupsonthe“ForwarderID”• ForwarderIDisconfigurableviaXMLconfig file• Yes,it’sanotherSplunkdeployment/controlmechanism

ManySolutions,OneGoal.

WheretouseSplunkStream

ManySolutions,OneGoal.

DedicatedStreamForwarders

• SenddataoffofaswitchSpanorTap• ToolslikeGigamon,Ixia,Etc.• Youneedtheseforreallybigpipestospreadthelove

• Purposebuilt• HigherCPUandRAM• Betternetworkcards

• AlsoagoodoptionisyouwanttoperformSSLdecrypt• Notethatifyoudothisyouwillwanttochangesomeofyourkernelsettings(buffersizes)• Makesuretomonitoryourforwardersforthruput warnings!

ManySolutions,OneGoal.

ManySolutions,OneGoal.

DeploytotheEndpoints

• Deploydirectlytothesystemsyouwanttomonitor• Goodforapplicationdebugging• NiceoptionforSplunkES• CanbedonefromDeploymentServer• Granularcontrolovergroups• Couldmeanalotof“handon”

ManySolutions,OneGoal.

ManySolutions,OneGoal.

DeployingSplunkStream

ManySolutions,OneGoal.

ManySolutions,OneGoal.

Twoparts

• TheSplunkAppforStream• Dashboardsforanalyticsonprotocols• Administrativepanelsforconfiguration• StreamEstimate(reallycool,morelater)• GoesonSearchHead/Controller

• SplunkStreamAdd-on• Binaries• Index-timeoperations(linebreaking,timestamping)• GoesonIndexersandForwarders(UForHF)

ManySolutions,OneGoal.

InstalltheSplunkAppforStream

• Canco-locatewithES• Canco-locatewithDMC• Insmaller(lessthan100forwarders)don’tusewiththeDS• Possibleexhaustedconnections(DSandStreampollseparately)

• InstallsjustlikeanyotherSplunkapp

ManySolutions,OneGoal.

HarvesttheAddOn• Installstoafewplaces• $SPLUNK_HOME/etc/apps/Splunk_TA_stream• $SPLUNK_HOME/etc/apps/splunk_app_stream/install/Splunk_TA_stream• $SPLUNK_HOME/etc/deployment-apps/Splunk_TA_stream• Willcreatethelocalinputs.conf withtheappserverlocation

*SkipthisisyourSHisyourDS

ManySolutions,OneGoal.

Makesureyourforwarderscantalkback

• YourforwarderswillneedtobeabletotalktotheSHwithsplunk_app_stream installed• TheportisthesameastheGUIforyourSH

ManySolutions,OneGoal.

Configureyourforwarders

• Don’thavetoberootonLinux• Usetheincludedsetuid.sh script

• MustbelocaladminorlocalsystemonWindows• OnUFsyoushouldmonitoryourthruput limits

ManySolutions,OneGoal.

Inputs.conf

• Rememberthattheinputs.conf islayerable• JustlikeotherSplunkconfigs• Doesn’thavetobeintheSplunk_TA_stream• OntheDSyoucandeploytwoapps,onewiththeinputtopointbacktothesplunk_app_stream• ThenalsodeploytheSplunk_TA_stream

ManySolutions,OneGoal.

Configureyourstreams• Thedefaultsmaysendmorefieldsthanyouneed• Cantellforwarderswhichpartsofthedatayouwant• Youcanhavedifferentconfigs fordifferentgroups!

ManySolutions,OneGoal.

Configureyourforwardergroups

• Usesgoodol’regex• LetsyousayaheadoftimeifEphemeralStreamsshouldbeallowed

ManySolutions,OneGoal.

GotchawithGroups• JustregexontheStreamforwarderID(notIP,hostname)• ThisisconfiguredinanXMLfile• Messy• The“defaultgroup”forwardergroupforallunmatchedhostswillgatherALLTHETHINGS

ManySolutions,OneGoal.

Waitfordatatoflowin

• That’sprettymuchit!• Docsmakeitlookalotharder

ManySolutions,OneGoal.

Questions?

ManySolutions,OneGoal.

Credits• ThankstotheBaltimoreAreaSplunkUserGroup• CoverSlide:UpperSwallowFallsinOakland,MD,ChrisFlees,http://fineartamerica.com/profiles/chris-flees.html?tab=artwork&page=7

• Slide3:PotomacRiverinMaryland,TerryJ.Adams,http://www.fhwa.dot.gov/byways/byways/60807/photos

• Slide7:Timanus MillontheJonesFallsinBaltimore,“MonumentCity”,http://www.panoramio.com/photo/57148558

• Slide8:“MissingHomeworkLog”by“RedBeetleRB”.https://www.teacherspayteachers.com/Product/Missing-Homework-Log-4112• Slide9:Rotton log,NationalWildlifeFoundation,https://www.nwf.org/kids/family-fun/outdoor-activities/investigate-a-rotten-log.aspx

• Slide10:TheSimpsons,http://i.imgur.com/91sn32Q.jpg?fb

• Slide11:BroNetworkSecurityMonitor,https://www.bro.org/

• Slide17:IanAdamsPhotography,http://ianadamsphotography.com/news/galleries/bridges/• Slides19and21:SplunkConf 2015,“SplunkAppforStreamDeploymentsintheRealWorld:EnhanceOperationalIntelligenceAcrossApplication

Delivery,ITOps,SecurityandMore”,http://conf.splunk.com/session/2015/conf2015_SUdovicic_CChing_MDickey_Splunk_SplunkEntWhatsNew_StreamDeploymentsInTheReal.pdf

• Slide22:GunpowderFallsinBaltimoreCounty,MD,http://hdrcreme.com/photos/1818-gunpowder-falls• Slide23:SplunkDocs,http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/DeploymentArchitecture

• Slide34:YoughioghenyRiveratFriendsville,MDbyJoeDawson,https://www.flickr.com/photos/jmd41280/5066756138