+ All Categories
Home > Documents > Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best...

Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best...

Date post: 06-Jun-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
21
Copyright © 2016 Splunk Inc. Tedd Hellmann / David Poncelow Product Manager / Senior SoDware Engineer, Splunk STEP Up Your App Development Game
Transcript
Page 1: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Copyright©2016SplunkInc.

TeddHellmann/DavidPoncelowProductManager/SeniorSoDwareEngineer,Splunk

STEPUpYourAppDevelopmentGame

Page 2: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Disclaimer

2

DuringthecourseofthispresentaMon,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.WecauMonyouthatsuchstatementsreflectourcurrentexpectaMonsandesMmatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthosecontainedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentaMonarebeingmadeasoftheMmeanddateofitslivepresentaMon.IfreviewedaDeritslivepresentaMon,thispresentaMonmaynotcontaincurrentor

accurateinformaMon.WedonotassumeanyobligaMontoupdateanyforwardlookingstatementswemaymake.InaddiMon,anyinformaMonaboutourroadmapoutlinesourgeneralproductdirecMonandis

subjecttochangeatanyMmewithoutnoMce.ItisforinformaMonalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.SplunkundertakesnoobligaMoneithertodevelopthefeaturesorfuncMonalitydescribedortoincludeanysuchfeatureorfuncMonalityinafuturerelease.

Page 3: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

HowshouldIbuildmyapp?

Page 4: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

EVERYTHINGYOUNEEDTOBUILD

SplunkDeveloperGuidance

Page 5: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

STEPupyourgame  STEP-interacMvelearningenvironment  ExploretopicsthroughTechniquesandRecipes  Technique:explorethedetailsoffeaturesyoucanuseinapps(modularinputs,customvisualizaMons,customalertacMons,…)

  Recipe:diveintothedetailsofbringingseveraltechniquestogethertoaddressabusinessgoal

Page 6: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

STEPupyourgame

6

DataIngest

HEC

BasicDataInput

IndexerAck

ModularInput

CheckpointResults

InputValidaMon

……

VisualizaMons

Search

SimpleXML

CustomViz

...

CustomCommands

Workflows

...

Page 7: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

STEPupyourgame

7

DataIngest

HEC

BasicDataInput

IndexerAck

ModularInput

CheckpointResults

InputValidaMon

……

VisualizaMons

Search

SimpleXML

CustomViz

...

CustomCommands

Workflows

...

Page 8: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

STEPupyourgame

8

STEPPreview2Techniques1Recipe

Page 9: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Nextsteps

9

Telluswhattechniquesandrecipesyouneed!

Page 10: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

STEPintoreal-worldexamples

Page 11: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Planningajourney

Pla'ormandtools:akitbagforourjourney

UIandvisualiza2ons:whattheappslooklike

Workingwithdata:whereitcomesfrom&howwemanageit

Addingcode:usingJavaScriptandSearchProcessingLanguage

Packaginganddeployment:reachingourfirstdesMnaMon

DealingwithOAuth

Aler2ng

Buildingintelemetrywithhigh-performancedatacollec2on

splk.it/devguide

Page 12: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

1.StartwithaQuesMonsBacklog  Architecture–  WhatdoesatypicalSplunkapplicaMonreferencearchitecturelooklike?–  WhatcommonparadigmsareapplicabletoSplunkappdevelopment?–  Whatarethetypicaldeploymenttopologies?WhyshouldIchooseaspecificone?Whataretheconfoundingfactors

onthechoiceofmytopology?–  HowdoIparMMonmySplunksoluMons?–  Whatarethetradeoffsofvarioustypesofinputs?–  HowdoIarchitectmySplunksoluMonanddeploymentforaverylargescale?–  HowdoIarchitectmySplunksoluMonforthecloud?WhatarespecificconsideraMonsfordeployingtoAWSorAzure?–  What’sthelandscapeofSplunkextensionpoints?–  HowdoIintegratedatafromSplunkintoexisMngapplicaMonsandsystems?–  HowdoIplananddesignarobustalerMngandmonitoringsubsystemontopofSplunk?–  WhatshouldIconsiderformysizingrequirements?–  WhatarerecommendedconfiguraMonsofSplunkdeploymenttomeetmysizingrequirements?–  ShouldIarchitectmysoluMontoindexmydatainlocaldatacenter(zone)orcentrally?–  WhatarethingswecanautomaMcallydegradesowecanmakesureourcoreexperienceisworking?–  Whensomethinghappens,howdoIeffecMvelypropagatetheinfoandreacttoit?–  HowareothersoluMonsonSplunkbuilt?Whatwerethechallenges?Howhavetheybeenaddressed?

  PackagingandDeployment–  HowdoIpiecetogethervariouspartsofaSplunkapp(customsearchcommands,modinputsetc.)?–  HowdoIpackageaSplunksoluMonwithasingleinstallthatautomaMcallyrollsoutallthenecessarydependencies?–  HowdoImanagemySplunksoluMonversioning,backwardandfuturecompat?–  What'sthebestwaytosplitupcustomappsfordeployment?

  Development–  HowshouldIsetupmydevelopmentenvironmenttobeproducMvewithSplunk?–  WhataredifferentwaysofhowIdevelopmySplunkapp?ProsandconsofusingspecificSDKvsRESTAPIs?

ProsandconsofusingSimpleXMLvsAdvancedXMLvsWebFramework…–  HowdoIanalyzeadatasourceforaTA?–  WhatarethedifferentwaysofenrichingthedatainSplunk?Whataretheirtradeoffs?–  WhenshouldIuseeventtypesandtransacMonsfordataclassificaMon?–  HowdoIextendSplunktodefineacustominputcapability?–  WhenshouldIusemodularinputsvsscriptedinputsvs..?–  Whatarestreamingvsnon-streamingoutputsconsideraMons?–  HowdoIdealwithlong-runningscripts?Handlingshutdown/restartofSplunk?Concurrency?Statepersistenceetc.

–  WhyshouldInotusetransacMons?–  WhenshouldIusepivotvststats?–  WhyshouldIusedatamodels?–  Whenmydatasourcetouchesonmanydatamodels,shouldIassumecompleteseparaMonorheavyinheritance?–  HowdoIextendanexisMngdatamodel?–  WhatdoesCIMofferandwhyshouldIbuildCIM-compliantapps?–  InthecontextofCIM,whatarethetradeoffsofusingmyprops.confandtransforms.confandrewriMngthemon

indexing,completelydiscardingthevendorsuppliedfieldnames?HowdoIreconciletheadvantagesofacleaninterface&normalisaMon,butatthecostoflosingalignmentwithpublishedvendordocumentaMon,andalearningcurveforexisMngusers?

–  HowdoImanagemysoluMondeclaraMveconfiguraMon?HowdoIdetect/troubleshootbadconfig?–  HowdoIlogandanalyzedatathatisnoteventdriven(certainwebfeeds,htmlparsing,imagemetadata)?–  Compareandcontrastad-hocsearchingvsbackgroundsearching–  HowdoIhandletransientfaults?–  HowdoIeffecMvelymanagecredenMals?–  What’stheeffectofsearchheadlocaMononmyappandtheoveralluserexperience?–  HowdoIdevelopanintegratedmechanismtoletmeconnectSplunktomyMOM(messagingmiddleware)andindex

mymessages?–  HowdoIhandletherequirementthatappconfigsmustbedifferentacrossdifferentservertypesinadistributed

environment(e.g.appsonsearchheadsshouldn'thaveinputsenabled)?  Quality/Compliance

–  WhatqualitygatesshouldIconsider?Whatkindofpara-funcMonalcharacterisMcsareimportanttoconsider?–  WhatheurisMcsdoIusetobless/blockarelease?–  HowdoItestadatamodel?–  HowdoIprepareeventgeneraMonwhenbuilding/tesMnganapp?–  WhatkindofperftesMngshouldIdoandhow?–  HowdoItestUI?–  HowdoIsecuritycerMfymysoluMon?–  HowdoIdesigntosaMsfymyretenMonandcompliancepolicies?–  HowdoIarchitecttodesignmyavailabilityrequirements?–  HowdoIhandlegeographicdisasterrecovery/faulttolerance?–  HowdoIproperlyinstrumentmysoluMonsothatIknowwhat’shappening?

  SustainedEngineering–  HowdoImaintain/service/supportSplunkapps?–  HowdomycustomershandleupdaMngtheircustomizedconfigsoncenewversionsofmyappcomeout?

  Business–  WhyshouldIbuildonSplunk?–  WhatkindofskilldoIneedmydevstohavetobuildaSplunksoluMon?–  Whatisthecommunitybuilding?HowarecurrentdevscreaMnguniqueexperiencesusingSplunk–Itypicallywantto

seesomemarketplacesuccess –  Costandpricingareveryimportanttomeasaentrepreneurdeveloper.IfIamcomingintobuildatoolthatwillbe

commercializedIneedtoknowthatthecoststructureofSplunkwon’tcausemyservicetobeeconomicallyunprofitable.

WhatdoesatypicalSplunkapplica2onarchitecturelooklike?

HowshouldIsetupmydevenvironmenttobeproduc2vewith

Splunk?

HowdoIintegrateSplunkintoexis2ngsystems?

HowdoIpreparemyeventgenera2onwhendeveloping&

tes2nganapp?

HowdoIpackageanapp?dealwithappversioningandupdates?

12

Page 13: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

2.IdenMfyExtensibilitySurfaceArea§  Datainges2on&indexing

–  Inputê  Modularinputsê  Custom(trained)sourcetypesê  Customsourcesê  HTTPEventCollecMon

–  Datainges2onpipelineê  FieldextracMonsê  FieldtransformaMons

–  Indexingê  Customindexes

§  Searching–  Searchauthoring

ê  Customsearchcommandsê  Macros(basic,parametrized)ê  Savedsearches

–  Dataclassifica2onê  Eventtypesê  TransacMons

–  Dataenrichmentê  Lookupsê  KVstorecollecMonsê  WorkflowacMons

–  Datanormaliza2onê  Tagsê  Aliases

–  Dataminingê  cluster&dedupê  anomalousvalueê  kmeansê  predictcommands…

§  Processing&repor2ng–  Search-2memapping

ê  Datamodels

–  CIMextensions

–  CustomVisualiza2ons

–  CustomUI–  Pages,views&dashboards

ê  JS,CSSExtensionsê  Customsetupscreens

–  Scheduledprocessingê  Scheduledreports

–  Aler2ngê  Scriptedalertsê  CustomalertacMons

–  Branding&naviga2onê  CustomappnavigaMonê  Appbranding

–  Manageabilityê  Customsplunkwebcontrollersê  Customsplunkdendpoints

13

Page 14: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

3.Minebusinessrequirements4.FormulatelearningobjecMves5.Designaround3and4

14

Page 15: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

§  Data§  Searchlanguage§  AggregaMngsiloedmetricsinto

meaningfulKPIs§  DatamanipulaMon§  DatanormalizaMon§  Sub-searches§  Config-driven§  PersistencewithKVstore§  Macros

§  Viz:§  Dynamicscaling§  Customizingin-theboxviz

controls

§  Generalsearchpaserns§  SearchopMmizaMons§  UxPrototyping§  AdapMng3rdpartyvizlibrary§  CompositechartswithinteracMons§  Dealingwithhigh-volumedatasets§  TroubleshooMngperfissues§  Post-processornot-post-process–

deploymentimplicaMons§  AutomatedUItesMng(w.Selenium)

§  Setngthestage§  OverallSplunkappstructure§  UItechnologyselecMon:

SimpleXMLvsSplunkJS§  Modularity§  Dev&testenv§  Devworkflow§  Modularity§  Dataonboarding§  CIMcompliance§  Tools

§  Post-processing§  IntegraMngwith3rdparty

component§  UnittesMng(w.Mocha)§  PersisMngstate(peruser)

§  Datamodeling§  Usinglookups§  Buildingabaselinelookuptable§  WindowsofMme/CustomMmeranges§  OverlayingMmedata

§  Usingsub-searchestocorrelatedata§  TroubleshooMngsearches

§  Customnav§  UxacMviMespermeaMngalldev

§  Datamining:§  ExploraMon§  PreparaMon:filtering/deduping/

buckeMng§  UsingadvancedstaMsMcsfuncMons§  Threshold-basedanomalydetecMon§  EvaluaMnggoodness/accuracy

Plusnon-funcMonaltopics: §  Appversioning§  PackagingInstallaMon§  Securityreview

§  Deployment§  Publishingtosplunkbase§  AppcerMficaMon

Page 16: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

BuildingSoluMonsontheSplunkPlauorm1.   SplunkReferenceApps

Complete,workingreal-worldSplunksoluMonsbuilttogetherwithpartners(Conducive,Auth0)

2.   SplunkDeveloperGuide

Thisisunbelievable,itcoversmosteverythingIlearnedthehardway…–BernieMacias,TechnicalArchitect,Zillow

dev.splunk.com/goto/devguide

Page 17: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

SplunkReferenceAppDemo

SplunkReferenceApp–PluggableAuditSystemsplunkbase.splunk.com/app/1934/ORsearchnamefromSplunkWebUI

Page 18: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Takeaways  Appdevelopment!=rocketscience

  STEPupyourgamewithtechniquesandrecipes

  Getintouchwithus

–  [email protected]

–  LeavefeedbackforSTEP

–  Comebyourbooth,getsomeswag

Page 19: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Resources

19

Page 20: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

Relatedbreakoutsessions&acMviMes

20

FasterSplunkAppCer2fica2onwithSplunkAppInspect(GrigoriMelnik/AndyNortrup)BestPrac2cesforWorkingwithSplunkCloud(DennisBourg/EricSix)HTTPEventCollectorinSplunk6.4-MoreSuperPowers!(GlennBlock/ItayNeeman)

BuildingSplunkVisualiza2onswiththeNewCustomVisualiza2onAPI(MarshallAgnew)DashboardWizardry(NicholasFilippi/SiegfriedPuchbauer)

BestPrac2cesforDevelopingSplunkAppsandAdd-ons

(JasonConger)

HowtoBuildaSolu2onfromScratch:ACaseStudyofPartnerEngagementandCo-Development

(VladimirMelnik/IgalVanier)

OnboardYourDataFasterUsingtheSplunkAdd-onBuilder

(EliasHaddad/GuodongWang)

Page 21: Tedd Hellmann / David Poncelow - Splunk › files › 2016 › slides › step-up-your...Best Prac2ces for Developing Splunk Apps and Add-ons (Jason Conger) How to Build a Solu2on

THANKYOU


Recommended