33
Copyright © 2014 Splunk Inc. Ian Thomas Problem Management Analyst, Paychex Enterprise Support Crowdsourcing the Matrix: Improving the Service Desk Experience and ITIL/ SDLC Processes
Copyright © 2014 Splunk Inc.
Ian Thomas Problem Management Analyst, Paychex Enterprise Support
Crowdsourcing the Matrix: Improving the Service Desk Experience and ITIL/SDLC Processes
Disclaimer
2
During the course of this presentaLon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauLon you that such statements reflect our current expectaLons and
esLmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements,
please review our filings with the SEC. The forward-‐looking statements made in the this presentaLon are being made as of the Lme and date of its live presentaLon. If reviewed aSer its live presentaLon, this presentaLon may not contain current or accurate informaLon. We do not assume any obligaLon to update any forward-‐looking statements we may make. In addiLon, any informaLon about our roadmap outlines our general product direcLon and is subject to change at any Lme without noLce. It is for informaLonal purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligaLon either to develop the features or funcLonality described or to
include any such feature or funcLonality in a future release.
Agenda ! About Me ! About Paychex ! Typical SDLC/ITIL Workflows Before and ASer Splunk ! SupporLng the Real World ! Crowdsourcing the Matrix ! Improving Everything ! What’s next? ! Q & A
3
About Me ! Senior Problem Management Analyst at Paychex ! ITIL FoundaLons CerLfied ! Lean Six Sigma Green Belt Pending ! 8 years with Paychex ! Over 13 years of industry experience ! Fan of sharing informaLon efficiently to reduce spin ! Splunk Fan
4
About Paychex
5
Typical SDLC/ ITIL Workflows – Before and ASer Splunk
Before Splunk – SDLC/ITIL InformaLon Flow
Defect Tracking/ Code Repository
RFC
Test Engineering Logs Developer
IT Support
Bug
ITSM
Developer
Defect
Bug Ticket
Logs
Pre-‐Prod Test
Prod Support
Monitoring
=
= + + +
Logs
+ + +
AlerLng System
= Script
+ Runbook
KCS Knowledgebase
ArLcle
+ + IT OC Alert
7
Problems/Risks
8
! Each group could be starLng from scratch ! Same issue gets recorded separately in each repository ! Knowledgebase searches are not 100% accurate ! PotenLal for mulLple Defects, RFCs, and Knowledgebase arLcles generated for same issue within each silo
ASer Splunk – SDLC/ITIL InformaLon Flow
Defect Tracking/ Code Repository
RFC
Developer Test Engineering
IT Support
Bug
ITSM
Developer
Defect
Bug Ticket
Pre-‐Prod Test
Prod Support
Monitoring
=
= + + +
+ + +
AlerLng System
= Script
+ Alert Runbook
KCS Knowledgebase
ArLcle
+ + IT OC
+
+
+ New Splunk
Charts/Graphs
New Splunk Charts/Graphs
New Splunk Based Alerts
9
ASer Splunk – SDLC/ITIL InformaLon Flow
Defect Tracking/ Code Repository
RFC
Logs Developer Test Engineering
IT Support
Bug
ITSM
Developer
Defect
Bug Ticket
Logs
Pre-‐Prod Test
Prod Support
Monitoring
=
= + + +
Logs
+ + +
AlerLng System
= Script
+ Alert Runbook
KCS Knowledgebase ArLcle
+ + IT OC
+
+
+
10
SupporLng the Real World
Lost Guest DirecLons to MGM Grand
12
Service Desk Lost Guest How can I help you?
Hi, I am lost. Can you help me get back to the MGM Grand?
Where are you? I don’t know, I see a Starbucks
Can you see any landmarks? I see a McDonalds…
Wait, I see another Starbucks Are you driving or walking?
Driving What road are you on?
Fremont Street What, you can’t drive there???!!!
I can and I am! What are you driving?
A Rascal Scooter
21 Starbucks 10 McDonalds
How is This Like Online ApplicaLon Support?
13
QuesLons commonly asked when supporLng online clients… • What plajorm/browser are you using? • What applicaLon are you in? • What are you trying to do? • Where are you, what are you seeing, and what is the issue?
IT Support
Developers
+ +
+
RFC
= ArLcle
Produc@on Support Process
Client Support Process Clients
Execu@ves
What app… What Issue?
Help!
Client Facing Service Desk
Client Facing Business Support
What app… What Issue?
Help!
KCS Knowledgebase
What do I do?
What do I do?
Incident/ Bug Ticket
No SoluLon Found
14
When Experts View Logs
15
GPS Tracker Logs – How Could They Help?
16
Lost Guest IT Expert Hi, I got lost. Can you help me get back to the MGM Grand?
Sure, I see where you are... (and where you have been)
Follow Koval down to Harmon take a right… etc.
Lost Guest Service Desk Hi, I got lost. Can you help me get back to the MGM Grand?
Sure, One second while I check your locaHon in our new tool…
(What does this say? This doesn’t help me!!!) Ok… Where are you now?...
Crowdsourcing the Matrix
360 View
18
William Service Desk Rep
I want to help the client now.
I want to ask as few questions as
possible to provide the
client with the right answer the
first time.
Miriam Test Engineer
If I encounter a bug, I want to know if we have seen it
before.
Tom IT Support Desk
I want to know
what to do when an issue is
escalated to me.
I want the Service Desk to resolve issues without
escalating to me.
Dave Developer
If we have
information about a logged event I want to know it.
I want to stop
known issues from being escalated
to me, from IT Support, Test
Engineering, and the Operations
Center.
Client Who we Serve
I want answers and help now.
I don’t want to be
asked lots of questions.
Paul OperaHons Center
Analyst When I get an alert, I want to
know to respond: Is
issue directly impacting our
clients and how?
Which Logged Events Do We Care About?
19
Client IT
All Events
INFO
ERROR
WARN
All Events
More Answers Fewer
Ques@ons
Fewer Annoyances
Known Event Records – EventKNow Approach ! What is a Known Event Record (KER)?
– Event class of interest idenLfied – Logical buckeLng schema created
ê Find fields you can combine into event ‘signature’ ê Mask event specific uniqueness
– Buckets are automaHcally assigned a Known Event Record Number
! When someone encounters an event with a Known Event Record perLnent to their issue, they update it with what they know in a simple interface
! Everyone using Splunk/EventKNow sees the informaLon in the Known Event Record and adds to it as more is learned
20
All Events
What does adding Known Event Records do?
21
All Events
KERs – Divide Event Space into Buckets
EventKNow – Known Event Record Viewer/Editor
22
EventKNow Access Points – Make it Easy Open Searches -‐ IT/Dev
and Splunk Ninjas Trending Dashboards: IT OC, Dev, Problem, KERs graphed by -‐ KER-‐ App -‐ RFC(APR at Paychex) and Status…
User Plajorm/ Browser Info
Username Search Client Facing Service Desks/Test/Support
23
Improving Everything
Before EventKNow -‐ SDLC/ITIL InformaLon Flow
Defect Tracking/ Code Repository
RFC
Developer Test Engineering
IT Support
Bug
ITSM
Developer
Defect
Bug Ticket
Pre-‐Prod Test
Prod Support
Monitoring
=
= + + +
+ + +
AlerLng System
= Script
+ Alert Runbook
KCS Knowledgebase
ArLcle
+ + IT OC
+
+
+ New Splunk
Charts/Graphs
New Splunk Charts/Graphs
New Splunk Based Alerts
25
Make Everyone Behave Like an IT Expert
26
Even
tKNow
ASer EventKNow – SDLC/ITIL InformaLon Flow
Defect Tracking/ Code Repository
RFC
Developer Test Engineering
IT Support
Bug
ITSM
Developer
Defect
Bug Ticket
Pre-‐Prod Test
Prod Support
Monitoring
=
= + + +
+ + +
AlerLng System
= Script
+ Alert Runbook
KCS Knowledgebase
ArLcle
+ + IT OC
+
+
+ EventKNow
Charts/Graphs
EventKNow Charts/Graphs
EventKNow Based Alerts
+
27
Client Support Process – ASer EventKNow
IT Support
Developers
+ + +
RFC
= ArLcle
Produc@on Support Process
Clients
Execu@ves
I know what you need…
Help!
Client Facing Service Desk
Client Facing Business Support
I know what you need…
Help!
KCS Knowledge
base
I know what to do…
I know what to do…
NEW Incident/ Bug Ticket
EventKNow U
sername/Even
t Search
28
Demo Of EventKNow Interfaces
29
! Support Interface ! Known Event Record Viewer/Editor ! Dashboard ! Open Searches
EventKNow
Execu@ves
KCS Knowledge
Base Self Service
Improve Business Processes
In App -‐ Real Time Event Specific Feedback
IntegraLon with Internal Support Tools
ProacLve CommunicaLon
In App Support Requests
People
EventKNow as Connector – Future OpportuniLes
30
What’s Next?
31
30 17 32 37 41 40 26 53 31 26 32 26
Special Offer: Try Splunk MINT Express for Free! Splunk MINT offers a fast path to mobile intelligence. How fast?
Find out with a 6-‐month trial*
• Register for your free trial: hsp://mint.splunk.com/conf2014offer
• Download the Splunk MINT SDKs • Add the Splunk MINT line of SDK code and publish**
• Start getng digital intelligence at your fingerLps!
*Offer valid for .conf2014 aTendees and coworkers of aTendees only.
**Trial allows monitoring of up to 750,000 monthly acHve users (MAUs).
32
