Date post: | 05-Apr-2018 |
Category: |
Documents |
Upload: | wahjoe-witjaksono |
View: | 215 times |
Download: | 0 times |
of 38
7/31/2019 10 Workshop 5
1/38
Key Performance
Indicators: Adapting an accountability tool for digitallibraries
Leslie Wolf & Lena Zentall, California Digital LibraryDLF Conference, November 3, 2010
7/31/2019 10 Workshop 5
2/38
Agenda
Why KPIs Matter
Part 1: Brief Theory
Part 2: Practical Application (+Individual Exercise)
Part 3: Your KPI Action Plan(Individual Exercise)
Part 4: LookingForward/CommunityEngagement (Discussion)
7/31/2019 10 Workshop 5
3/38
Whats The Benefit Of Learning About KPIs?
You: Leader & Innovator
Your Organization: Track & Act
7/31/2019 10 Workshop 5
4/38
Why KPIs Are Valuable
What gets measured gets done
Anything you produce of value can
be measured
Create dialog and sharedunderstanding
7/31/2019 10 Workshop 5
5/38
Why Did Our Team Decide ToUse KPIs?
Incoming University Presidentlaunched new type of accountabilityinitiative
Decision-making tool
Tells our story
7/31/2019 10 Workshop 5
6/38
By The End of This Workshop
Youll be able to explain KPIs and developthem for your own institutions
Youll start an action plan
Youll have some handouts for reference
Youll understand how other participants areusing KPIs
Youll have an opportunity to get involved ina larger community of practice
7/31/2019 10 Workshop 5
7/38
Part 1: Brief Theory
What is a Key Performance Indicator?
What is the difference between statistics,metrics, and KPIs?
What are the components of a KPI?
What does a KPI look like?
Who uses them? Where do they find them?
7/31/2019 10 Workshop 5
8/38
Body Temperature as a KeyPerformance Indicator
Widely recognized andunderstood
Measured the same way byeveryone
Easy to collect and record
Critical indicator of health
Helps with diagnosis:If fever, investigate whyIf no fever, investigate differentsymptoms
7/31/2019 10 Workshop 5
9/38
More Definitions of KPIs
Easily collected from a reliable source
Quantifiable (can be calculated)
Reflects critical success measures
Measures progress toward goals
One of many tools in the management toolkit
A snapshot over time
7/31/2019 10 Workshop 5
10/38
Statistics? Metrics? KPIs?
Statistics for dailytracking. Some statisticsmay become
Metrics to manage your service. A very fewmetrics may become
KPIs for strategicplanning.
7/31/2019 10 Workshop 5
11/38
Possible Components for aKPI
Volumetransactionspage viewssearches
Cost
Timeliness
Quality
Customer satisfaction
7/31/2019 10 Workshop 5
12/38
How to Express a KPI
Sample Equation
Total Costs Number of Searches = CostPer Search
Two Ways to Express a KPI
1. Number of searches is aweak indicator
Is that a lot? Too little?
Whats the context?2. Cost per search is a better
indicator Ratio of 2 elements that arestrongly related
Provides more contextBetter diagnostic tool moreinformation to work with
7/31/2019 10 Workshop 5
13/38
KPI Example: Cost per Search
Quarter KPI
January February March $0.06 per search
April May June $0.09 per search
July August September $0.11 per search
7/31/2019 10 Workshop 5
14/38
Diagnosing Cost per Search
Quarter KPI Diagnostic
January February March
$0.06 per search
$15,000 cost/250,000searches
April May June $0.09 per search
$15,000/175,000Cost is stable butsearches are down
July August September
$0.11 per search
$20,000/175,000Cost is up and searchesare the same level as
April
7/31/2019 10 Workshop 5
15/38
Digging Down on Cost Per Search
Why are searches down?Users dont like the service or The recent upgrade made searches moreefficient
Why are costs up?Did a big upgrade and used moredevelopers
Can we release the developers andreduce cost?
Can we renegotiate the license fee?
What else can we do to change theequation?
7/31/2019 10 Workshop 5
16/38
Who is the Audience for KPIs
Traditionally for senior management
Typically reported via a dashboard
Goal is that in a few minutes a senior manager can get a quick update onimportant services
A senior manager can self-serve: no lastminute request for data
7/31/2019 10 Workshop 5
17/38
Simple Dashboard(Spreadsheet)
7/31/2019 10 Workshop 5
18/38
Cool Dashboard: IndianapolisMuseum of Art
7/31/2019 10 Workshop 5
19/38
Cool Dashboard: IndianapolisMuseum of Art
7/31/2019 10 Workshop 5
20/38
Part 2: Lets Get Practical
Well show you how we created KPIs for UC-eLinks, which is our name for SFX (ExLibriss link resolver software)
In this part, well let you brainstorm a fewpossible KPIs for your own service.
7/31/2019 10 Workshop 5
21/38
Step 1. Prepare
Determine who should be part of thediscussion.
Have a good mix of technical, productmanagement, and customer service.Invite senior managers to join in youllget quicker buy-in and approval thatway.
Send some background information aboutKPIs in advance (since its new to mostpeople).Determine what a successful meeting willaccomplish: what is the desired outcome?
7/31/2019 10 Workshop 5
22/38
Step 2. Brainstorm AllPossible Things to Track
What data do we already track regularly?In an ideal world, what would we want totrack?
What will senior management beinterested in?What do we care about most ?What KPIs will tell our story best?
7/31/2019 10 Workshop 5
23/38
Step 3. Make Sure You CanCapture the Data
After brainstorming, start to narrow downthe list to whats realistic.
If the data is too difficult to extract, your
effort will just fall apart.
Make sure your data source is valid andyou understand the definitions of the fieldsbeing captured.
7/31/2019 10 Workshop 5
24/38
Step 4. Narrow it Down to aFew KPIs
What we chose for UC-eLinks:Cost per Transaction (when a UC-eLinkswindow appears)
% of Successful Transactions (click- thrus)User Flow
Track where users are going using UC-eLinks(targets)
Track where users are coming from using UC-eLinks (sources)
Quality of Systems
7/31/2019 10 Workshop 5
25/38
Step 5. Define the DataElements
Define whats in a transaction.
The term means something different in eachdiscipline.
Define what costs you are going to track.What can you track without too much burden?
Define what you mean by quality of systems.It can be a mind- twister; dont make it toocomplicated.
Figure out ways to track user satisfaction . Are there existing surveys or trouble ticketingsystems?
7/31/2019 10 Workshop 5
26/38
Step 6. Put It In Writing
For cost: Well use staffing, infrastructureand licensing costs. We wont use costs for the 10 campuses.
For quality of systems: Well use a weightedaverage that our developers created. Welltrack Reliability, Availability, Maintainability,and Performance (RAMP).
For user satisfaction: Well use only reported
complaints since we have no survey data.
7/31/2019 10 Workshop 5
27/38
Now Its Your Turn (Exercise)
Interactive Discussion:
In your services, what do you think isimportant to track? For example:
Cost of serviceUser satisfactionQualityVolumeUser satisfaction
Would it be difficult or easy to track?
What could stop you? What are thechallenges?
7/31/2019 10 Workshop 5
28/38
Recap
We defined what a KPI is
We understand the difference betweenstatistics, metrics and KPIs
We know the components of a KPI and whata KPI equation looks like
Weve seen a reporting dashboard
We know how senior management usesKPIs
7/31/2019 10 Workshop 5
29/38
How Do You Get Started?
Get management approval for proposedKPIs
Set up a place to track the data (e.g. Excel
worksheet)
Get agreement on how often to track (wedecided quarterly)
Set up a test period (hands-off!)
Start tracking and reporting
At the end of the test period, assess andmodify as needed
7/31/2019 10 Workshop 5
30/38
What We Learned
Our timing was good. We had the data whenit was requested (no fire drills).
When everyone is focused on the KPIs the
team is more results- oriented: What getsmeasured gets done.
We started real dialog and sometimeslively discussions about how well weredoing (especially on quality of systems,
which is somewhat subjective).
KPIs are now integral to our serviceroadmap and strategic planning.
7/31/2019 10 Workshop 5
31/38
More Lessons
We hadnt accounted for all our expenses for instance, we didnt add benefits to our staffing costs . (We decided not to go backand change the test KPIs.)
Biggest problem: No benchmarks/baseline.We didnt know what was a good cost per transaction.
Whats the cost of improvement?
Whats the sweet spot to balance cost andquality?When is it good enough (whats theopportunity cost)?
7/31/2019 10 Workshop 5
32/38
Words of Wisdom for You
Everything that can be counted does notnecessarily count; everything that countscannot necessarily be counted.
Albert Einstein
7/31/2019 10 Workshop 5
33/38
More Words of Wisdom
You cant do this from the bottom up youneed senior management support.
Numbers tell a story understand that story
before you share it , not after you are askedto explain the results.
You cant control what senior managementdoes with the numbers when you leave theroom!
KPIs will reveal hidden truths.We worried: What if the service is moreexpensive and less effective than wethought?
7/31/2019 10 Workshop 5
34/38
Questions?
7/31/2019 10 Workshop 5
35/38
Part 3: Your KPI ActionPlan (Exercise)
Take a few minutes to fill out your action plan (see the Handout)
7/31/2019 10 Workshop 5
36/38
Action Plan Questions
Whats the first thing youll do when you getback to your office?
What was the most valuable thing you
learned today?
Key Points/Things you want to explorefurther?
What barriers, if any, do you foresee?
7/31/2019 10 Workshop 5
37/38
Part 4: Looking Forward
Interactive Discussion:
Where do we go from here?
Is there value in having a community of practice to share KPIs with peer institutions? If so,
Whats the value to you?
How do we make it happen?
7/31/2019 10 Workshop 5
38/38
Thank You!
Leslie Wolf [email protected] Zentall [email protected]