. . . the Good, the Bad, and the Ugly!

Jerry ConcannonJerry ConcannonSenior AuditorSenior AuditorService Strategies CorporationService Strategies Corporationjconcannon@servicestrategies.com

Metrics for Support Metrics for Support

"What cannot be defined, cannot be

measured; what cannot be measured cannot

be improved; and what cannot be improved

will eventually deteriorate." Quality axiom

Commonly Found MetricsCommonly Found Metrics Customer Sat Event Survey* Customer Sat Periodic Survey* Response Time* Initial Callback Response Time* Resolve Time* Customer Hold Time* First Contact Resolution* Customer Abandon Rate* Call Monitoring Performance Mystery Shopper Results Calls closed within (parameter

e.g. Day, 4 hours, 8 hours etc.) Severity Level

Calls Escalated

Time to defect / escalation Percent escalated with actual

defect

Electronic Support Request Acknowledgement*

Electronic Support Response Time* Cost per Case

total labor non-labor

Staff Productivity Measures* Financial Management* Overall Satisfaction Buy Again Rating Recommend to others Accuracy of reviewed solutions Access to Solution Knowledgebase

Customer Support Agent

Recall Rate

*SCP Requirement

The Good . . . The Good . . .

Measure performance relative to all strategic goals.

Measure processes correlated to the delivery of strategic results.

Are derived from solid baseline data.

Are balanced e.g. (Balanced Scorecard).

Hold people personally accountable for results.

Strategic RelevanceStrategic Relevance

In their recent book The Strategy-Focused Organization, Kaplan and Norton* note that, according to an abundance of research data:

Only 5% of the workforce understand their company's strategy.

Only 25% of managers have incentives linked to strategy.

60% of organizations don't link budgets to strategy.

85% of executive teams spend less than one hour per month discussing strategy.

*Also authors of “The Balance Scorecard” and “Strategy Maps”.*Also authors of “The Balance Scorecard” and “Strategy Maps”.

The Support Strategy & Profit The Support Strategy & Profit ContinuumContinuum

1. Develop and 1. Develop and Communicate Communicate

Support Support Strategies that Strategies that

sustain the sustain the company’s company’s

product strategiesproduct strategies

2. Develop and 2. Develop and implement implement

Support Metrics Support Metrics that measure that measure

strategic results strategic results and processesand processes

3. Profits reward 3. Profits reward shareholders and shareholders and

employees and employees and enable continued enable continued investment and investment and

growth.growth.

Strategic Themes for Strategic Themes for SupportSupport

Strategic Themes For Support should be based on a few selected key value-creating processes.

Operations Management (e.g. producing and delivering Support Solutions, Managing Knowledge etc.).

Customer Management (Customer relevant attributes of your Service Product).

Innovation of products and processes (such as Product-Lifecycle Management, Product Enhancements etc.)

Change and Evolution : (conforming to regulations, societal expectations, platform enhancements, etc.)

CorrelationCorrelation

Excel’s CORREL( ) function returns the correlation coefficient for two ranges of data. The syntax of the function is

CORREL(range1, range2) where range1 and range2 are data sets with the same number of elements.

The correlation coefficient determines the degree of linear association between data sets.

A value of 1 means there is a perfect positive linear relationship between the data

A value of 0 indicates there is no linear relationship between the data

A value of -1 represents a perfect negative linear relationship.

Correlation Coefficient -0.777446Resolution

HoursCustomer

Sat4 34 31 42 41 51 42 52 41 5

What Is The Balanced What Is The Balanced Scorecard?Scorecard?

The balanced scorecard is a 4-pillared management system to help a business focus on: achieving financial results while at the

same time… creating future value through

strategic activities.

It translates mission and strategy into four dimensions -- customer, financial, internal processes, and innovation and learning-- and seeks measures for them.

In this book, the four dimensions of strategy are "Financial, Customer, Operations, and Innovation and Learning". The authors strongly believe that there should be a powerful connection between these four dimensions if organizations are to be successful in an environment in which stiff competition dominates. According to the authors, one of the most important causes of business failure is that some companies place an excess emphasis on financial objectives and ignore the ways to realize these objectives. This book explains how to develop a system which places an equal emphasis on four dimensions of strategy mentioned above. For managers who want to learn how to make a plan that will be functional and measurable, this book is a must.

This book explains how an organization can measure and manage performance with the Balanced Scorecard methodology. It provides extensive background on performance management and the Balanced Scorecard, and focuses on guiding a team through the step-by-step development and ongoing implementation of a Balanced Scorecard system. Corporations, public sector agencies, and not for profit organizations have all reaped success from the Balanced Scorecard. This book supplies detailed implementation advice that is readily applied to any and all of these organization types.

More than a decade ago, Robert S. Kaplan and David P. Norton introduced the Balanced Scorecard, a revolutionary performance measurement system that allowed organizations to quantify intangible assets such as people, information, and customer relationships.

Now, using their ongoing research with hundreds of Balanced Scorecard adopters across the globe, the authors have created a powerful new tool - The Strategy Map - that enables companies to describe the links between intangible assets and value creation with a clarity and precision never before possible.

Kaplan and Norton argue that the most critical aspect of strategy -implementing it in a way that ensures sustained value creation - depends on managing four key internal processes: operations, customer relationships, innovation, and regulatory and social processes.

The authors show how companies can use strategy maps to link those processes to desired outcomes; evaluate, measure, and improve the processes most critical to success; and target investments in human, informational, and organizational capital.

Providing a visual epiphany for executives everywhere who can't figure out why their strategy isn't working, Strategy Maps is a blueprint any organization can follow to align processes, people, and information technology for superior performance.

Balanced Scorecard

Cu

sto

mer

K

no

wle

dg

e

Fin

anci

al

Per

form

ance

Inte

rnal

Bu

sin

ess

P

roce

sses

Inn

ov

atio

n a

nd

L

earn

ing

The Four Measurement The Four Measurement CategoriesCategories

CUSTOMER KNOWLEDGE: How do we become our customers’ most valued supplier?

FINANCIAL PERFORMANCE: How do we look to shareholders?

INTERNAL BUSINESS PROCESSES: What processes must we excel at to achieve our objectives?

INNOVATION AND LEARNING: How can we continue to improve and create value with employees, customers, and processes?

SMARTSMART metrics metrics

SSpecific: - direct, unambiguous and targeted to the area you are measuring.

MMeasurable: - economical collection of data that is true, accurate and complete .

AActionable: - clearly delineates good results and bad results, can show a trend (so that can be pro-active).

RRelevant: - measures important things. A common downfall of support centers is to measure everything that is measurable, which produces many trivial or pointless measures.

TTimely: - is responsive to change, data available when you need it, enables timely feedback on actions taken.

One of the most popular characterizations of indicators is the one known as SMART*SMART* metric: (Ref. The Basics of Performance Measurement, J.L. Harbour, Productivity Press, 1997)

Primary Metric Primary Metric Categories:Categories:Metrics generally fall into two categories:

1. Results Metrics: typically measure what you are doing. They are typically tied to outputs, customer requirements, and strategic business requirements.

e.g. customer satisfaction, gross support profit

2. Process Metrics: typically measure how you are doing things. They tend to be internally focused and are usually measures of compliance to key strategic processes within an organization.

e.g. case escalation process adherence, cases added to knowledge base per month Common mistakes:

Primarily focusing on process measures - measuring yourself internally, rather than beginning with an external focus, namely your customer.

Primarily focusing on financial measures – measuring only things that directly impact the bottom line, rather than having a balanced set of metrics that allow you to measure the overall effectiveness and sustainability of the operation.

The Bad and the Ugly . . . The Bad and the Ugly . . .

Average Customer Satisfaction Scores

Average Time Metrics

Analysis Paralysis and/or Trivial Metrics

Tyranny of Averages of Tyranny of Averages of Customer Satisfaction Customer Satisfaction ScoresScores

Avg. %sat4.40 95% 4 5 4 4 5 5 4 5 4 5 4 5 4 4 5 5 4 3 4 53.50 40% 2 3 4 5 4 3 3 3 3 3 4 5 4 3 3 3 3 3 4 54.70 95% 5 5 5 5 5 5 4 5 5 5 5 5 4 4 5 5 3 5 4 53.50 80% 4 4 4 4 4 4 4 4 4 4 4 1 1 1 4 4 4 4 4 33.25 25% 3 3 3 3 3 3 3 3 5 4 3 2 1 2 3 4 5 4 3 2

Customer Satisfaction

Bad

Even when using a fully-anchored ordinal scale, we are averaging results from different people who have different perceptions of the interval between each point on the rating scale.WorseThis is compounded if the scale is not anchored to the extent that we are also averaging different perceptions of the definitions of each point on the ordinal scale.QuestionableWhen a respondent is asked to rate the importance of something on an ordinal scale, then to rate their satisfaction with that something on another ordinal scale and then somehow the difference between these two averages is used on yet another ordinal scale to determine the relative need for change or action.

4 & 5 are considered satisfied.

0

5

10

15

20

25

30

1 2 3 4 5

0

5

10

15

20

25

30

1 2 3 4 5

Both Average 3.0.

Measures of time . . . Measures of time . . .

Since we can’t have negative time, measures of time start at 0 and move forward.

While durations can’t go below 0, they can go quite long, unless constrained.

So the model for the distribution of time based measures has to start at 0 and move toward infinity.

Exponential Distribution Model Exponential Distribution Model

Exponential Distribution of actual duration probabilities =

18%

15%

12%

10%

8%

7%

5%

4%

4%3%

2%2%

2%1% 1% 1% 1% 1% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930

If average duration = 5.00 minutes or hours(days, weeks or any unit of time)

Exponential Distribution of Resolve Times if average is 5 hours.

Examples of time based events that Examples of time based events that typically occur according to an typically occur according to an exponential distributions . . exponential distributions . .

The time between accidents at a specific intersection

The the time from now until you have your next car accident (since the

Exponential Distribution is memory-less)

The time until you get your next phone call

The distance between roadkill

The time until there is a breakdown in an assembly line.

The lifetime of a light-bulb, toaster, refrigerator, or any other useful

object

The time until the arrival of the next customer to a given business

The time until the arrival of the next customer case to a support center.

Tyranny of Averages & Tyranny of Averages & Support MetricsSupport Metrics

For support metrics such as resolution time, hold time, and the average speed to answer example shown above, it is more accurate and actionable to set a target for support metrics and measure the percent that meets this target against the desired percent.

In many cases not only are the averages problematical but a single outlier, such as a call that isn’t answered for 10 minutes, can throw an entire day or maybe even a week of data out of line with operational trends and can even result in significant wasted management and analysis time.

ASA % in 3025.67 80% 30 12 5 6 18 15 15 18 21 47 24 13 17 46 22 73 77 28 9 1737.45 80% 30 12 5 6 18 15 15 18 21 89 24 13 17 240 22 73 77 28 9 1718.15 80% 30 12 5 6 18 9 15 12 8 35 24 13 8 32 22 32 35 28 9 1025.79 80% 27 12 5 6 18 15 15 18 21 90 24 13 17 60 22 73 31 28 9 1226.60 80% 27 12 5 6 18 15 15 18 8 90 24 5 17 46 22 73 77 28 9 17

Avg Speed of Answer

Target is 80% in 30 seconds

Analysis Paralysis and/or Analysis Paralysis and/or Trivial Metrics . . . Trivial Metrics . . .

Having too many metrics is almost as bad as not having any. Some Support organizations have a myriad of metrics that measure everything no matter how trivial, every activity, result and process in the Support Center. They generate a false sense of security and exhibit the simplistic belief that the more metrics they have the better their decision making process will be.

Avoid this temptation:

Limit the number of Metrics in the Support Operation

Organize them strategically Balanced Scorecard pillar.

Open DiscussionOpen Discussion

Thank you!