ProACT: A Solution for Contact Center Analytics

IBM India Research Lab

© 2004 IBM Corporation

ProACT: A Solution for Contact Center Analytics

Unstructured Information Management GroupIBM India Research Lab

Shourya Roy <[email protected]>

Behind the Scene: Raghu, Sree, Diwakar, Rahul, Shantanu, Sumeet, Venkat…

Unstructured Information Management Group2


I have been associated with IBM Research since 2002

Brief Intro

Working in the area of text analytics

Prior to that between 2000-02, I used to be seen mostly in H1 Mess, TT Room and TV Room - sometimes in CSE Dept. classrooms and rarely in my advisor Prof. Soumen Chakrabarti’s office



Contact Center : Application of Structured-Unstructured Data Integration

Contact Points

Branch office

Web

IVRCall Center

CustomerEnterprise

Products& Services

Integrate & Analyze Structured& Unstructured Data

Unstructured

Call logs & transcriptsEmails, Surveys

Self ServiceAgent

Structured

Customer/Product Transaction Data

Instant Market Intelligence Customer preferences

Dissatisfaction Drivers

Lifetime Value Management

Analyze Agent Performance

Improve C-Sat, Upsell Rate

Analyze Contact Drivers

Improve FAQs, Web pages

Structured

Agent Data

Automation of C-Sat analysis

Analytics for Agent PerformanceCustomer Preferences



I will be Talking About

Customer Satisfaction Analysis– Scenario and Importance– Solution and derived BI– Issues

Agent Performance Analysis– Analysis of telephonic transcriptions to identify scope

of improvement in a contact center Automatically Building Domain Models

– Automatically building Domain Models from noisy telephonic transcriptions

– Possible applications




Customer Satisfaction Analysis

Agent Performance Analysis

Automatically Building Domain Models



Customer Satisfaction (C-Sat)

Wikipedia says “Customer satisfaction is a business term which is used to capture the idea of measuring how satisfied an enterprise's customers are with the organization's efforts in a marketplace.”

In BPO scenario, it is crucial from client’s point of view to monitor QoS provided by Contact Center

C-Sat analysis is mostly a part of the agreement between Contact Center and client

C-Sat is different from SLAs



C-Sat Scenario

Email DB

C-Sat DB Report

1 9 9 7

Query

ResponseFeedback Request

Feedback

CustomerSpecialist

Analysts

Domain Knowledge

Immediate and helpful response



Sample Verbatims with Labels



Architecture

Integrate & Analyze Structured& Unstructured Data

TranscribeSpeech

Emails, Surveys logs

Language Skill &Cust-Sat annotators

Upsell/Product SentimentAnnotators

Business IntelligenceExplore/ View/Report

Self-ServiceSpeechWeb

Application Specific Data

Integrated Views ofAgent Performance

Summary of customerViews on products

Identification of CrossUp Sell opportunities

Agent Training, DeploymentMarket Intelligence

forEnterprise Clients



Demonstration

ProACT

BI T

ool BI











Challenges Technical Challenges

– Too many class labels(>35) and insufficient data(<10 cases for some of the categories). Examples are IncompleteResolution, CannedResponse, PolicyIssues. • Grouped into a higher level categories. Examples are Resolution,

Communication, Uncontrollables– Short, poorly written text– Noisy Data

• No fixed rule for manual labelling leading to inconsistencies• Same/similar verbatims being assigned different labels by human

labellers– Changing labels

• Labels tend to change over time

Business Challenges– Smooth transition from exisiting manual C-Sat analysis process to a

complete automated one



Summary

Accuracy

– Subjective as manual labelling is not accurate

– Ballpark accuracy figures range from 60-75%

Going forward

– Real-life deployment in different contact centers

– Insightful Business Intelligence (BI) Tool

– Can we introduce C-Sat analysis in a new process without requiring any training data?









Call Analysis to Improve Sales and Agent Performance

Scenario

– A car rental process outsourced to a call center – people calling up to rent cars

Objective

– Call centers want maximum number of car bookings as well as car pick ups

– INCREASE agent conversion rate

Approach

– Analyse transcriptions of telephonic conversation and find out the key actionable and differentiating insights

Architecture





Application of TAKMI for CRM

Customer Contact Center

Textual DataTextual Data

Calls aretranscribed

Analysis of reasonsfor agents making

Customers rent cars

Agent Language(choice ofphrases etc.) and compliance to guidelines

AAA MemberSegment

AAA MemberSegment

Characteristic ina Customer Segment

AgentAgent

Contact Center

Textual DataTextual Data

Enter Bookingsinformation

CustomerEvaluate Effects of different segments

BookedBooked

UnbookedUnbooked

Analyze Reasonsand Retry

Analyze Reasonsand Retry

Investigate for Enhancements

(to other agents and cust segments)

Investigate for Enhancements

(to other agents and cust segments)

Knowledge

Campaign

CustomerModels

CustomerModels

Application of TAKMI to Customer Relationship Management in Car Rental Process

Evaluation



Highlights : Identifying Actionable Insights

– Detect customer intent at start of call and suggest actions.• Weak start (“I would like to know the rate”, ”I just want to

get a price on midsize car”)• Strong start (“Hey, I would like to pick up a car”, “I need to

make a reservation please”)

– In weak start case, “pick up” is improved by mentioning discount phrases.

– In strong start case, “pick up” is concretized by mentioning value selling phrases and discount phrases.

– Asking for clean driving record decreases “pick up” in strong start case



Highlights : Call Flow and Compliance Mandatory questions for call process checking

– 1. Brand name in opening “welcome to Alamo”

– 2. Proper opening “My name is”

– 3. Confirms age 25 “age 25”

– 4. Confirms check/debit card in their own names “check card in your name”

– 5. Confirms clean driving record and license “you need clean driving record”

– 6. Ask for future reservations “anything else “

– 7. Brand name in conclusion “thank you for calling National”

In 137 reservation calls…Agents are not confirming “age over 25” in 36% calls.Agents are not confirming “clean driving record” in 44% calls.

In total 936 calls…Agents are not starting with brand name in 11% calls.



Difference between Strong and Weak Start

pick up information

Customer intent at start of call

Based on the customer’s start, “not picked up (NS or CC)” is predictable.

65%

35%

23% 9%

49% 8%

pick up not picked up

strong

weak



Difference between pick up and not picked up in Weak Start

10/13 * 100 = 76.9 % 9/21 * 100 = 42.9 %

Discount relating phrases are mentioned by the agent more frequently in “pick up” data.

pick up not picked up

21

13

10 9

Number of calls containing discount relating phrases



Detection of Improper Call Process (cont)

In 16 reservation calls, only less than 3 questions are mentioned.

How many mandatory questions are mentioned by the agent ?



Detection of Improper Call Process (cont)

In these 16 reservation calls, 2 questions are not mentioned at all.









Scenario Call centers handle customer complaints, issues for

computer sales to mobile phones to apparels

Typically domains have manually created domain models which contain types of problems solved in each category, solutions library, typical question-answers, appropriate call opening and closing styles etc

Each instance in a domain requires separate domain model

These models are dynamic in nature and change over time

Our objective is “automatic generation of domain models from largely available noisy transcriptions of telephonic conversations between call center agents and customers”



Example: Snippet of Automatic Transcription

SPEAKER 1: windows thanks for calling and you can

learn yes i don't mind it so then i went to

SPEAKER 2: well and ok bring the machine front

end loaded with a standard um and that's um it's

a desktop machine and i did that everything was

working wonderfully um I went ahead connected

into my my network um so i i changed my network

settings to um to my home network so i i can you

know it's showing me for my workroom um and then

it is said it had to reboot in order for changes

to take effect so i rebooted and now it's asking

me for a password which i never i never said

anything up

SPEAKER 1: ok just press the escape key i can

doesn't do anything can you pull up so that i mean



Domain Model

We define the Domain Model as a Topic Taxonomy where every node is characterized by– Topics

– Typical Question-Answers (QAs)

– Typical Actions

– Call Statistics



Block Diagram

StopwordRemoval

N-gramExtraction

Database,

archive,

replicate

Can you acc

ess ya

hoo?

Is modem on?

Call statistics

Feature Engineering

ASR

Clusterer TaxonomyBuilder

ModelBuilder

Component

Clusters of different granularity

Voice help-desk data

1

2

3 4

5



replicate error unable to find path to servergo to the workspaceon left hand side look under server icon………..

are you using lotus notes sixdo you have the lotus notes closed…………………..

avg. transcription length = 1214.540984 wordsavg. call duration = 712.7395 secs







Conclusion

Huge amount of unstructured data is being produced everyday in contact centers– Analysis can help to improve customer satisfaction, agent

productivity, call handling time

Opportunity to play with real “real-life data”– Learning experience

Importance of handling noise in unstructured data

– Workshop on Analytics for Noisy Unstructured Text Data (at IJCAI 07) [http://research.ihost.com/and2007/] – deadline 25 Sep (day after tomorrow!!)



Thanks!!



BACKUP



Bharti PoC

Demonstrate how text analytics can add value to the existing Complaint Management Systems and make it more efficient

Demonstration of the software

Possible ways to extend this work

Discussion



How Does The Data Look Like?

Comments"SAURABH AGARWAL/02/09/2005/ -

MR. SHAMBHU CALLED UP FOR

COMPLAINT !! IRREGULAR DIAL

TONE...PLZ CHK !! CONTACT NO.-

9826335426 !!THE COMMUNICATED SLA TO

SUBSCRIBER IS 03/09/2005 02:00:00

PM

AMANDEEP KAUR/03/09/2005

11:53:17 - ANIL//OLC//SPOKE TO MS-

RENA (SR NO-830201 BLACK

BETAL!NEW INST PROVIDE )TECH-RAMESH"

Ins

talla

tion

Addre

ss

EWS

24/1

5 NEA

R WATE

R TANK O

LD

SUBHASH

NAGAR B

HOPAL

MP B

PL-22



Some Relevant Text Analytics Techniques

Cleaning up of data– Spelling correction

• irregular dial, iregular dialtone,irregular dt fone,irrgular dt psl,irregular dt so plss,irregular dt due are grouped together

– Abbreviation expansion – ….

Annotators– Extracted problem areas such as Intermittent Dial Tone, Rosette

Issue etc. Hints taken from questions provided by Bharti– Address segmentation such as Subhash Nagar, Bhopal, M.P. etc.– Sentiment, Product, Services – Application specific– ….



Then?

Relevant structured and extracted annotated data is loaded into star-schema and a Business Intelligence (BI) application is developed on top of that

The BI application is capable of showing different views of the data by doing slice-and-dice, rollup-drilldown, association, comparison etc. operations

Lets see the demo of BI application developed on hard-faults data collected from M.P.



Date post:	04-Jan-2016
Category:	Documents
Upload:	hesper
View:	50 times
Download:	2 times

ProACT: A Solution for Contact Center Analytics

Documents