Post on 04-Jan-2016
description
transcript
IBM India Research Lab
© 2004 IBM Corporation
ProACT: A Solution for Contact Center Analytics
Unstructured Information Management GroupIBM India Research Lab
Shourya Roy <rshourya@in.ibm.com>
Behind the Scene: Raghu, Sree, Diwakar, Rahul, Shantanu, Sumeet, Venkat…
Unstructured Information Management Group2
IBM India Research Lab
I have been associated with IBM Research since 2002
Brief Intro
Working in the area of text analytics
Prior to that between 2000-02, I used to be seen mostly in H1 Mess, TT Room and TV Room - sometimes in CSE Dept. classrooms and rarely in my advisor Prof. Soumen Chakrabarti’s office
Unstructured Information Management Group3
IBM India Research Lab
Contact Center : Application of Structured-Unstructured Data Integration
Contact Points
Branch office
Web
IVRCall Center
CustomerEnterprise
Products& Services
Integrate & Analyze Structured& Unstructured Data
Unstructured
Call logs & transcriptsEmails, Surveys
Self ServiceAgent
Structured
Customer/Product Transaction Data
Instant Market Intelligence Customer preferences
Dissatisfaction Drivers
Lifetime Value Management
Analyze Agent Performance
Improve C-Sat, Upsell Rate
Analyze Contact Drivers
Improve FAQs, Web pages
Structured
Agent Data
Automation of C-Sat analysis
Analytics for Agent PerformanceCustomer Preferences
Unstructured Information Management Group4
IBM India Research Lab
I will be Talking About
Customer Satisfaction Analysis– Scenario and Importance– Solution and derived BI– Issues
Agent Performance Analysis– Analysis of telephonic transcriptions to identify scope
of improvement in a contact center Automatically Building Domain Models
– Automatically building Domain Models from noisy telephonic transcriptions
– Possible applications
Unstructured Information Management Group5
IBM India Research Lab
I will be Talking About
Customer Satisfaction Analysis
Agent Performance Analysis
Automatically Building Domain Models
Unstructured Information Management Group6
IBM India Research Lab
Customer Satisfaction (C-Sat)
Wikipedia says “Customer satisfaction is a business term which is used to capture the idea of measuring how satisfied an enterprise's customers are with the organization's efforts in a marketplace.”
In BPO scenario, it is crucial from client’s point of view to monitor QoS provided by Contact Center
C-Sat analysis is mostly a part of the agreement between Contact Center and client
C-Sat is different from SLAs
Unstructured Information Management Group7
IBM India Research Lab
C-Sat Scenario
Email DB
C-Sat DB Report
1 9 9 7
Query
ResponseFeedback Request
Feedback
CustomerSpecialist
Analysts
Domain Knowledge
Immediate and helpful response
Unstructured Information Management Group9
IBM India Research Lab
Sample Verbatims with Labels
Unstructured Information Management Group12
IBM India Research Lab
Architecture
Integrate & Analyze Structured& Unstructured Data
TranscribeSpeech
Emails, Surveys logs
Language Skill &Cust-Sat annotators
Upsell/Product SentimentAnnotators
Business IntelligenceExplore/ View/Report
Self-ServiceSpeechWeb
Application Specific Data
Integrated Views ofAgent Performance
Summary of customerViews on products
Identification of CrossUp Sell opportunities
Agent Training, DeploymentMarket Intelligence
forEnterprise Clients
Unstructured Information Management Group13
IBM India Research Lab
Demonstration
ProACT
BI T
ool BI
Unstructured Information Management Group14
IBM India Research Lab
Unstructured Information Management Group15
IBM India Research Lab
Unstructured Information Management Group16
IBM India Research Lab
Unstructured Information Management Group17
IBM India Research Lab
Unstructured Information Management Group18
IBM India Research Lab
Challenges Technical Challenges
– Too many class labels(>35) and insufficient data(<10 cases for some of the categories). Examples are IncompleteResolution, CannedResponse, PolicyIssues. • Grouped into a higher level categories. Examples are Resolution,
Communication, Uncontrollables– Short, poorly written text– Noisy Data
• No fixed rule for manual labelling leading to inconsistencies• Same/similar verbatims being assigned different labels by human
labellers– Changing labels
• Labels tend to change over time
Business Challenges– Smooth transition from exisiting manual C-Sat analysis process to a
complete automated one
Unstructured Information Management Group19
IBM India Research Lab
Summary
Accuracy
– Subjective as manual labelling is not accurate
– Ballpark accuracy figures range from 60-75%
Going forward
– Real-life deployment in different contact centers
– Insightful Business Intelligence (BI) Tool
– Can we introduce C-Sat analysis in a new process without requiring any training data?
Unstructured Information Management Group20
IBM India Research Lab
I will be Talking About
Customer Satisfaction Analysis
Agent Performance Analysis
Automatically Building Domain Models
Unstructured Information Management Group21
IBM India Research Lab
Call Analysis to Improve Sales and Agent Performance
Scenario
– A car rental process outsourced to a call center – people calling up to rent cars
Objective
– Call centers want maximum number of car bookings as well as car pick ups
– INCREASE agent conversion rate
Approach
– Analyse transcriptions of telephonic conversation and find out the key actionable and differentiating insights
Architecture
Unstructured Information Management Group22
IBM India Research Lab
Unstructured Information Management Group23
IBM India Research Lab
Application of TAKMI for CRM
Customer Contact Center
Textual DataTextual Data
Calls aretranscribed
Analysis of reasonsfor agents making
Customers rent cars
Agent Language(choice ofphrases etc.) and compliance to guidelines
AAA MemberSegment
AAA MemberSegment
Characteristic ina Customer Segment
AgentAgent
Contact Center
Textual DataTextual Data
Enter Bookingsinformation
CustomerEvaluate Effects of different segments
BookedBooked
UnbookedUnbooked
Analyze Reasonsand Retry
Analyze Reasonsand Retry
Investigate for Enhancements
(to other agents and cust segments)
Investigate for Enhancements
(to other agents and cust segments)
Knowledge
Campaign
CustomerModels
CustomerModels
Application of TAKMI to Customer Relationship Management in Car Rental Process
Evaluation
Unstructured Information Management Group24
IBM India Research Lab
Highlights : Identifying Actionable Insights
– Detect customer intent at start of call and suggest actions.• Weak start (“I would like to know the rate”, ”I just want to
get a price on midsize car”)• Strong start (“Hey, I would like to pick up a car”, “I need to
make a reservation please”)
– In weak start case, “pick up” is improved by mentioning discount phrases.
– In strong start case, “pick up” is concretized by mentioning value selling phrases and discount phrases.
– Asking for clean driving record decreases “pick up” in strong start case
Unstructured Information Management Group25
IBM India Research Lab
Highlights : Call Flow and Compliance Mandatory questions for call process checking
– 1. Brand name in opening “welcome to Alamo”
– 2. Proper opening “My name is”
– 3. Confirms age 25 “age 25”
– 4. Confirms check/debit card in their own names “check card in your name”
– 5. Confirms clean driving record and license “you need clean driving record”
– 6. Ask for future reservations “anything else “
– 7. Brand name in conclusion “thank you for calling National”
In 137 reservation calls…Agents are not confirming “age over 25” in 36% calls.Agents are not confirming “clean driving record” in 44% calls.
In total 936 calls…Agents are not starting with brand name in 11% calls.
Unstructured Information Management Group26
IBM India Research Lab
Difference between Strong and Weak Start
pick up information
Customer intent at start of call
Based on the customer’s start, “not picked up (NS or CC)” is predictable.
65%
35%
23% 9%
49% 8%
pick up not picked up
strong
weak
Unstructured Information Management Group27
IBM India Research Lab
Difference between pick up and not picked up in Weak Start
10/13 * 100 = 76.9 % 9/21 * 100 = 42.9 %
Discount relating phrases are mentioned by the agent more frequently in “pick up” data.
pick up not picked up
21
13
10 9
Number of calls containing discount relating phrases
Unstructured Information Management Group28
IBM India Research Lab
Detection of Improper Call Process (cont)
In 16 reservation calls, only less than 3 questions are mentioned.
How many mandatory questions are mentioned by the agent ?
Unstructured Information Management Group29
IBM India Research Lab
Detection of Improper Call Process (cont)
In these 16 reservation calls, 2 questions are not mentioned at all.
Unstructured Information Management Group30
IBM India Research Lab
I will be Talking About
Customer Satisfaction Analysis
Agent Performance Analysis
Automatically Building Domain Models
Unstructured Information Management Group32
IBM India Research Lab
Scenario Call centers handle customer complaints, issues for
computer sales to mobile phones to apparels
Typically domains have manually created domain models which contain types of problems solved in each category, solutions library, typical question-answers, appropriate call opening and closing styles etc
Each instance in a domain requires separate domain model
These models are dynamic in nature and change over time
Our objective is “automatic generation of domain models from largely available noisy transcriptions of telephonic conversations between call center agents and customers”
Unstructured Information Management Group34
IBM India Research Lab
Example: Snippet of Automatic Transcription
SPEAKER 1: windows thanks for calling and you can
learn yes i don't mind it so then i went to
SPEAKER 2: well and ok bring the machine front
end loaded with a standard um and that's um it's
a desktop machine and i did that everything was
working wonderfully um I went ahead connected
into my my network um so i i changed my network
settings to um to my home network so i i can you
know it's showing me for my workroom um and then
it is said it had to reboot in order for changes
to take effect so i rebooted and now it's asking
me for a password which i never i never said
anything up
SPEAKER 1: ok just press the escape key i can
doesn't do anything can you pull up so that i mean
Unstructured Information Management Group35
IBM India Research Lab
Domain Model
We define the Domain Model as a Topic Taxonomy where every node is characterized by– Topics
– Typical Question-Answers (QAs)
– Typical Actions
– Call Statistics
Unstructured Information Management Group36
IBM India Research Lab
Block Diagram
StopwordRemoval
N-gramExtraction
Database,
archive,
replicate
Can you acc
ess ya
hoo?
Is modem on?
Call statistics
Feature Engineering
ASR
Clusterer TaxonomyBuilder
ModelBuilder
Component
Clusters of different granularity
Voice help-desk data
1
2
3 4
5
Unstructured Information Management Group37
IBM India Research Lab
replicate error unable to find path to servergo to the workspaceon left hand side look under server icon………..
are you using lotus notes sixdo you have the lotus notes closed…………………..
avg. transcription length = 1214.540984 wordsavg. call duration = 712.7395 secs
Unstructured Information Management Group38
IBM India Research Lab
Unstructured Information Management Group39
IBM India Research Lab
Unstructured Information Management Group41
IBM India Research Lab
Conclusion
Huge amount of unstructured data is being produced everyday in contact centers– Analysis can help to improve customer satisfaction, agent
productivity, call handling time
Opportunity to play with real “real-life data”– Learning experience
Importance of handling noise in unstructured data
– Workshop on Analytics for Noisy Unstructured Text Data (at IJCAI 07) [http://research.ihost.com/and2007/] – deadline 25 Sep (day after tomorrow!!)
Unstructured Information Management Group42
IBM India Research Lab
Thanks!!
Unstructured Information Management Group43
IBM India Research Lab
BACKUP
Unstructured Information Management Group44
IBM India Research Lab
Bharti PoC
Demonstrate how text analytics can add value to the existing Complaint Management Systems and make it more efficient
Demonstration of the software
Possible ways to extend this work
Discussion
Unstructured Information Management Group45
IBM India Research Lab
How Does The Data Look Like?
Comments"SAURABH AGARWAL/02/09/2005/ -
MR. SHAMBHU CALLED UP FOR
COMPLAINT !! IRREGULAR DIAL
TONE...PLZ CHK !! CONTACT NO.-
9826335426 !!THE COMMUNICATED SLA TO
SUBSCRIBER IS 03/09/2005 02:00:00
PM
AMANDEEP KAUR/03/09/2005
11:53:17 - ANIL//OLC//SPOKE TO MS-
RENA (SR NO-830201 BLACK
BETAL!NEW INST PROVIDE )TECH-RAMESH"
Ins
talla
tion
Addre
ss
EWS
24/1
5 NEA
R WATE
R TANK O
LD
SUBHASH
NAGAR B
HOPAL
MP B
PL-22
Unstructured Information Management Group46
IBM India Research Lab
Some Relevant Text Analytics Techniques
Cleaning up of data– Spelling correction
• irregular dial, iregular dialtone,irregular dt fone,irrgular dt psl,irregular dt so plss,irregular dt due are grouped together
– Abbreviation expansion – ….
Annotators– Extracted problem areas such as Intermittent Dial Tone, Rosette
Issue etc. Hints taken from questions provided by Bharti– Address segmentation such as Subhash Nagar, Bhopal, M.P. etc.– Sentiment, Product, Services – Application specific– ….
Unstructured Information Management Group47
IBM India Research Lab
Then?
Relevant structured and extracted annotated data is loaded into star-schema and a Business Intelligence (BI) application is developed on top of that
The BI application is capable of showing different views of the data by doing slice-and-dice, rollup-drilldown, association, comparison etc. operations
Lets see the demo of BI application developed on hard-faults data collected from M.P.
Unstructured Information Management Group48
IBM India Research Lab