ROUNDTABLE
June, 2018
Chris Ezekiel, Founder
and CEO, Creative
Virtual
Rick Britt, VP of AI,
CallMiner
Troy Surdick, Senior
Product Manager,
NICE Nexidia
Artificial
Intelligence in
the Contact
Center
www.creativevirtual.com
CRMXchange Tech Tank
Artificial Intelligence in the Contact Center
Chris Ezekiel, Founder & CEO
@chrisezekiel @creativevirtual
How important is customer experience in making purchase decisions?
Source: PwC Future of Customer Experience Survey 2017/18
Gartner predicts by 2020:
25% of customer service and support operations will integrate virtual customer assistants technology across engagement channels, up from less than 2% in 2017.
Source: Market Guide for Virtual Customer Assistants, Gartner, 2017
Where should my contact center focus?
Centralizing knowledge management
Integrating chatbots & live agents
Combining AI & human input
There are huge benefits to using AI within the CX space, but it requires a solid foundation in knowledge management.
The future of customer engagement lies in humans and machines working in harmony.
Pure AI is not the right solution on its own for providing customer service and support.
Live Demos
▪ Transport for NSW (Web, Facebook, Facebook Messenger, Amazon Alexa)
▪ HSBC Commercial Banking
▪ V-Person Live Chat™ & V-Portal™
RSPCA: consistent
information across web, contact center, mobile
Update Cycle: Combining AI & human input
Learn: System ‘learns’ potential new customer behavior.
Approve: Human editor approves AI suggestions.
Deploy: Updated knowledgebase is
published with improved understanding.
The focus moves from trying to retain knowledge to building better relationships with customers.
The contact center benefits from:
▪ Lower costs
▪ Reduced staff turnover
▪ More engaged, skilled and happier agents
Get in Touch with Me
By email:
On Twitter:
@chrisezekiel
On the web:
www.creativevirtual.com
Presenter photo
Presenter photo
Intelligence from Customer Interactions
Artificial Intelligence in the Contact Center -
Tech Tank
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
An interaction analytical engine capable of either real-time (intra-contact) or after-contact processing
For QA, speech analysts & data scientists
For QA, supervisors & agents
For developers, data scientists & partners
For compliance & security officers
For supervisors, agents & executives
post-contact real-time
Analytics app for analyzing trends, discovering root cause, and configuring contact categories & scores.
Performance management portal, providing direct automated feedback to supervisors and agents
Comprehensive APIs for contact/data ingestion, and data extraction for app integration & development.
Remove PCI and sensitive data from call audio and transcripts to help retain compliance.
Real-time monitoring, alerting, agent next-best-action, delivered as an API stream for app integration.
Product Portfolio
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Starting with Data Prep and Feature Extraction
Interactions are converted to data using the our Eureka Analytics platform based on our Mercury LVCSR and redacted for PCI.
“Thank you for calling ABC Bank.. of your social security number [REDACTED]..”
The file and text are tagged with relevant categorical and semantic data
Thank you for calling ABC Bank. How can I help you?
This is my third time calling! You overcharged me on my last bill. I need to speak with a manager
May I confirm your name, address , and last four digits of your social security number?
I’ve already entered my account information in the IVR! You people are ridiculous!
Proper Greeting
Right Party Contact
EmpathyPaymentLanguage
Dissatisfaction CollectorEffectiveness
PolitenessChurnLanguage
CloseLanguage
Features are extracted as well, metadata, speaker, start time of word, word count, scores, outcome dispositions…etc
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Real-time Analytics Platform
❑ Features‒ Real-time audio/metadata ingestion
‒ Real-time transcription
‒ Real-time PCI audio/transcript redaction
‒ Real-time rules processing & alerting API (contextual memory)
❑ API benefits‒ Integration into existing agent desktops
‒ Customized action such as messaging
❑ Integration into Eureka platform‒ Feeds real-time transcripts & alert data
to Eureka platform/analyze/coach
‒ Single admin and content reuse
Event:CallSnippetSimple
{
Start: 243.3,
Snippet: “I’d like to speak with a
supervisor”
}
Event:CallAlert
{
Start: 243.7,
Alert: “Escalation”
} Alert GenerationWeighted rules-based pattern
recognition & alerting
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
www.callminer.com/demo
LISTEN TO YOUR CUSTOMERS. IMPROVE YOUR BUSINESS.
A no cost, “Proof of Concept” AI Audit of your customer interactions- your audio recordings and metadata through our contact analytics platform.
Get a free AI Audit with
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Elements of an AI prediction
CallMiner Training Data (Generated from Billions of
Category matches)
CallMiner or Partner or Customer created Models
CallMiner Eureka Platform Executes Model for results
In Real Time or Batch
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
First Call Resolution
CustomerSatisfaction
CustomerChurn
Upsell/ Revenue Collection
Practical Prediction and Modeling Applications of AI
Probability that a future interaction will happen because of the
current interaction, and recommend
actions to prevent it
Predict the NPS or CX score intracell based
on sentiment and language
Probability of churn based on in call signals,
sentiment, and semantics
Predicting the ideal environment and timing of a sale,
resolution, or upsell.
Strategic Outcome Models
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Remaining Data
Hold Out
Training Data
Data Set D
Remaining Data
Hold Out
Training Data
Data Set C
Remaining Data
Hold Out
Training Data
Data Set B
Remaining Data
This becomes our data set to model and the fun begins
Hold Out
Training Data
Data Set A Model Selection
Train & Evaluate the Model
Precision = 𝑇𝑃
𝑇𝑃+𝐹𝑃=
14
14+6=
14
20= .7
Recall = 𝑇𝑃
𝑇𝑃+𝐹𝑁=
14
14+2=
14
16= .875
F1 = 2∗𝑃∗𝑅
𝑃+𝑅=
2∗.7∗.875
.7+.875=
1.225
1.575= . 77
Confusion Matrix A𝑇0 𝑇1
𝑃0 2012 82𝑃1 116 1814
Run it for Live Prediction
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Enriched Interaction Data through NLP and Eureka
Based on our rich data sets we are finding several powerful use cases in enriched NLP data
Basic Feed Forward NN
Recurrent NN
Convolutional NN
Bag of words & Ngrams
Logistic Regression
Neural Nets have been useful
on our data
• Classification Tasks
• Sequencing data
• Building up data sets
Bag of words and word sequencing
is very useful to map interaction
genomes for cause and effect
LR’s and vector
machines are
powerful tools to
predict, separate, and
enhance features
Interaction prediction
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
As we are evaluating methods strong predictive features have emerged
features NB F1 1 NB F1 0 LR F1 1 LR F1 0 NN F1 1 NN F1 0
count vectorizer 76% 71% 93% 93% 93% 93%
word2vec min 60% 69% 87% 87% 1% 67%
word2vec min + max 82% 81% 87% 86% 80% 74%
word2vec max 71% 64% 85% 86% 70% 70%
word2vec sum 70% 56% 90% 90% 45% 70%
word2vec mean 58% 71% 87% 87% 74% 77%
word2vec clusters meanshift 67% 12% 54% 60% 67% 0%
word2vec clusters gaussian 42% 66% 60% 44% 63% 5%
word2vec clusters birch 69% 31% 69% 31% 69% 31%
word2vec embedded layer 83% 82%
count vectorizer 76% 71% 94% 94% 92% 92%
Due to the size and richness of our interaction datasets we have achieved impressive predictive variables
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Remaining Data
Hold Out
Training Data
Data Set D
Remaining Data
Hold Out
Training Data
Data Set C
Remaining Data
Hold Out
Training Data
Data Set B
Remaining Data
This becomes our data set to model and the fun begins
Hold Out
Training Data
Data Set A
Model Selection
Train & Evaluate the Model
Precision = 𝑇𝑃
𝑇𝑃+𝐹𝑃=
14
14+6= 14
20= .7
Recall = 𝑇𝑃
𝑇𝑃+𝐹𝑁=
14
14+2=
14
16= .875
F1 = 2∗𝑃∗𝑅
𝑃+𝑅=
2∗.7∗.875
.7+.875= 1.225
1.575= . 77
Confusion Matrix A𝑇0 𝑇1
𝑃0 2012 82𝑃1 116 1814
Run it for Live Prediction
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Biggest Challenge of Conversational ML is tagged data
Typically, machine learning
models need to be trained on
large amounts of data to ensure
that they are accurate, but for
many problems, that large data
set simply doesn’t exist.
- Google’s Former AI chief
John Giannandrea
“
Proprietary & Confidential, CallMiner Inc.
Intelligence from Customer Interactions
‹#›
Push button prediction based on wide and deep data is on the near horizon
As an interaction is occurring we soon should be
able to predict the outcome based on the strategic
direction, post interaction provide a series of
predictive scores
Example:
FCR: 87% chance of call back
NPS: 4 unhappy
Churn: 63% chance of attrition
CRMXchangeWebinar
June 2018
AI in Analytics
COMPONENTS OF SENTIMENT DETECTION
• Language models identify positive and negative
words and phrases
• Positive phrases can
off-set negative ones
SPOKEN WORDS
• Indicates positive emotional state
• Pervasive element
• Can prevent false positives
and unnecessary alerts
LAUGHTER DETECTION
• Customer and agent talk at the same time,
interrupting one another
CROSS-TALK
• Intensity
• Pitch
• Jitter
• Shimmer
• Speaking rate
PITCH & TONE
27
awesomethis is ridiculous
No problem
I’m so annoyed
TRAINING THE SENTIMENT MODEL
28
SENTIMENT
PREDICTIVE
MODEL
LARGE AMOUNTS OF
LABELED DATA:
RECORDED CONTACT
CENTER CALLS
+
AFTER-CALL SURVEYS
+
CONTACT CENTER
METADATA
TRAINING DATA SET
VALIDATION DATA SET
80%
20%
NEGATIVE CALLS
POSITIVE CALLS
TRAINING THE MODEL
VALIDATE
ADJUST
SENTIMENT AS PROXY FOR AFTER-CALL SURVEYS
Statistical process to:• Normalize sentiment results to
specifics of your customer
environment
• Verify predictive value (statistical
validation of p-scores, means,
medians, etc.)
• Identify meaningful sentiment
ranges
29
2%2% 28% 40% 23% 7%
-31 -2 0 +1 +30+3
Highly Negative Negative Neutral Positive Highly Positive
30
NEXIDIA NEURAL PHONETIC SPEECH ANALYTICS
YEAR FOM
2005 0.48
2009 0.53
2013 0.54
2014 0.65
2015 0.71
2016 0.73
2018 0.78
Represents 10x
reduction in
False Alarm
Rate due to
Deep Neural
Networks
Embedding AI into Nexidia Analytics
31
• IVR Optimization
• Automatic Categorization
• “Neural Net Queries”
• General Purpose Predictive
Model Creation
• Journey Excellence Score
32
Automatically recommend IVR improvements
IVR JOURNEY OPTIMIZATION
32
33
Machine suggests topics for a set of media based on all available data
AUTOMATIC INTERACTION TOPIC CATEGORIZATION
33
34
“Neural Net Queries”• Categorization based on examples
• Machine learns new category based on examples of that category / not of that category
• Semi-supervised machine learning
TOPIC (QUERY) DEFINITION BY EXAMPLE
Cancel
Premium
Service
NEXIDIA
QUERY
RESULTS
Machine
Learning
INTERACTIONS
Expanding the Use
of AI
36
• Score individual interactions
to identify those that most
meet the criteria of interest
(creating a metric)
• Create many metrics that
can be used elsewhere
• Requires large, well-formed
training data sets
INTERACTION-LEVEL PREDICTIVE MODELING
80 42 11 94 31 73 22 75 81 14 77 64
Machine
LearningData Data
PREDICTIVE SCORES FOR EACH INTERACTION
INTERACTIONS
37
Single metric for cross-journey customer experience
CUSTOMER JOURNEY EXCELLENCE SCORE
Consider all customer
touch-points (not just
in the contact center)
Easily identify
Journeys that have
negative impact to
customer experience
38
Simulate Customer Journey scenarios and predict the impact to JES
PREDICTIVE JOURNEY ANALYTICS
Understand how
changes to journey
touchpoints impact the
customer experience
www.nice.com/analytics
ROUNDTABLE
Q&A
Chris Ezekiel, Founder
and CEO, Creative
Virtual
www.creativevirtual.com
Rick Britt, VP of AI,
CallMiner
www.callminer.com
Troy Surdick, Senior
Product Manager,
NICE Nexidia
www.nexidia.com
Artificial
Intelligence in
the Contact
Center