Post on 29-May-2020
transcript
© SAMRA 2017 – Annual Conference Delegate Copy 1
Customer Centric Artificial Intelligence - Using Text and Sentiment Analysis & Deep Neural
Network Learning to make Chatbots Reply in a more Customer Centric Fashion
Prof Adré Schreuder, Arné Schreuder and Jeannie van Wyk (né Schreuder)
The increase of different customer feedback channels, sentiment analysis and deep neural network
learning has become one of the biggest trends of the world. The aim of this study was to better
understand the views, opinions and perceptions towards the advancements that have been made in
market research tools and information technology innovations today. The key findings of the study
included that respondents are aware of Artificial Intelligence and the definition thereof. Most of the
respondents indicated that they prefer human interactions compared to automated feedback
communications and the source of interaction in important. Simple criteria were proposed that could
be used to ultimately assign a Customer Centricity Score (CCS) to chatbots responses. It was found
that an ANN could be used to learn the hypothetical function that would lead to an accurate CCS.
This model could be used to guide AI Chatbots to be more customer centric in customer feedback.
© SAMRA 2017 – Annual Conference Delegate Copy 2
1. Introduction
Customer centricity is the modern day focus of the dynamic organization and justifiably so since the
power balance have shifted from an organizational centric focus to a customer led focus. The
growth of digital & mobile technology had a massive impact in the way that companies engage with
customers. Not only did this bring more agility for organizations, but also a substantial increase in
complexity and ability to manage the omni-channel experience of customers. It is especially true for
customer engagement platforms where digital and mobile technology brought exciting new ability
and scale to organizations. Statistics in customer engagement channels now include new channels
that did not exist as options how to interact with customers even 5 years ago. Take for example the
degree to which automated reply to customer enquiries have alleviated pressure on human resources
and ultimately cost-to-serve. The modern customer has become used to automated replies generated
from email communication from the company that was queried – i.e. auto generated confirmation
emails when you submitted a user request to the companies help desk over email. The irony of this
new technology is the message and basic underlying lack of customer centric focus when you
receive an email with the clear instruction – “Please do not reply to this message. Replies to this
message are routed to an unmonitored mailbox”.
The lack of customer centric focus and subsequent frustration generated with customers is seen in
the following blog post (Seth, 2013) (amongst many other similar type of customer complaints)
where the author list three insane reasons for this non-customer centric type of reply:
“Quite plainly, “DO-NOT-REPLY" email addresses tell your customer that you don't give a
f@*k what they have to say.
“DO-NOT-REPLY” email addresses imply that you will dictate the relationship. Conversation
is not welcome here.
“DO-NOT-REPLY” email addresses say that you’re too lazy to consider your customer’s
convenience, and instead value your time over theirs.”
This paper will focus on turning customer replies during daily interaction into a more enjoyable
customer experience – basically making automated chatbot replies more customer centric. It is an
attempt to bring the world of digital & mobile technology closer to customer centric principles of
world class customer experience.
1.1 Review of Previous Research & Literature
The body of knowledge related to this paper include (albeit not limited to) the following main
conceptual topics:
Customer centricity
Artificial Intelligence (AI)
Text and Sentiment analysis
Due to the comprehensive nature of this literature a separate section of the paper is devoted to the
thorough review and integration of these three large bodies of knowledge (see Section 0 below)
1.2 The Rationale for and Importance of the Research
A 2014 Gartner survey (Sorofman, 2014) published on the role of marketing in customer experience
and reported that 89% of marketers expect that by 2016 the new battlefield to be a central focus on
customer experience.
© SAMRA 2017 – Annual Conference Delegate Copy 3
Customer experience can be defined as (Verhoef, P; Lemon, K; Parasuraman, A; Roggeveen, A;
Tsiros, M; Schlesinger, L, 2009): “…holistic in nature and involves the customer’s cognitive,
affective, emotional, social and physical responses to the retailer. This experience is created not
only by those elements which the retailer can control (e.g., service interface, retail atmosphere,
assortment, price), but also by elements that are outside of the retailer’s control (e.g., influence of
others, purpose of shopping). Additionally, we submit that the customer experience encompasses
the total experience, including the search, purchase, consumption, and after-sale phases of the
experience, and may involve multiple retail channels.”
The definition clearly underlines the complexity and holistic nature of customer experience as
organizations must ensure that all the products, channels, customer segments and engagement
moments are well aligned - from bricks-and-mortar retail outlets to digital marketing processes that
trigger social media interaction and engagement.
According to the same Gartner research (Sorofman, 2014) the battlefield is far from won, since less
than 50% of companies see their current customer experience programs as substandard to achieve
the envisaged vision. The author reports that two-thirds of CMO’s expect their customer experience
capabilities to be industry leading or much more successful than their peers within five years.
Paradoxically many organizations believe that this superior state of customer experience
capabilities will be achieved through increased reliance on technology, whilst the core of customer
experience talks about a massive cultural shift that is required in the hearts and minds of the entire
organization. It seems that the chasm between technological efficient cold programmatic customer
response management and warm empathetic customer centric customer management will be a
“bridge too far”. In order to contribute to this very real business challenge this paper will attempt to
show how deep neural network learning in text and sentiment analysis can bring the efficiency of
technology (via the use of Chatbots) in customer reply closer to customer centric reply that is
characteristic of a warm human response.
The paper suggests the application of the Turing test (Turing, 1950) to turn AI driven Chatbot
replies into more customer centric replies.
1.3 Research Problem
Facebook, Google, WeChat and Skype has been very successful in using AI for customer queries
and it appears that more service providers will adopt AI driven chatbots to handle customer queries,
replies and general frequently asked questions. Although the application of AI in these settings
clearly demonstrated their superior capabilities in terms of scale and efficiency, the customer centric
effectiveness has been lacking. The mere fact that Alan Turing suggested in 1950 that a key to the
Turing test (Turing, 1950) is natural language has inexplicably linked AI with Natural Language
Processing (NLP). By using large datasets in conjunction with text and sentiment analytics and deep
neural network learning, one can teach these AI entities to engage with customers in a more
Customer Centric fashion.
The research problem is thus how to ensure that the rich cultural and language diversity is
maintained in customer replies by applying text and sentiment analysis that is primarily English
language based, whilst maintaining a dynamic customer centric manner in replying back to
customers. Furthermore, how can the research techniques (text and sentiment analysis & deep
neural network learning) be used in this new feedback technology through AI-driven Chatbots?
What is the perception of customer service managers and researchers about this highly advanced
merge between market research tools and information technology innovation? What are the ethical
considerations – especially in absence of clear guidelines for this new technology, i.e. can an AI
© SAMRA 2017 – Annual Conference Delegate Copy 4
Chatbot be unethical in its reply (how should we prevent, control & manage this). How would
customers in general react to knowing that the reply was an AI chatbot and not a real human reply?
What are the customer preferences between human personalized reply vs AI Chatbot reply given the
faster turnaround, cost saving and efficiency gains?
Some of the questions above will be addressed in this paper, but the primary research problem is the
following:
Can deep neural network technology be used in conjunction with text and sentiment analysis
to ensure culturally relevant, ethically correct, yet honest, customer centric feedback to
customers?
1.4 Hypothesis
The basic hypothesis of this paper is thus formulated as:
H0: Deep neural network technology can NOT be applied to text and sentiment analysis to ensure
more customer centric AI chatbot replies to customers.
H1: Deep neural network technology can be applied to text and sentiment analysis to ensure more
customer centric AI chatbot replies to customers.
1.5 Research Objectives
Two different objectives were formulated for this research paper:
Primary Objective:
To investigate how text and sentiment analysis can be used in conjunction with deep neural network
learning to teach AI Chatbots to reply in a more customer centric fashion.
Secondary objectives:
What are the cultural diversity challenges to overcome when using a predominant English based
text and sentiment analysis methodology – to identify how different cultural groups identify
with AI Chatbot replies as being culturally sensitive and correct.
How does researchers and suppliers foresee that this methodology be adopted in organizations
in future? What are the anticipated adoption, challenges and ethical considerations?
With speech analytics customer service managers and researchers now live in real time – is this
a threat to researchers? Would periodic insight reviews be conducive to a better understanding
of customers and thus help to provide a sustainable improved reduction in customer complaints
and problem resolution statistics?
Test the customer preferences for queries - using AI Chatbots or real human replies. What are
the challenges, perceptions and benefits seen from a customer perspective?
© SAMRA 2017 – Annual Conference Delegate Copy 5
2. Previous Research & Literature Review
2.1 Introduction
The extensive scope of conceptual topics covered in this paper (Figure 1: Conceptual Framework
for Customer Centricity) would not allow for a thorough review of every topic and the authors thus
propose to acknowledge the conceptual foundation of these topics. The following topics deemed
central to the literature review of this paper:
Customer Centricity
Customer Engagement & Customer Experience
Social Media as the focused channel of customer engagement
Text and Sentiment Analysis as criteria for customer centric feedback & customer interaction
Figure 1: Conceptual Framework for Customer Centricity
The second foundational focus of the paper is the Artificial Intelligence and specifically deep
learning neural networks. Similarly, to the conceptual schema above the topic coverage of Artificial
Intelligence is far too extensive for a full discussion of every topic and as such the literature review
in this regard would concentrate on (
Figure 2: Conceptual Framework of Artificial Intelligence):
Artificial Intelligence as scientific domain
Machine learning and Neural Networks
Types of Neural Networks
Customer Centricity
Defining Customer Centricity
Customer Engagement & Customer Experience
Surveys
Metrics
Type of Metric
Customer Satisfaction
Net Promoter Score
Effort Score
Text & Sentiment Analysis
Chatbot Interaction
Frequency of Measurement
Transactional
Reputational
Social Media
ORM (Online Reputation Management)
Social Media Feedback & Reply
Text & Sentiment Analysis
Chatbot Interaction
Contact Centre
Outbound
Inbound
Speech Analytics
(Speech to Text)
Chatbot Interaction
Website
FAQ's
Helpdesk
Contact us
Community
Text & Sentiment Analysis
Chatbot Interaction
Support Desk
Email correspondence
Text & Sentiment Analysis
Chatbot Interaction
Face-to-Face
Walk-in Centre Help Desk
Speech Analytics (Speech to Text)
Chatbot Interaction
SMS
Helpdesk/ Contact Us
Text & Sentiment Analysis
Chatbot Interaction
© SAMRA 2017 – Annual Conference Delegate Copy 6
Figure 2: Conceptual Framework of Artificial Intelligence
2.2 Customer Centricity
Customer Centricity is essentially about offering goods and services to customers based on a proper
understanding of customer needs, wants, and behaviors. Although most leaders would not disagree
with this business model it is certainly not as widely accepted and implemented by organizations. It
seems easy to agree with, but has proven to be difficult to build, implement and sustain in
organizations.
To formally defined customer centricity we revert to two sources – first the most credible scientific
source from the Journal of Service Research (JSR) (Shah, Rust, Parasuraman, Staelin, & Day, 2006)
and second to a respective practitioner publication by late Doug Leather (Leather, 2013). The
Journal of Service Research’s definition of Customer Centricity: “… the true essence of the
customer centricity paradigm lies not in how to sell products but rather on creating value for the
customer and, in the process, creating value for the firm; in other words, customer centricity is
concerned with the process of dual value creation.”
The four key drivers of customer centric transformation according to the authors are shown in the
illustration below:
Artifical Intelligence
Computational Intelligence
Machine Learning
Neural Networks
Single Perceptron
Net Input Signal
Activation Functions
Architecture
Layers
Training of Neural Networks
Feed Forward
Data Preparation
Fitness Function
Gradient Descent
Swarm Intelligence
Types of Neural Networks
Simple Recurrent Neural Networks
Deep Neural Networks
Long Short Term Memory (LSTM)
Chatbot Interaction
Sequence-to-Sequence (Sec2Sec)
Chatbot Interaction
Evolutionary Computation
Swarm Intelligence
Artificial Immune Systems
Fuzzy Systems
Probabilistic Techniques
© SAMRA 2017 – Annual Conference Delegate Copy 7
Figure 3: Organizational Barriers to Customer Centricity - Adapted from (Shah, et al., 2006)
REAP Consulting Definition of Customer Centricity (Leather, 2013): “Customer-centricity can,
therefore, be defined as the eco-system and operating model that enables an organization to design
and deliver a unique and distinctive customer experience. This architecture enables the business to
acquire, retain and develop targeted customers efficiently for the benefit of employees, customers
and stakeholders.”
Another important source of literature comes from the World Bank in their program called Closing
the Gap of the Poor (CGAP) that formulated the following model that outlines the five pillars of
customer centricity.
Figure 4: Pillars of Customer Centricity - Adapted from World bank – CGAP Customer
Centricity Unit (CGAP, 2017)
Culture
FinancialMetrics
ProcessesStructure
OrganizationalBarriers
ProductCentricity
CustomerCentricity
People,toolsandinsightsInformation,insights,strategy,customervaluepropositions&empoweringemployees
CustomerexperienceBasedonInsights- Design,test,build,deliver,scale&renew
FocusingoperationsFocusingoperationsonthecustomer
LeadershipandcultureCustomerfocusedleadershipandculture
ValueCreatingandmeasuringvalu e@customer,firm&societylevel
© SAMRA 2017 – Annual Conference Delegate Copy 8
2.2.1 Customer Engagement and Customer Experience
In a 2014 study by Rosetta Consulting (Platt & Schnoes, 2014), customer engagement has been
studied and the following statistics are worth noting – especially in terms of the value bi-directional
Voice of Customer results:
Engaged customers are 4 time more likely to advocate the brand
Engaged customers do 90% more frequent purchases
Engaged customers spend 300% more than non- or low engaged customers
Companies with high levels of customer engagement were 2.2 times more likely to show
increase in market share & 86% reported increase in revenue
It clearly makes the case for an iterative bi-directional step-by-step process to better serve our
customers. Another study published in the peer reviewed Journal of Marketing Science in 2006,
(Gupta & Zeithaml, 2006) report on meta-findings relating to the effect of customer metrics on
financial performance. The following non-linear effect of improvement in Customer Satisfaction
and firm ROI is particularly relevant:
Figure 5: The link between Customer Satisfaction and Financial Returns (Adapted from Gupta
& Zeithaml, 2006)
2.3 Feedback channels
Feedback from a customer is one of the most valuable pieces of information a business can receive.
Successfully utilizing customers’ feedback is essential in understanding the voice of the customer
and ultimately becoming customer centric. While listening and learning from customers has long
been recognised as important, different social media platforms have fundamentally changed the
interaction between business’s and customers (Gallaugher & Ransbotham, 2010). There a lot of
different channels that could be used by a customer to provide feedback, whether it would be to
complain or to simply ask for help. Customer feedback flourishes on sites such as Twitter and
Facebook, but takes more than merely monitoring the conversations to be able to control these
social sources. In the next section we will explore the different channels that could be utilized to
provide feedback to any organisation.
2.3.1 Social media
Social media has become one of the most important technological developments in the world
(Leung; Law; van Hoof; Buhalis, 2013). Over the past decade, several innovative social media
platforms have emerged that provides customers around the world with a fast amount of
communication possibilities. The increase of social media interactions is rapidly changing the way
organisations communicate and interact with their customers and vice versa (Ngai, Tao, Moon,
2015). As a very cost-effective way to engage online, social media gives customers and
organisations a broader reach beyond traditional communication methods like, for example, email
2.37%
+1% ROI
Customer
Satisfaction
-1%
-5.08%
© SAMRA 2017 – Annual Conference Delegate Copy 9
(Kietzman, Hermkens, McCarthy, Silvestre, 2011). With a simple ‘post’ or ‘tweet’, businesses and
customers can promote products and services, provide instant feedback or support and create an
online community of brand enthusiasts (Mangold & Faulds, 2009). On the other hand, social media
offers businesses the opportunity to step into the conversations that customers are having on social
media, by offering a solution that could surprise customers. As Krasne (2017) rightfully said: “We
now live in an age where a single Tweet can circumnavigate the globe in less than a minute, right
or wrong, the customer can never be ignored” (Krasne, 2017).
Managers are often unaware about how much the service aspect of their business influences their
clientele’s experience of their operation. Websites such as HelloPeter.com are dedicated singly to
reporting on good and bad service and can in some ways be customers’ attempts to bring to
management’s attention the seriousness of this problem. Providing poor service to discerning
customers could potentially be detrimental to business in the future in terms of aspects such as loss
of consumer loyalty as well as possible financial losses and many others as mentioned by (Zeithalm,
et al., 1996).
Although the concept dates to the 1960s, viral growth and commercial interest only arose well after
the advent of the Internet (Gross & Acquisti, 2005). Social media is defined as “a group of
Interned-based applications that build on the ideological and technological foundations of Web 2.0,
and that allow the creation and exchange of User Generated Content” (Kaplan & Haenlein, 2010).
These websites and applications enable users to interact among people, communities and companies
in which they create, share and exchange content, information, ideas, pictures or videos on different
social media platforms (Ahiqvist, et al., 2008). The terms Web 2.0, social media, and creative
consumers are often used imprecisely and interchangeably; largely because they are closely related
and are, indeed, interdependent (Berthon, Pitt, Plangger & Shapiro, 2012). Social media can take on
many different forms such as (Harris & Rae, 2009)( (Kaplan & Haenlein, 2010): blogs and micro-
blogs, collaborative projects (e.g. Wikipedia.com), picture-sharing websites (e.g. Flickr.com),
video-sharing websites (e.g. YouTube.com), Virtual game worlds (e.g. Second Life). networks (e.g.
Facebook.com) and instant messaging (e.g. Whatsapp and Mxit).
Listening through social media can prove particularly useful for gathering candid feedback from
customers. In practice, it is referred to as ‘social listening” because the direct comments or mentions
on social networks aren’t the only way for your business to get responses (Ciotti, 2016). The high
volume of information exchanged in real-time on social media is immense, making these channels a
rich source of customer insights and competitive intelligence (Clarabridge, 2017).
Nowadays, different online reputation programs, social media monitoring tools and analytics are in
place to facilitate, manage and track a company’s presence especially referring to what customers
are saying about them on social media. Social media reputation management is crucial when it
comes to tracking, monitoring and ultimately eliminating negative social media material about your
brand to improve your name or standing (Clarabridge, 2017). Online reputation management,
sometimes abbreviated as ORM, includes mug shot removal sites, astroturfing review sites,
censoring negative complaints, and using search engine optimisation tactics to influence results
(O'Hara, 2013). If done properly, social media reputation management builds your credibility to
customers, which strengthens their trust in your brand and one the other hand, the loss of reputation
affects competitiveness, local positioning, trust and loyalty of stakeholders, media relations, and the
legitimacy of operations, even the license to exist (Aula, 2010).
The next section (Section 2.4) will explore the literature around text and sentiment analysis and the
importance thereof.
© SAMRA 2017 – Annual Conference Delegate Copy 10
2.4 Text and Sentiment Analysis
For the last number of years the Greenbook Research Industry Trends (GRIT) annual survey have
consistently shown the importance, current use and future potential of Text and Sentiment Analysis
as an emergent market research method (see Figure 6: GRIT Report 2016 Emerging Research
Methods). The fact that Social Media Analytics is shown within the same top 5 methods provides
further support for the interest in the topic of this paper.
Figure 6: GRIT Report 2016 Emerging Research Methods (Murphy, 2016)
The link between Text analytics and Artificial Intelligence has been previously identified as Natural
Language Processing, which is the core methodological basis of Text and Sentiment analysis. It is
therefore important to briefly review the origins and foundation of Natural Language Processing
and the relevance to Text and Sentiment Analysis.
2.4.1 Natural Language Processing (NLP)
According to (Grishman, 1984) Natural Language Processing “involves the development of
computer programs which can analyse natural language and act appropriately on the information
contained in the text or the utterance”. The two primary roles in the storage and retrieval of large
volumes of information are:
Retrieval Role – providing an easily-learned and user-friendly approach to retrieve data from
free format text verbatim data.
Storage Role – automatically structuring free format text based data in order for the information
to be processed and retrieved.
The following section is based on a combination of sources (primarily (Grishman, 1984),
Clarabridge and own practical experience).
The history of NLP can be traced back to the evolution of human intelligence, especially as
divergent from communication between animals (lacking an intelligible language). It shares the rich
history of homo sapience as we developed our cognitive intelligence as foundation to our semantic
intelligence. The 1950s can be cited as the milestone where the modern-day science of Natural
Language Processing found its early formative phase. In view of Alan Turing’s 1950 article titled
"Computing Machinery and Intelligence" where the famous Turing test was proposed as criterion of
intelligence. The key proposed by Turing that would distinguish Natural (Human) to Artificial
Intelligence was to be found in how we communicate through language (natural language).
© SAMRA 2017 – Annual Conference Delegate Copy 11
Some authors acknowledge the Georgetown experiment in 1954, as described in (Gordin, 2016),
especially the field of Machine Translation as a key moment in the history of NLP. The experiment
involved the use of an IBM 701 with 11 separate units almost the size of a tennis court to automatic
translate of more than sixty Russian sentences into English from a vocabulary of 250 Russian words
and six rules of ‘operational syntax’. Machine Translation was hailed to open the door to full
automatic translation of any text in one language into any other.
The initial enthusiasm then slowed when a decade of long research has failed to fulfil the initial
high expectations. This resulted in a significant reduction in research funding and consequently
slowed the progress until late 1980s, when the first statistical machine translation systems were
developed. The few exceptions during this phase was the development of some successful NLP
systems such as SHRDLU and ELIZA – a simulation of a Rogerian psychotherapist by Joseph
Weizenbaum (1964-1966).
Most early NLP systems used complex sets of hand-written rules and first machine learning
algorithms such as Decisions Trees provided further impetus to the development of Part-of-Speech
Tagging (POST) through the use of hidden Markov models. This phase in the NLP history was
accelerated through the advancements in statistical models that introduced probabilistic decision
models to the processing of the text based input data.
Lastly the stepwise process of Text and Sentiment Analytics with software such as Clarabridge is
shown below (Clarabridge, 2017):
Step 1: Normalization.
Tokenizer - breaks the stream of text into words, phrases, symbols or other meaningful elements
called ‘Tokens’
Sentence Detection & Morphological Identification
Step 2: Parts-of-Speech Tagging
Machine Learning disambiguates POS (part of speech). Words have many potential meanings,
e.g. ‘kind’ can be either a noun (“kind of sweater”) or a verb (“she was kind to the puppy”).
Parts of speech can be explicitly determined and written as a rule during semantic analysis. This
may be necessary with certain product names, brands, slang, etc.
Step 3: Named Entity Recognition (NER)
Four toolsets within that can be used:
1. Out-of-the-box models
2. Linguist patterns – Named entities (New York)
3. Customisable rules engine
4. Machine learning – Similar to POS tagging, machine learning enables the models to be trained
to more accurately categorise within a specific dataset or to add additional entity types.
Step 4: Semantic Recognition
Semantic recognition extracts the relationships between words, phrases, sentences, and larger
units of text.
Root cause analysis - By extracting relationships between words (like an adjective and a noun,
New and MP3 player), the full semantic parse enables the software to understand how the
© SAMRA 2017 – Annual Conference Delegate Copy 12
adjective and noun relate to each other. This capability is critical in understanding what is
driving the feedback text in the first place. This “why” powers Root Cause Analysis.
Step 5: Advanced Linguistics
Clause detection Anaphora resolution – “The car was really great; it [the anaphor] cruised along like it was
gliding through the air.”
Disambiguation “bat” (animal) vs “bat” (sports equipment)
- single words,
- multiple word phrases, and
- lexical affinities
Step 6: Fact Extraction
Fact extraction automatically pulls out and relates information within unstructured text
Enables logical reasoning by drawing inferences based on the text & allows users to discover:
- Co-occurrence of brands or companies
- People's associations or activities
- Market and competitive intelligence
- Familial and interpersonal relationships
- People's attributes
- People's appearance
- Organizational relationships
- People and organizations' actions
2.5 Artificial Intelligence (AI)
Humans are no longer the only advanced intellectual beings on planet Earth. A strong statement to
make, but we have come a long way since rubbing sticks together to make fire.
Without a doubt, humans have long taken the title as the most evolved intelligent beings on planet
Earth. Maurice Conti (Conti, 2016), summarizes four major time periods throughout history. This
includes the Hunter-Gatherer Age which lasted several million years, the Agricultural Age which
lasted several thousand years, the Industrial Age which lasted several centuries and now, finally, the
Information Age which only started a few decades ago. With each major period that passes, the
level of mankind's intelligence has grown in leaps and bounds. It is within this modern Information
Age that Alan M. Turing developed the first Computational Machines. Today, we refer to these
machines as computers.
Ever since the creation of the first Computational Machines, man has been fascinated with the
possibilities of what computers can do. A piece of literature that particularly sparked the interest in
the field of computation, is the paper titled "Intelligent Machinery" by Turing (Turing, 1950), who
is widely regarded as the father of computers (Beavers, 2013). In the first chapter, called "The
Imitation Game", Turing proposes a very interesting question: "Can machines think?".
Artificial Intelligence (AI) is the field of study where machines are programmed to exhibit some
form of intelligence. Other sources such as (Merriam-Webster, n.d.) state AI as the ability for
intelligent agents to behave like humans. It is within the field of Artificial Intelligence that
computers will finally surpass us in intellectual capabilities and learn "how to think". Hawking
© SAMRA 2017 – Annual Conference Delegate Copy 13
(Hale, 2016) takes this a step further and states that Artificial Intelligence could evolve faster than
humans. This emanates from the belief that one day, we will be able to create Artificial General
Intelligence (AGI) using these intelligent machines, which could theoretically solve any problem.
The field of AI, although still in its infancy, has received a lot of attention the last few decades,
especially the last few years. So much so that some of the technical conglomerates of the world
such as Google has recently produced and published extraordinary advances in the field, including
the notable achievements with AlphaGo (Silver, 2016), which demonstrated that a computer can
learn how to play the world’s most difficult game and be quite good at it. In fact, so good, that
AlphaGo beat the world's number one Go player, Lee Sedol in October 2015 (DeepMind, 2016).
Surely one must admit, AI started as a basic Tic-Tac-Toe opponent, but is catching up quickly.
The aim of this chapter is to look at a very specific field of Artificial Intelligence called
Computational Intelligence (CI). The remaining sections in this chapter will contain a short
literature review of how CI works and how it can be used in the field of Customer Centricity.
2.5.1 Computational Intelligence (CI)
Computational Intelligence is the sub-field of AI that focusses specifically on the development,
design and implementation of algorithmic models that are used to solve complex problems
(Engelbrecht, 2007). CI largely finds its inspiration from nature. Particularly from biological
systems such as the study of the human brain. Engelbrecht points out that CI can be sub-divided
further into Evolutionary Computation (EC), Swarm Intelligence (SI), Fuzzy Systems (FS),
Probabilistic Methods and finally, Machine Learning (ML). For this paper, the authors will only
focus on Machine Learning.
2.5.2 Machine Learning (ML)
ML is the sub-field of CI, where computers are given the ability to learn without being explicitly
programmed to solve a specific problem. ML, on a high level, works by providing cleverly designed
algorithms with some input (training), which it then uses to learn how to adjust itself to best
represent the input data it was exposed to. Though there are many ML paradigms, for the purpose of
this paper, we will only focus on the paradigm of Artificial Neural Networks.
2.5.3 Artificial Neural Networks (ANNs)
As mentioned in Section 2.5.1, CI largely draws its inspiration from biological systems. We also
mentioned that much inspiration comes from the human brain itself.
The human brain is comprised of a complex network of nerve cells, referred to as Neurons.
According to (Engelbrecht, 2007), a Neuron consists of a cell body, dendrites and an axon.
Interconnectivity (networks) of Neurons are formed by the connection that is made between a
particular Neuron's dendrites and another Neuron's axon. Figure 7 illustrates this interconnectivity.
Neurons communicate with each other through the flow of emitted ("fired") electrical signals/pulses
over these connections. It is from this flow of signals between Neurons that intelligent behavior
emerges.
© SAMRA 2017 – Annual Conference Delegate Copy 14
Figure 7: The Biological Neuron
Computer Scientists artificially model this biological structure by means of software or hardware to
form Artificial Neural Networks (ANNs). (Engelbrecht, 2007) categorizes the applications of ANNs
as classification, pattern matching, pattern completion, optimization, control, function
approximation and Data Mining. In this paper we will focus on ANNs' abilities to classify,
optimize, complete patterns and approximate functions to solve the problem of Customer Centricity
in AI Chatbots. The following section will give a very high level explanation of the makings of an
ANN.
2.5.3.1 Theory by Example
This section aims to give the user a high-level abstraction of ANNs and how they work. It does not
go without saying that there is a vast amount of technical details and detailed literature that go into
the field of ANNs. Apart from the fact, ANNs is not only a discipline in Computer Science, but it
also contains components of Mathematics, Physics, Psychology and more. The authors of this paper
feel it is unnecessary to cover all the technical details of ANNs as it is not necessary to understand
every underlying component to understand how ANNs work. As such, the authors have thought it
best to explain the theory by means of an example.
As we have mentioned in Section 2.5.3, ANNs draw inspirations from our biological anatomy.
Particularly our networks of billions of Neurons in our brains that make up our thoughts, memories,
feelings and such. Networks of Neurons are formed, layer by layer, in the brain. These layers of
Neurons constantly send electrical signals between each other. Each Neuron fires its signal with a
certain strength. The process of learning is thus just a process of learning the correct combinations
of signals, with specific firing strengths that lead to an answer which closest represents that of
which is to be learnt. So, for example, as a child, you are exposed to many encounters with images
of cats and dogs, whether in real life or on some photo. Over time, Neurons align themselves to best
represent that which is a “cat” in your brain and that which is a “dog” in your brain. It is this never-
ending process that gives us the ability to learn, to remember, to recall and to think.
Taking the above-mentioned concept of learning into consideration, suppose we have a
classification problem that we want to solve. The authors propose a model that can distinguish
between images of cats and dogs. One needs to artificially replicate the functionality of a NN.
Therefore, the following components are needed to model NNs:
Artificial Neurons
Artificial connections
Artificial signals (and their firing strengths) with triggers
Some training data
Some way to determine how well it is performing
Some way to learn
© SAMRA 2017 – Annual Conference Delegate Copy 15
For a comprehensive Glossary of Terms in Artificial Intelligence please refer to Appendix B.
A very basic layout of an ANN is given below in Figure 8: A basic ANN structure along with all its
components
Figure 8: A basic ANN structure along with all its components
It is important to remember the higher-level purpose of an ANN. The purpose of an ANN is to
represent some function 𝐹𝐴𝑁𝑁 that could best map the input data that is fed to it, to some output
which would yield the smallest error between what is evaluated by the ANN and what is real target
outcome. The outcome 𝑜 for pattern 𝑝 as evaluated by 𝐹𝐴𝑁𝑁 is illustrated below in Equation 1:
Equation 1: Output as calculated by the ANN
The error function (𝑆𝑆𝐸) output, relative to epoch 𝑒, is given as is given in Equation 2 below:
Equation 2: Error function used to evaluate ANN performance
where 𝑃 denotes the set of all training data patterns used during epoch 𝑒. The purpose of the
Heuristic is thus to minimize this error over time. This minimization of the error is an indication of
the ability of the ANN to learn.
If we now take into consideration the components above, one can now begin to imagine how each
component finds its place in the structure and operations of the ANN. Figure 9 below show how an
ANN can be used as a dog/cat classifier. Please note that this is a special type of ANN called a
Deep Neural Network (DNN). DNNs will be discussed later in this paper.
© SAMRA 2017 – Annual Conference Delegate Copy 16
Figure 9: A basic layout of a Deep Neural Network image classifier (for dogs and cats)
From Figure 9 above, one can see how Nodes are grouped together in small units, which we refer to
as Layers. Each Layer represents some level of information about the training data (image of cat or
dog). The bigger Layers represent whole structures such as “facial structure”, while smaller Layers
represent refined detail such as “whiskers”. Data passes over each Layer of the DNN, and finally
the ANN deduces a conclusion; either “cat” or “dog”. This is achieved by abstracting levels of
detail at each layer of the DNN, until eventually only a choice between “cat” and “dog” is left. At
first, when an ANN is shown images of cats and dogs, it will predict the category wrong almost
every time. But with each image that is fed, it learns from its previous mistakes by adjusting its
weights to represent the input data more correctly. Over time, it starts to predict correctly,
eventually till the point where it could predict with a 99.7% accuracy. This is how ANNs learn and
this is the reason they are powerful.
The way that these structures are designed greatly influence the type of problem that ANN can
solve. The more Nodes we have in our Network; the more information we can represent about a
training data set. However, understand that more Nodes do not necessarily mean we can solve more
and more problems. It can therefore be said that the capabilities of an ANN are not just determined
by its structure, but by various other components, most of them are mentioned above. The following
Section discusses general ANNs that can be used for an array of problems.
2.5.3.2 ANNs for everything
Section 2.5.3.1 explained the various components of ANNs to the reader. One can quickly conclude
from the many different components that there are many different types of every component. The
example above in Figure 9 is formally a Deep Convolutional Neural Network that has 2D Layers,
each containing a smaller number of Nodes as the layer before it. It is:
Fully connected
Uses Sigmoid Activation functions
Summation Units for Net Input signals
Error evaluation is done by SSE
Training is done using Gradient Descent.
Swapping components out for alternative approaches yields:
1D layers
Mean Squared Error (MSE) for Error evaluation
Training is done by applying Particle Swarm Optimization (PSO) on the weights to train the
Neural Network (Rakitianskaia & Engelbrecht, 2009)
© SAMRA 2017 – Annual Conference Delegate Copy 17
However, it is tedious to have many different ANNs for many different problems. As researchers,
we dream of a general ANN that can be used to solve many problems. Obviously such an ANN is
mythical and we are not yet sure if this is even possible, however, there are two specific types of
ANNs that have made a lot of progress in the journey towards smarter intelligence. These two types
of ANNs are Recurrent Neural Networks (RNNs) and Deep Neural Networks (DNNs) but falls
outside the scope of this paper and will not be discussed in more detail.
2.6 Chatbots, past, present, future
In the past, chatbots where designed by hardcoding responses to common patterns of messages into
the bot itself. Every possible response to every incoming message had to be identified and
hardcoded. This is obviously a very painstaking and tedious job and can quickly become
unmaintainable.
In modern times, an alternate approach to Chatbot implementations is to teach the chatbot how to
respond. Before the reader continues, the authors would like to emphasize, that when the paper
refers to “Chatbot responses”, it could take up any form on any channel of the reader’s imagination.
Typical interactions that require Chatbot responses is support/help desk interactions, inbound Call
Centres, feedback agents and so forth. For this paper, consider a Chatbot that learns to measure the
Customer Centricity of our own interactions with clients over email or social media. The authors of
this paper suggest a Chatbot that uses a specially designed Fitness Function to measure how
Customer Centric our own interactions with clients are. The proposed Chatbot will implement an
LSTM-Deep RNN that will be exposed to some email text as input. The Chatbot will thus learn
what types of responses are Customer Centric.
From this basic research, Chatbots can be extended to generate lexical output by implementing the
models defined above. The readers are referred to (Gelston, 2017) for an example of an AI chatbot
that has already been built. This Chatbot implements Seq2Seq models by means of a ML
framework called Tensorflow (Google, n.d.). This bot was trained on millions of example
conversations from the popular website
how to generate text that is a relevant, in context and in response to some initial message. If the
above research is then applied on top of this proposed model, chatbots would learn to not just
generate a response, but to generate the appropriate response that adheres to Customer Centricity
guidelines.
2.7 Proposed ANN for Customer Centric Chatbots
It was previously mentioned that a specially designed Fitness Function is needed that could help
learn the Chatbot which responses are customer centric. The authors of this paper propose that the
following elements are accounted into the design of the Fitness Function:
Initial message:
- Message text
- Sentiment analysis
• Polarity score (POs)
• Subjectivity score (Ss)
• Relavancy Score (Rs)
• Usefulness Score (Us)
• Personalisation Score (Ps)
© SAMRA 2017 – Annual Conference Delegate Copy 18
Response message:
- Message text
- Sentiment analysis
• Polarity score
• Subjectivity score
The purpose of the chatbot is thus to learn what makes a response customer centric. For the purpose
of this paper, the authors will consult the evaluation score, given by a QA expert after evaluating
our responses to client emails by hand. The ANN thus tries to learn what score a certain response
will be given by a QA expert.
The criteria that is followed by the Quality Assessor (QA) expert when evaluating a response is
given in below (Table 1):
Table 1: Criteria followed by Quality Assessor expert when evaluating a response
Criteria Notation Scoring mechanism
Is the response relevant? Relevancy Score (Rs) Score out of 10
Was the response sufficient and useful? Usefulness Score (Us) Score out of 10
Was the response personalised for the
customer? Personalisation Score (Ps) Score out of 10
The Fitness Function to be used is then simply the 𝑆𝑆𝐸 or 𝑀𝑆𝐸 between the predicted CC score by
the ANN and the CC score as given by the QA expert.
3. Methodology
This section describes the research design off the study that includes the research approach and
techniques used to respond to the research questions and objectives. The measuring instruments and
how the instruments were administered to ensure reliability and validity of the results are also
included in the chapter. Lastly, the chapter explains how the researcher collected the data and how
the data collected were analysed to be able to answer the research question for this study.
3.1 Research Design
The research approach is quantitative as well as experimental in nature (Salkind, 2012).
Furthermore, the research objective is exploratory and descriptive as the study intended to explore
and describe the perception of Artificial Intelligence (Leedy & Ormrod, 2010) as it is used to train
Chatbots to respond to human queries. The primary research methodology included text and
sentiment analysis of social media verbatim queries and the secondary research methodology
included survey research that was conducted by a formulated web-based questionnaire. The primary
methodology involved the collection of unstructured verbatim data from social media, that was
transformed into structured data by the means of text and sentiment analysis
3.2 Data Sources
The study included two different types of data collection method.
Firstly, several large social media and customer query databases was used for “training” of the deep
neural network learning AI Chatbots. Neural network technology was used to optimise the customer
centric replies of an experimental AI Chatbot(s). A software engineer designed the Artificial
Intelligence engine and AI Chatbot reply system using Tensorflow technology. The large dataset
© SAMRA 2017 – Annual Conference Delegate Copy 19
primarily consisted of English based social media verbatim queries and reply-back to customers. All
personal details of customers were treated confidentially and anonymous since the verbatim
commentary will be analysed with text and sentiment analysis with no need for identification of
individual responses, rather than the deep learning of the combined data responses.
Secondly, the researchers made use of ConsultaPanel Community – an online and telephonic
community consisting out of 180 000 consenting members, that willingly participate in different
research activities.
The sampling methodology used for the survey research comprised of convenient sampling (De
Vos, et al., 2012). The sampling method was completely random to ensure that all potential
respondents (qualifying panel members) have an equal chance of being selected. By doing so it
provided a realistic representation of the grouping under the survey. The study included a sample of
(n) 251.
3.3 Research Instruments
The secondary research instrument was an internet-based-questionnaire, which aimed to investigate
the key aspects of the study. The use of an internet-based-questionnaire allowed the execution of a
survey data collection can be more cost-effective (Leedy & Ormrod, 2010)The questionnaire design
was finalised after pre-testing and then rendered in the Verint Workbench Internet format with the
assistance of a research assistant at Consulta.
The first section of the questionnaire was designed to establish the respondent’s knowledge and
perceptions of Artificial Intelligence, using a scale type question, using a 0 to 10-point scale
(Strongly disagree/agree). The respondents also made use of a “drag-and-drop” type question, were
they could include all the words associated with AI. Lastly, the questionnaire made use of nominal
data such as a feedback channel questions that differentiated between different feedback channel
respondents have used for complaints, compliments and queries.
Clarabridge, was used to transforms unstructured (qualitative) data into structured data
(quantitative). The text analytics software natural language processing (NLP) technologies for
performing the sentiment scoring. The software accepts any source of text content including social
media feeds, enterprise feedback management (EFM) systems, and contact centre systems to
perform automated natural language processing for accurate classification and sentiment analysis.
The key criterion for validity in data collection is reliability (De Vos, et al., 2012). This refers to the
degree to which the variables that were measured were indeed free from errors of measurement to
the extent that when the same test is repeated on other subjects, the same results would be obtained
(Salkind, 2012). Reliability was ensured in this study by the following:
Reliability was enhanced by a combination of data collection methods.
A structured questionnaire was set up to collect all the key elements of the study.
A cover letter was included in the email invitation to indicate the purpose of the study and
ensured confidentiality and anonymity.
The questionnaire was constructed in such a manner that it did not take long to complete (10
minutes).
The questionnaire included questions that were easy to understand and relevant to the topic.
A pilot study was implemented to ensure that respondents understood everything and the data to
be collected is according to the study’s objectives.
To enhance the theoretical and construct validity of this study, all the key concepts of the study
were identified through a thorough literature review and conceptual framework (Addendum 1)
© SAMRA 2017 – Annual Conference Delegate Copy 20
to clearly indicate all the key elements and objectives of the study and assist in the design of the
questionnaire.
Experts in the fields of Customer Satisfaction, Customer Experience Management and
Consumer Sciences evaluated the questionnaire in terms of content as well as the measurement
of validity.
To ensure that the sample provided valid information that could be referred to the larger
population, participants were selected by means of convenient sampling. The realised sample of
n=251 was statistically large enough to satisfy the minimum criteria of statistical confidence
(90% with 5% error margin).
3.4 Data Collection
A web-based questionnaire was used to collect all the key elements of the study. The use of a web-
based questionnaire offers the research study to be more cost effective and faster as the responses
can be recorded electronically into a database (Fricker & Schonlau, 2002). When combined with
other survey modes, web-based surveys yield higher response rates than conventional survey
modes. An email cover letter invitation was designed that included the purpose of the study,
ensured confidentiality, anonymity and motivated respondents to participate in this research study.
The email invitation was send to 20 000 randomly sampled respondents on the Panel Community.
Strydom in De Vos et al. (2011) stated that: “ethics are a set of moral principles which are widely
accepted and offer rules and behavioural expectations about the most correct conduct towards
respondents, other researchers, students and sponsors”. A research design that includes the
protection against causing any harm (mental or physical) to participants and makes the data
integrity a first priority should be highly respected. The following ethical issues was taken into
consideration for this research:
The researcher ensured that the objectives of the study were clearly communicated to
respondents.
Informed consent was sought from respondents to participate in the study willingly, not by force
or intimidation in the questionnaire administration.
The data collected for the research will remain confidential and private and the information
provided by respondents was treated with confidentiality.
The respondents were informed that the survey was for research purposes and all information
provided will remain confidential.
Consulta is a Corporate Member of the Southern African Marketing Research Association
(SAMRA).
The CEO is a SAMRA Accredited Researcher (SAR) and the two co-authors are both
organisational members of SAMRA.
3.5 Data Processing
Descriptive statistics were implemented to illustrate the data visually, together with basic
frequencies and tendencies for certain sections of the data. According to Cooper and Schindler
(1998), descriptive statistics refer to measures of location (mean, median and mode), dispersion of
variability (variance, standard deviation, range and quartile deviation) and measures of shape
(skewness and kurtosis). In terms of this study it included the organisation and summation of the
data in a more comprehensible format, which included graphs, frequency, means and percentage
distributions (Cooper & Schindler, 2011).
© SAMRA 2017 – Annual Conference Delegate Copy 21
3.6 Experimental procedure: Implementing CC scoring ANN
In the previous section the authors provided the reader with all the background information required
to understand the basics of ANNs. The following section describes the experimental setup and
procedure that was followed in order to train an ANN to be able to accurately calculate a CC score.
3.6.1 Data
A dataset of 100 conversational responses from a well know South African service provider was
scraped from Twitter, using the Tweepy library. These conversations include initial tweets by
customers, specifically requiring feedback.
3.6.2 ANN Design
A simple 3 layer ANN was implemented using the following libraries:
Tensorflow: Used for ML paradigms as well as setup of ANN.
Scikit-learn: Used for ML paradigms.
Numpy: Uses for structural elements such as Tensors (Ranked matrices).
Pandas: Uses for data manipulation.
The following structure was used for the ANN:
Input Layer: 5 nodes + 1 bias, one for each criterion proposed to contribute to a global CC score
(CCS)
Hidden Layer: 256 hidden nodes used + biases
Output Layer: 1 node, depicting the predicted CCS.
3.6.3 Fitness Function and evaluation
As discussed in the previous section a QA expert analyzed each response and assigned a
hypothetical customer centricity score based on expertise. The ANN was given input and tried to
map a hypothetical function that would best fit the CC scores to input data as given by the expert.
The Fitness Function that is used is the Mean Squared Error (MSE), which measures the mean error
between the evaluated outcome of the ANN and the targeted CC score as provided by the expert.
3.6.4 Parameters
Table 2 provides all the parameters for the experiment:
Table 2: Experimental Parameters
Parameter Value
Epochs 100
Runs 30
Gradient Descent Learning Rate 0.01
Testing data set 20% of training data
© SAMRA 2017 – Annual Conference Delegate Copy 22
4. Results and Application of the Research Findings
This section provides the main findings of the study, formulates conclusions and finally make
insightful recommendations. A thorough review and discussion of the practical implications,
limitations of the research and recommendations for future research will be done.
4.1 Research Findings
4.1.1 Perceptions of Artificial Intelligence
This section provides the main findings obtained in the web-based perception survey and it presents
an evaluation of the study in terms of contribution to Artificial Intelligence and the use thereof in
improving customer feedback.
Figure 10: The use of different feedback channels
Figure 10 illustrates the different customer feedback channels that customers have used in the past
to express their complaints, compliments and queries. Email was indicated as the channel that is
mostly used for customer feedback, with 89% of the sample (n) mentioning that they have used this
form of communication. A large portion of the sample indicated that they still make use of face-to-
face interaction (68%), followed by Call Centers and Websites/Apps. Only 42% of the sample
reported that they have use social media to provide customer feedback.
Although World Wide Worx and Ornico’s Social Media Landscape report (Goldstruck & du
Plessis, 2017) confirmed that the majority of the South African population are accessing different
social media platforms by means of their mobile devices, such as smartphone and tablets,
specifically through mobile applications, customer still prefer to revert to email and face-to-face
interactions, when it comes to providing customer feedback. Another fact to consider, is the one of
data cost. The cost of data in South Africa is very high, indicating that a lot of the lower LSM
groups in South Africa do not have access to internet using a broadband connection. Akamai’s
released the fourth quarter State of the Internet report that provides insights into global connectivity
and Internet metrics, including connection speeds, attack traffic, broadband adoption and Internet
availability. The report indicated that South Africa’s broadband speeds and adoptions are one of the
lowest in the world (Mybroadband, 2012)
89%
68%
65%
50%
42%
36%
2%
F2F
Call centers
Websites/Apps
Social media
SMS
Other
© SAMRA 2017 – Annual Conference Delegate Copy 23
Table 3: Knowledge of Artificial Intelligence and the use thereof in customer feedback
Knowledge of Artificial Intelligence
Sample (n): 251 251
Definition of AI 61.6
Artificial Intelligence in customer feedback 53.4
Table 3 above illustrates respondents’ knowledge regarding Artificial Intelligence and use thereof in
customer feedback. Respondents indicated that they have some knowledge of the word AI (average
of 61.6) but they are relatively unknown of the application of AI in customer feedback.
Figure 11: Words associated with Artificial Intelligence
The respondents were asked to indicate the words they associate with AI (Figure 12), and the top 5
words were as follows (Figure 11): Machine learning (60%), Automation (55%), Computer brain
(53%), Collection of data (49%) and Robots (46%).
Table 4: Perception of aspects regarding Artificial Intelligence
Perception of Artificial Intelligence
Sample (n): 251
Different feedback channels 78.5
Human replies are being perceived as more customer centric than automated replies
from Chatbot 77.1
Indicate the source - AI/Human 71.7
Text and Sentiment analytics could be used to train computers to become more
customer centric in a response to customer feedback. 68.5
Automated feedback communication: Impersonal 64.7
Automated feedback communication: Fast & Efficient 61.0
General customers would automatically know from a response that is was send from
Artificial Intelligence/ Chatbot and not from a human being. 58.7
Automated feedback communication: Customer Centric 51.8
Automated feedback communication: Unethical in response to
queries/complaints/compliments 48.9
Automated feedback communication: Personalised 45.7
© SAMRA 2017 – Annual Conference Delegate Copy 24
Table 4 illustrates how the respondents rated different attributes regarding their perception of AI.
Respondents are aware that they are different channels where one can provide feedback regarding
complaints, compliment or queries (78.5). The other two top rated attributes were: human replies
are being perceived as more customer centric than automated replies from Chatbot (77.1) and the
need of respondents to know the source of any automated response (71.7) – whether or not the
sources is from a human being or AI. Respondents indicated that they do not perceive automated
feedback communications as being customer centric (51.8) and personalised (48.9), creating a need
for any feedback given to customer to become more customer centric. Important to note that
respondents do not necessarily perceive automated feedback communications to be unethical.
Table 5: Preference for Chatbots
Preference to Chatbots
Sample (n): 251
Personalised reply from a human being, rather from a Chatbot. 78.0
Personalised reply from a highly trained Chatbot given the fact that it would provide
me with faster turn-around time, cost saving and efficiency gains 60.2
Personalised reply from highly trained Chatbot within a few seconds 53.1
Importantly, Table 5 refers to respondent’s preference for Automated feedback communication or
in this case, Chatbots. The figure clearly indicates that respondents would prefer to receive a
personalised response from a human being, rather from a Chatbot (78.0) and that they might not be
important to receive a personalised response within a few seconds (53.1). Respondents did indicate
that a personalised response could be valuable given the fact that it would provide me with faster
turn-around time, cost saving and efficiency gains (60.2).
4.1.2 Experimental findings
The following section provides the reader with the experimental findings as retrieved from the
empirical analysis
Figure 12 below shows the training and testing accuracy of the ANN over time. From the graph it is
easy to conclude that the ANN was able to learn how to accurately measure the CCS of responses
based just on the input/criterion as identified by the authors in Section 2.7. At first, the ANN was
able to predict the correct CCS score of a response, 67% of the time. At the end of training, the
ANN was able to accurately predict almost 80% of CCS scores.
Figure 12: Training and Testing accuracy of the ANN
© SAMRA 2017 – Annual Conference Delegate Copy 25
A very interesting finding can be seen in the error of the ANN over time. It is clear from Figure 13
that the error decreased in the start of the training process, but it is expected from the nature of the
experiment that overfitting will take place on a small dataset. This can be concluded from the
polynomial output of the error function.
Figure 13: MSE of ANN over time
In conclusion, it has been shown that a basic set of criterion can be used to train an ANN to
accurately predict the CCS of a response, given the guidance of an expert.
4.2 Practical Application of Research Findings
In the digital age it seems that there is a notion that technology is the be-all and end-all answer,
whilst it may be true for convenience, it seems that customers truly are not guaranteed with a more
customer centric feedback response and more is needed. The findings of a recent study by Verint
International (Verint Systems Inc., 2016) corroborates the findings of our survey and underlines the
recent trend towards more human centric interaction with clients. Amongst other very interesting
findings the following are very relevant and provides further practical application of the
experimental premise of this paper:
Figure 14: Digital Tipping Point – Verint Survey confirms need for human centric interaction
(Verint Systems Inc., 2016)
It is particular importance to note that the practical application of our research addresses the unique
interplay between human and artificial intelligence. Context of interaction plays a role in the need
© SAMRA 2017 – Annual Conference Delegate Copy 26
for more human centric interaction and it seems that customer still have a long way to go before
trusting a pure artificial source of reply (see below).
Figure 15: Man or Machine – The Digital Tipping Point in Context (Verint Systems Inc.,
2016)
This paper has shown that Chatbots and technology will be increasingly used to scale customer
feedback, whilst it does not directly improve on the customer centric outcome of the reply and there
is a substantial contribution that Neural Network machine learning can make by using Text and
Sentiment analytics to better respond to clients in cases where the technology does have a benefit of
speed, scale and agility.
4.3 Limitations of the Research
The paper deliberately focusses on a very specific challenge in the human-artificial interaction
within the domain of customer feedback and successfully demonstrated that the market research
methods applied within text and sentiment analytics have a sufficient share foundation with Natural
Language Processing, Machine learning and this Artificial Intelligence.
The paper does not purport to set up the ultimate “Turing” test for customer centric artificial
intelligence and is therefore restricted to the field of customer feedback. The research is also further
limited in sample coverage across multiple industries, cross cultural (with specific reference to
language and semantic challenges). The deep learning ANN model tested in this paper is also very
experimental in nature and should be expanded with much larger data sets, variety of industries,
different feedback challenges as well as the comparison between different languages.
© SAMRA 2017 – Annual Conference Delegate Copy 27
4.4 Recommendations for Future Research
Future multi-scientific domain research (with specific reference to market research, text and
sentiment analytics and artificial intelligence) should be encouraged and research funding should be
sought for this highly futuristic, yet very practical applied form of research. Many unanswered
questions remain and should be the focus of a future research agenda in this blended conceptual
research domain. We should look for better ways to address the human-artificial interaction in order
to not only improve efficiency and speed, but also customer experience and ultimately true
customer satisfaction in the omni-channel, omni-tech, omni-product world that we increasingly face
in future.
5. Conclusion
Customer Experience will be the most important strategic focus for companies in the years to come
and digital technology most certainly will continue to have a profound impact on how organisations
interact with customers. The fact that customers in the digital age still (and seemingly increasing)
prefer a form of human interaction in the engagement with companies that they do business with
provides a very unique challenge of convergence of intelligence systems. Artificial Intelligence is at
the brink of another industrial revolution in terms of disruptive innovation and impact on how we
will interact and engage with customers. The converged world of customer intelligence, emotional
customer experience embedded in customer sentiment and more intelligent artificial intelligence
will slingshot the breakthrough disruptions to come. This paper respectfully asks the question to
what degree can the market research industry cooperate, and contribute to this eminent and exciting
new future of integrated scientific fields of customer centric artificial intelligence and customer
interaction.
I am Customer Centric i-Robot – signing off. Thank you.
© SAMRA 2017 – Annual Conference Delegate Copy 28
6. References
Ahiqvist, Bäck, Halonen & Heinonen, 2008. Social media road maps exploring the futures
triggered by social media. [Online] Available at:
http://www.vtt.fi/inf/pdf/tiedotteet/2008/T2454.pdf [Accessed 2 December 2012].
Aula, P., 2010. Social media, reputation risk and ambient publicity. Strategy & Leadership, 38(6),
pp. 43-46.
Beavers, A., 2013. Alan Turing: Mathematical Mechanist. s.l.:Waltham: Elsevier.
Boyd, D. M. & Ellison, N. B., 2007. Social Network Sites: Definition, History and Scholarship..
Journal of Computer-Mediated Communication, 13(1), pp. 210-230..
CGAP, 2017. CGAP. [Online] Available at: (http://www.cgap.org/site-
search/pillars%20of%20customer%20centricity) [Accessed 23 April 2017].
Ciotti, G., 2016. The 7 Best Ways to Gather Customer Feedback. [Online] Available at:
https://www.helpscout.net/blog/customer-feedback/ [Accessed 16 April 2017].
Clarabridge, 2017. Clarabridge. [Online] Available at: http://cxsocial.clarabridge.com/social-
listening/ [Accessed 20 April 2017].
Conti, M., 2016. The Incredible Inventions of Intuitive AI. [Online] Available at:
https://www.ted.com/talks/maurice_conti_the_incredible_inventions_of_intuitive_ai [Accessed
April 2017].
Cooper, D. R. & Schindler, P. S., 2011. Business Research Methods. International ed. s.l.:McGraw-
Hill.
De Vos, A. S., Strydom, H., Fouche, C. B. & Delport, C. S., 2012. Research at grassroots level. 4
ed. Pretoria: van Schaik.
DeepMind, 2016. AlphaGo. [Online] Available at: https://deepmind.com/research/alphago/
Engelbrecht, A. P., 2007. Computational intelligence: an introduction. s.l.:John Wiley \& Sons.
Euromonitor, 2016. Euromonitor International. [Online] Available at: EUROMONITOR
INTERNATIONAL, 2016. http://www.euromonitor.com/new-concepts-in-consumer-foodservice-
modern-technology-and-alternative-formats/report [Accessed 25 January 2017].
Fricker, R. D. & Schonlau, M., 2002. Advantages and Disadvantages of Internet Research Surveys:
Evidence from the Literature. Field Methods, 14(4), pp. 347-367.
Gallaugher, J. & Ransbotham, S., 2010. Social media and customer dialog management at
starbucks. MIS Quarterly Executive, 9(4), pp. 197-212.
Gelston, L., 2017. Chat with SpeakEasy AI. [Online] Available at:
http://lauragelston.ghost.io/speakeasy/
Goldstruck, A. & du Plessis, T., 2017. World Wide Worx. [Online] Available at:
http://www.worldwideworx.com/wp-content/uploads/2016/09/Social-Media-2017-Executive-
Summary.pdf [Accessed 12 March 2017].
Google, n.d. Sequence-2-Sequence models. [Online] Available at:
https://www.tensorflow.org/tutorials/seq2seq
Gordin, M. D., 2016. The Dostoevsky Machine in Georgetown: scientific translation in the Cold
War. Annals of Science, 73(2).
Grishman, R., 1984. Natural Language Processing. Journal of the American Society for Information
Science, Sep.35(5).
Gross, R. & Acquisti, A., 2005. Information Revelation and Privacy in Online Social Networks
(The Facebook case). ACM, pp. 71-80.
Gupta, S. & Zeithaml, V., 2006. Customer Metrics and Their Impact on Financial Performance.
Marketing Science, Nov-Dec.Volume 6.
Hale, T., 2016. Stephen Hawking Has Another Message For The Rest Of Humanity. [Online]
Available at: http://www.iflscience.com/editors-blog/stephen-hawking-has-another-message-for-
the-rest-of-humanity/ [Accessed April 2017].
Harris, L. & Rae, A., 2009. Social networks: The future of marketing for small businesses. Journal
of Business Strategy, 30(5), pp. 24-31.
© SAMRA 2017 – Annual Conference Delegate Copy 29
Jalil, M., 2013. Practical Guidelines for conducting research: Summarizing good research practice
in line with the DCED Standard, Donor Committee for Enterprise Development. [Online]
Available at: http://www.enterprise-development.org/page [Accessed 4 August 2014].
Kang, J., 2011. Social media marketing in the hospitality industry: The role of benefits in increasing
brand community, Iowa: Iowa State University.
Kaplan, A. M. & Haenlein, M., 2010. Users of the world, Unite! The challenges and opportunities
of Social Media. Business Horizons, 53(1), pp. 59-68.
Kietzman, Hermkens, McCarthy, Silvestre, 2011. Social media? Get serious! Understanding the
functional building blocks of social media. Business Horizons, 54(1), pp. 241-251.
Krasne, A., 2017. Marketing.Twitter. [Online] Available at:
https://marketing.twitter.com/na/en/insights/twitter-transforms-conversations-between-companies-
and-customers.html [Accessed 20 April 2017].
Leather, D., 2013. The Customer-Centric Blueprint. Johannesburg: REAP Publishing.
Leedy, P. D. & Ormrod, J. E., 2010. Practical research: Planning and design. 8 ed. New York:
Pearson Prentice Hall.
Leung; Law; van Hoof; Buhalis, 2013. Social media in Tourism and Hospitality: A Literature
Review. Journal of Travel & Tourism Marketing, 30(1), pp. 3-22.
Mangold, G. & Faulds, D., 2009. Social media: The new hybrid element of the promotion mix.
Business Horizons,. Business Horizon, 52(1), pp. 357-365..
Merriam-Webster, n.d. Artificial Intelligence. [Online] Available at: https://www.merriam-
webster.com/dictionary/artificial%20intelligence
Murphy, L., 2016. Grit Report - Greenbook Research Industry Trends Report, s.l.: s.n.
Mybroadband, 2012. My broadband. [Online] Available at:
http://mybroadband.co.za/news/broadband/101370-south-africas-broadband-shocker.html
[Accessed 3 June 2015].
Ngai, Tao, Moon, 2015. Social media research: Theories, constructs, and conceptual frameworks.
International Journal of Information Management, 35(1), pp. 33-44.
O'Hara, C., 2013. Forbes. [Online] Available at: Online reputation management, sometimes
abbreviated as ORM, focuses on the management of product and service search website results.[2]
Ethical grey areas include mug shot removal sites, astroturfing review sites, censoring negative
complaints, and using [Accessed 23 April 2017].
Olah, C., 2015. Understanding LSTM Networks. [Online] Available at:
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
Omar, M. S., Ariffin, H. F. & Ahmad, R., 2016. Service Quality, Customers’ Satisfaction and the
Moderating Effects. Social and Behavioral Sciences, 224(1), pp. 384-392.
Pillay, C., 2014. SME South Africa. [Online] Available at: http://www.smesouthafrica.co.za/5-of-
SAs-best-performing-home-grown-food-franchises/ [Accessed 25 January 2017].
Platt, P. & Schnoes, K., 2014. Customer Engagement from the Consumer’s Perspective. [Online]
Available at: http://www.rosetta.com/reports/customer-engagement-rosetta-consulting-
study/customer-engagement-from-the-consumers-perspective/
Rakitianskaia, A. & Engelbrecht, A. P., 2009. Training neural networks with PSO in dynamic
environments. s.l., s.n., pp. 667-673.
Rushe, D., 2014. Facebook posts record quarterly results and report $1.5bn profit for 2013.
[Online] Available at: www.theguardian.com/technology/2014/jan/29/facebook-record-quarterly-
results [Accessed 12 January 2015].
Rye, K. & Han, H., 2009. Influence of the Quality of Food, Service, and Physical Environment on
Customer Satisfaction and Behavioral Intention in Quick-Casual Restaurants: Moderating Role of
Perceived Price. Journal of Hospitality & Tourism Research, 34(3), p. 329.
Salkind, J. N., 2012. Exploring research. 8 ed. New York: Pearson Prestige Hall.
Seth, B., 2013. DO NOT REPLY — The best way to tell customers you DGAF. [Online] Available
at: http://subimage.com/blog/2013/12/05/do-not-reply-means-you-dgaf/#.WPNcTlOGO2w
[Accessed Apr 2017].
© SAMRA 2017 – Annual Conference Delegate Copy 30
Shah, D. et al., 2006. The Path to Customer Centricity. Journal of Service Research, 9(2).
Silver, D. 2016. Mastering the game of Go with deep neural networks and tree search. Nature,
529(7587), pp. 484-489.
Sorofman, J., 2014. Agenda Overview for Customer Experience, 2015, s.l.: Gartner, Inc..
Teilo, S., 2016. Uservoice. [Online] Available at: http://community.uservoice.com/blog/customer-
feedback-best-practices/ [Accessed 16 April 2017].
Turing, A. M., 1950. Computing Machinery and Intelligence. Mind. A Quarterly Review of
Psychology & Philosophy, LIX(236).
Van Mierlo, T., 2014. The 1% rule in four digital health social networks: an observational study.
Journal of medical Internet research, 16(2).
Verhoef, P; Lemon, K; Parasuraman, A; Roggeveen, A; Tsiros, M; Schlesinger, L, 2009. Customer
Experience Creation: Determinants, Dynamics and Management Strategies. Journal of Retailing,
85(1).
Verint Systems Inc., 2016. The Digital Tipping Point: How do organiza ons balance the demands
for digital and human customer service?, s.l.: Verint.
Wakeman, D. &., 2016. Social Media Marketing Communications Effect on Attitudes Among
Millennials in South Africa. The African Journal of Information Systems, 8(2).
Walliman, N., 2011. Your research project. London: Sage.
Zeithalm, V. A., Berry, L. L. & Parasuruman, A., 1996. The behavioural consequences of service
quality. The Journal of Marketing, 60(2), pp. 31-46.
© SAMRA 2017 – Annual Conference Delegate Copy 31
7. Appendix A: Email Invitation
© SAMRA 2017 – Annual Conference Delegate Copy 32
8. Appendix B: Glossary of Natural Language Processing Terms
The following Glossary of Terms (alphabetical) are relevant to Natural Language Processing and
Text Analytics (Clarabridge, 2017) (Gordin, 2016):
Automatic Summarization & Categorization. NLP can produce a readable summary of a
piece of text such as a sentence, paragraph or even longer forms of text data. The automation of
content analysis of large volumes of text data is an obvious advantage.
Co-reference resolution. Ability to determine which words refer to the same objects Anaphora
resolution is a good example of this capability – NLP can match pronouns with the nouns or
names to which they refer. For example: “The car was really great; it [the anaphor] cruised
along like it was gliding through the air.”
Discourse analysis involves a range of specific NLP tasks, such as identifying the discourse
structure of the input text (the nature of discourse relationships between sentences - e.g.
elaboration, explanation, contrast). Another discourse analysis task involves recognition and
classification of the text (e.g. yes-no question, content question, statement, assertion, etc.).
Machine translation. The software algorithms used to automatically translate text from one
language to another. This is one of the most difficult challenges of NLP and requires a
comprehensive understanding of the different types of natural (human) knowledge that humans
possess (grammar, semantics, facts about the real world, etc.).
Morphological Segmentation - words can be treated as individual morphemes. In
morphological segmentation, the class of morpheme is firstly identified (i.e. the structure of
words) based on the specific language being analysed and processed (most modern Text
Analytic Software have language detection). The complexity of morphology between language
differ vastly and adds complexity to standardize across different languages.
Named entity recognition (NER) – the NLP capability to determine which items in the text
relate/refer to proper names, such as people or places, and what the type of each such name is
(e.g. person, location, organization). Capitalization of words is a typical indicator of named
entities in languages such as English. For example, “New York” would be identified as the US
city. This functionality becomes very complex for some languages where capitalization is not
used at all or not consistently use to indicate a named entity.
Natural language generation. Convert information from computer databases or semantic
intents into readable human language.
Natural language understanding. The ability to understand sentences and paragraphs through
the identification of the intended semantic from the multiple possible semantics into organized
notations of natural languages concepts. Subjective Yes/No vs. objective True/False is expected
for the construction of a basis of semantics formalization.
Parsing – NLP will do grammatical analysis to determine the parse tree of a sentence or
paragraph of text. Grammar for natural languages is generally ambiguous and sentences mostly
have multiple possible analyses. The machine learning algorithm will have to learn which of the
vast number of meanings are sensible and logic.
Part-of-speech Tagging – in any sentence the Part-of-speech consist of words that can have
different meanings. For example, ‘kind’ can be either a noun (“kind of sweater”) or a verb (“she
was kind to the puppy”).
Question answering. NLP machine learning capability to answer human-based questions
ranging to from specific answers to a question – i.e. “What is the currency of South Africa?” to
open-ended (even deep philosophical questions) can also be answered such as "What is the
meaning of life?"). In recent versions of Siri, Apple have adjusted the Siri-algorithms to answer
very complex controversial questions in a subtler way to prevent anyone from being offended.
© SAMRA 2017 – Annual Conference Delegate Copy 33
Relationship extraction. NLP can identify the relationships among named entities (e.g. who is
married to whom, a person living in Johannesburg will be in South Africa).
Sentence breaking (or sentence boundary disambiguation). It is important to identify the
sentence boundaries - often indicated by periods or other sentence punctuation. The algorithms
become very complex since the use of different forms of punctuation differ between languages
and could even serve different purposes (e.g. a full stop used in an abbreviation does not mean
the end of a sentence).
Sentiment analysis. A more recent and very useful functionality of NLP is the ability to extract
subjective information such as the underlying emotions of a sentence. The most basic form is to
identify the polarity (either positive or negative). The more advanced algorithms would allow
for culturally rich emotional constructs on different levels of sentiment (providing more
granular quantification of subjective meaning).
Speech recognition & Analysis. The most advance and recent technological development in
the field of NLP is Speech Analytics (or sometimes referred to as Speech-to-Text). This is the
capability of the NLP model to digitally transcribe the spoken word (sound clip) into written
text. Once the text data is available NLP can use all the available algorithms to apply text
analytics to the spoken sound clip. The most basic form is pure word-sound recognition through
a fixed wave signature of specific words and or phrases.
© SAMRA 2017 – Annual Conference Delegate Copy 34
9. Appendix C: Glossary of Artificial Intelligence Terms
Nodes: Representing the Artificial Neurons.
Connections: Representing the relationship between Artificial Neurons.
Artificial Neural Network (ANN): A collection of Nodes and Connections in a specific
arrangement.
Weights: Serve as signal strength modifiers. Signals are passed from one Node to another, via
their relationship, indicated by Connections.
Net Input: Indicates the total weighted input to a Node from all its preceding Nodes.
Activation Function: Serves as the mechanism for determining when a Node fires a signal.
Layers: Groups of Nodes are assembled in different layers, each layer is fully connected,
meaning all Nodes in one layer are connected to all Nodes in the succeeding layer.
Topology: Structure of layers in the ANN. Different structures provide different abilities.
Feed Forward: The process of flowing data from one end of the ANN all the way to the other.
This process involves loading input data into the ANN via the first (input) layer, calculation net
input signals from each layer, feeding that net input to the Activation Function of a Neuron,
retrieving the output to the next layer, all the way to the last (output) layer, which is finally used
as the evaluated result of the ANN.
Fitness Function: Also referred to as Optimization Function or Error Function. A Fitness
Function is a mechanism that measures the degree to which an ANN can represent the training
data. One such an indicator is The Sum Squared Error (SSE), which sums the difference
between the actual output and target output of all out Nodes, and then squares it to get a total
error indication. In this explanation, actual output refers to the evaluated result by the ANN,
while target output refers to the training data and the correct outcome that relates to that pattern.
Heuristic: Search algorithm to be used as a mechanism to train the ANN to better fit the input
(training data). The Heuristic makes use of the Fitness Function as a guide to finding the best
combination of weights.
Back Propagation: The process of adjusting the weights of the ANN according to the guide
provided by the Heuristic.
Gradient Descent Optimizer: A Mathematical Continuous Function Optimizer (Heuristic) that
uses the derivative of the error (relative to each Node’s contribution to the error) as discussed
above.
Epoch: The number of iterations to which the ANN is exposed to training data.