ORIGINAL ARTICLE

Multi-class approach for user behavior prediction using deeplearning framework on twitter election dataset

Krishna Kumar Mohbey1

Received: 29 August 2019 /Accepted: 7 October 2019# Springer Nature Switzerland AG 2019

AbstractAmong the broad assortment of Machine Learning approaches, deep learning has recently attracted attention particularly in thedomain of user behavior analysis. The notion to study user behavior from the unstructured tweets shared on social media is aninteresting yet challenging task. A social platform such as Twitter yield access to the unprompted views of the wide-ranging userson particular events like election. These views cater government and corporates to remold strategies, assess the areas where bettermeasures need to be put forward and monitor common opinion. With the advent of the general election in India (largestdemocracy) people tend to articulate their views or issues. Tweets related to general elections 2019 of India is used as datacorpus for the study. Multi-class classification fabricated with novel deep learning approach is implemented to analyses the useropinion. Here, we have used nine different classes, which is representing larger issues in the nation for election agenda.Moreover,comparative analysis between tradition approaches such as Naïve Bayes, SVM, decision tree, logistic regression and employedapproach with deep learning method is presented. Experimental results revels that the proposed model can reach up to 98.70%accuracy on multiclass based prediction in machine learning. The results assist the government and businesses to know aboutgrave issue offering a shot to revise strategic policy and make welfare scheme program.

Keywords Behavior prediction .Multi-class classification . Deep learning . Twitter . General election 2019 .Machine learning

1 Introduction

The advent of deep learning has triggered a paradigm shift inMachine Learning with its remarkable performance.Numerous researches have been done to improve the accuracyof user behavior mining from simple linear model to complexnew deep approaches. This disruptive approach has widerange of application including user behavior mining (Alharbi& de Doncker, 2019), traffic control (Ouyang et al., 2015),sentiment analysis (Shuang et al., 2019), pattern mining (Zhuet al., 2019), semantic analysis (Kumar et al., 2018).

Multiple social media platforms available today such asFacebook,WhatsApp, and Twitter, etc. are utilized to circulateinformation as well as user view. Twitter can also be used forsentiment analysis or opinion mining (Mäntylä et al., 2018).Sentiment analysis is a computational process in which user’s

opinion can be identified towards a topic, subject, product orservices. Twitter is chosen as a data source in our study be-cause of its popularity, ease of use to express opinion andbrevity of tweets up to 280 characters. It providesTwitterAPI to crawl on tweets according to a particularhashtag (Cha et al., 2010; Trusov et al., 2010; Messias et al.,2013) (#IndianElection2019, #2019Election). Several ap-proaches are already proposed for sentiment analysis usingTwitter data such as unigram (Andrea et al., 2019),Word2Vec (Kapočiute-Dzikiene et al., 2019), Part-of-Speech(Agarwal et al., 2011), emoticon mining (Kumar & Sebastian,2012), N-gram (Pak & Paroubek, 2010), parse tree (Martínez-Cámara et al., 2012; Saif et al., 2012).

Most of the available models classify user’s tweet into pos-itive and negative classes using traditional approaches likenaïve Bayes, SVM, decision tree. Sharma and Moh (Sharma& Moh, 2016) has performed twitter user opinion analysis onIndian election using traditional classifier algorithm which hasaccomplished an accuracy of 62.1% and 78.4% in NB andSVM respectively. To improve the performance, it is neededto classify tweet on more than two classes, i.e., multi-class(Bouazizi & Ohtsuki, 2017; Bouazizi & Ohtsuki, 2018) and

* Krishna Kumar Mohbeykmohbey@gmail.com

1 Department of Computer Science, Central University of Rajasthan,Ajmer, Rajasthan, India

https://doi.org/10.1007/s42488-019-00013-yJournal of Data, Information and Management (2020) 2:1–14

/Published online: 29 October 2019

in deep learning framework if a model classifies tweets inmore than two opinions that are known as multi-class senti-ment analysis.

User behavior analysis helps companies and government toget an insight into people thinking. The proposed approachhelps government to identify the major national issues provid-ing a golden opportunity to take positive step to resolve theproblems of the masses.

In this study, we have collected the user’s opinion such astweets from Twitter. All the tweets are related to general elec-tions 2019 of India. With the announcement of election, peo-ple started posting their views related to different issues na-tion-wide. Based on collected tweets, we have identified ma-jor significant issues. In election 2019, peoples mostly tweetedregarding agriculture, infrastructure development, education,employment, religion, GST, demonetization, and women em-powerment issues. Infrastructure development also includeselectrical issue, water issue, and road issues and so on.

In this paper, we have proposed a multiclass approach indeep learning framework to classify a tweet in one of theavailable class. We have defined nine different classes of theelection issues that were mostly discussed by the politicalparties as well as persons. These classes are agriculture, infra-structure development, education, employment, religion,GST, demonetization, women empowerment, and others.Another class is used for all the other issues except aboveeight. Furthermore, proposed model is compared with tradi-tional approaches like naïve Bayes, SVM, decision tree, ran-dom forest to signify the importance of the novel deep learn-ing approach.

Figure 1 shows a word cloud of terms that was mostly usedduring these election times. The contribution of this work is asfollows:

i) We proposed a multiclass approach for user’s viewsclassification using general election 2019 tweets.

ii) Implemented on the novel deep-learning frame for clas-sification on massive real-time upcoming tweets.

iii) We can able to identify what are the major issues thatwere discussed during this election time.

iv) User’s sentiments are identified about the general elec-tions 2019.

v) Comparison of the deep learning with a traditional ap-proach to highlight the better accuracy in former one.

The remainder of this paper is structured as follows.Section 2 discusses the related study of sentiment analysis aswell as a description of multiclass classification. Section 3introduces the notion and definition of deep learning, itstuning parameters, and overfitting mechanisms. Traditionaltweet classification approach such as naïve Bayes, SVM, ran-dom forest and decision tree are illustrated in section 4. Datacollection and pre-processing steps along with proposed userbehavior prediction framework in deep learning withmulticlass identification algorithm are described in section 5.Section 6 reports experimental results and discussion aboutthe proposed work. It also represents the comparison of deeplearning approaches with traditional methods. The conclusionand future work are included in section 7.

2 Related work

Recently, deep learning methods have attained the interest ofboth academia and industry. This machine learning fork hasversatile application including sentiment analysis and textclassification. In the proposed approach, we aim to identifythe most dominant sentiment from a given tweet. This ap-proach finds a more accurate feeling instead of telling whetherit is positive, negative or neutral (Rosenthal et al., 2017).Based on the multiclass classification in deep learning, a clas-sified tweet represents the actual sense of the proposed tweet.Multiclass sentiment analysis draws out the most accurateclass out of the available classes.

Twitter is one of the popular social media platforms thatprovide the facility of online microblogging services. Some ofthe works identify hidden communities and relationships be-tween different users (Java et al., 2007; Kuncheva&Montana,2015; Bizid et al., 2015). Twitter is a widely accessible plat-form and used for opinion mining from available tweets(O’Connor et al., 2010). In this regard, Achananuparp et al.(Achananuparp et al., 2012) study user behavior usingFig. 1 Mostly used terms in general elections 2019

J. of Data, Inf. and Manag. (2020) 2:1–142

Table1

Aglance

ofworkby

variousauthors

Reference

DataSource

Sam

ples

size

Class

Approach

Accuracy

Rem

ark

Yin

etal.(Yeetal.,2012)2012

New

sarticle

208,802

Multi8classes-touched,surprise,

empathy,boredom,sadness,

amusem

ent,anger,andwarmness

Traditio

nal-SV

MSVM-50.07%

Random

k-labelsetsclassifier

(RAkE

L)performsgood.

Nogueiraetal.(Dos

Santos

&Gatti,

2014)2014

Twitter

(Pang&

Lee,2005;

Goetal.,2009)

96,498

Binary-positiv

eandnegativ

eDeepandtraditional

charSC

NN-86.4%

NB-82.7%

SVM-82.2%

MaxEnt-83%

charSC

NNmodelisproposed.

Stojanovskietal.(Stojanovski

etal.,2015)2015

Twitter

20,632

Binaryand5Multi-class

Deep-

employsboth

CNN&

LSTN

84.8%

Finki,proposed

modelworks

well

onquantificationtask.

Vateekuletal.(Vateekul&

Koomsubha,2016)2016

Twitter

3,813,173

Binary-positiv

e&

negativ

eDeep-

LST

N&

Dynam

icDNN

LST

N-75.30%

DDNN-75.35%

NB-74.04%

SVM-74.71%

Max

Ent-75.13%

BestisDDNNfollo

wed

byLSTN.

Dzinienietal.(K

apočiute-

Dzikieneetal.,2019)2019

Lith

uanian

Internetcommenta

10,570

Multi-

Positiv

e,NegativeandNeutral

Traditio

nal-SV

M,

multin

omialN

BDeep-

CNN,L

STN

NBM-735%

SVM-72.4%

CNN-70.6%

UsedWord2Vec

andFastTextb

areused

forem

bedding.

Zhang

etal.(Zhang

etal.,2019)2019

Twitter

6951

Multi-Po

sitiv

e,Negative,Neutral

DeepLearning

71.25%

Multi-layerAttentionCNN

(Mul-AT-CNN)isproposed.

aLietuvosRytas

newsportalhttps://w

ww.lrytas.lt/

bFastText

https://fasttext.cc/docs/en/craw

l-vectors.html

J. of Data, Inf. and Manag. (2020) 2:1–14 3

microblogging services such as Twitter. In this study, theyhave used tweet and retweet both to identify user behavior,for so, have proposed several models (Achananuparp et al.,2012). Kassens-Noor (Kassens-Noor, 2012) uses Twitter forteaching and learning. In their study, they have exploredtweeter as a simple learning tool that is efficiently used foreducation. Twitter can be investigated in different areas suchas motivation, academic and psychological development ofstudents. In this context, Dhir et al. (Dhir et al., 2013) focusedon the twitter impact on different areas for educationimprovements.

Multiclass classification refers to the identification of spe-cific sense available in a text or tweet. It extracts an accurateview of the user instead of getting positive, negative, andneutral polarity. Ye et al. proposed an approach that usesmulticlass classification for news article classification; in thiswork, they classified a news article into appropriate category(Ye et al., 2012). Lin et al. proposed an approach for extractingsimilarity feature from available tweets (Lin et al., 2007; Lin

et al., 2008). Krawczyk et al. tackled the problem of imbal-anced tweets collected from Twitter (Krawczyk et al., 2017).Sentiment analysis is explored in various domains withTwitter (Mäntylä et al., 2018). Zainuddin et al. proposed ahybrid sentiment classification, in which they use twitter attri-butes as a feature to improve aspect-based analysis categoriz-ing tweets for different elements (Zainuddin et al., 2017).

Deep learning as a subset of machine learning has shownits capabilities in both supervised (Qin et al., 2018) and unsu-pervised learning (Shuang et al., 2019). It uses activation anddropout feature to improve accuracy. Bhoi and Joshi sug-gested various sentiment classification approaches using ma-chine learning and deep learning concepts to enhance aspect-based sentiment analysis (Bhoi & Joshi, 2018). Zhang et al.(Zhang et al., 2019) have proposed the model for sentimentanalysis along with multi-neural network. Twitter data is ex-plored to reap semantic, syntactic and context information.The suggested model classifies dataset into positive, negativeand neutral classes using multi-layer neural network.

Fig. 3 Basic structure of a neuralnetwork

Fig. 2 Simple artificial neuronarchitecture

J. of Data, Inf. and Manag. (2020) 2:1–144

Sentiment polarity is computed through attention function.Comparison of proposed approach is done with various deepmodel gaining an accuracy of 71.25%. Dzikiene et al.(Kapočiute-Dzikiene et al., 2019) have used the Word2vecapproach to classify in multinomial classes where best deepleaning result was 70.6%. Quyang et al. (Ouyang et al., 2015)used movie review excerpts to segregate data under five la-bels: negative, somewhat negative, neural, somewhat positiveand positive. Furthermore, also used the Parametric RectifiedLinear Unit (PReLU), Normalization and Dropout technologyto improve the performance and generalizability of deep learn-ing proposed model.

Stojanovski et al. in (Stojanovski et al., 2015) have con-structed a system called finki using Gated RNN and CNN andfor class prediction has exercised softmax function. The net-work is skilled with GloVe (Global Vectors) word embeddingon above of tweets representation with 300 dimensionalities.RNN and CNN permit the tweets of variable size and itsconcatenation to generate fix size sample which is given todeep layers for sentiment analysis. The model classifiesdataset into binary or multiple class (5 labels: very negative,negative, neutral, positive and very positive), which was asdeclared as second best in SemEval-2016 competition.

As stated in (Dos Santos &Gatti, 2014), a DNN are appliedon Stanford Sentiment Treebank (Pang& Lee, 2005) of moviereview dataset and Stanford Twitter data (Go et al., 2009) forclassification task in DNN. The network can be worked uponsentence level, word level, character level and achieve outputfor binary classification (positive or negative).

Some works use, tweets as the primary source for theirexperiment, meanwhile machine learning algorithms are usedfor preparing a model (Manuel et al., 2010). Vateekul et al.(Vateekul & Koomsubha, 2016) have included two modelsDynamic CNN and LSTN for the study of Thai Twitter datafor emotion mining. They have used the popular Word2Vecapproach to divide tweets into positive and negative achievingthe best accuracy of 75.35% surpassing NB, SVM andMaximum Entropy.

Multiple organizations and government are showing theirinterest in user behavior prediction. They are eager to knowabout the needs of the people using which new plan and wel-fare measures can be taken. In (Hodeghatta, 2013; Cabanlit &Espinosa, 2014), presents an approach to mine the user’spoint-of-view about service or product using data frommicroblogging sites.

Summary of different approaches and works by variousauthors is represented in Table 1.

The proposed work is entirely different from availableworks. Its aim to predict user’s actual behavior from tweetsof Lokshabha elections 2019 of India. This work accuratelyclassifies a user tweet in the relevant class with deep learningapproach. It uses a multiclass classification concept to predictthe user’s behavior. We have demonstrated the results of

semantic analyzers and their machine learning validation intabular formats and graphs to depict a complete picture ofaccuracy gained. Besides, have analyzed SVM, NB and deci-sion tree traditional techniques on the unstructured real-timeelection tweets. We have attained the accuracy of 98.70% issuggested model that is outperforming all the competingmethods.

3 Deep learning framework

Deep learning has become the most crucial research area inpresent days. It is a sub-category of machine learning thatapplies neural networks to study multiple layers of abstraction(Huang, 2004). Deep learning architecture consists of variouslayers. These various layers are used for feature extraction andtransformation. The output from the previous layer is feed intothe successive ones. Deep networks possess the capability ofdeveloping algorithms to handle tasks such as opinion mining(Alharbi & de Doncker, 2019). In deep learning, deep refers tothe multiple transformation and representation of data frominput layer to output layer. There are multiple hidden layersbetween input and output layers. In present days deep learningapproaches are available for big data handling and uses

Fig. 4 Tweet collection process using Twitter API

J. of Data, Inf. and Manag. (2020) 2:1–14 5

graphic processing units (GPU). Deep learning also uses dif-ferent software frameworks such as TensorFlow, Keras,Teano, and Pytorch. Convolutional Neural Network (CNN),Recurrent Neural Network (RNN), Deep belief, Long-ShortTermNetwork (LSTN) is some of deep learningmodels. CNNis feed-forward neural network which only regards currentinput while RNN takes into account current input along withpreviously received inputs. RNN has the proficiency to mem-orize prior input due to internal memory.

3.1 Neural network

Neural networks are computational models and work on neu-rons. They work in the same fashion as human brain neuronswork. They try to observe the behavior of neurons and transferto the next level of abstraction. A neuron is a basic unit ofprocessing in neural network and it is used as a node in anetwork. Figure 2 shows a simple form of neuron. It isconsisting of input data (a1, a2 …an) which transformed inoutput of another neuron. a0 is a constant value known as biasthat is added to the input of neuron inactivation function. Therelevance of neurons is identified by the input weights in themodel (Hernández-Blanco et al., 2019). An α is the output ofthe neurons that are computed by eq. 1.

α ¼ f ∑ni¼0wi:ai

� � ð1Þ

Where f is the activation function of the neuron.A basic structure of the neural network is shown in Fig. 3.

The first layer is known as the input layer which is used toprovide input data to the network. The prediction of the modelis provided by the output layer. The depth of the network canbe determined by the hidden layers. More hidden layers can beused to learn more complex function. These hidden layersautomatically learn when the model is trained with data.

3.2 Dropout layer

Most deep learning models put into practice the dropout asdifferent layer. It is a regular, densely connected neural net-work layer. It is a mechanism where randomly chosen neuronare dropped or ignored to resolve the problem of overfittingand enhance the generalizability of the network (Srivastavaet al., 2014). This method prohibits the co-adapting of neuronstoo much during training. To use the method, it is prerequisiteto specify the dropout probability. This is the percent of thedropped neuron.

3.3 Activation function

In, machine learning activation function is also known as thetransfer function, which is applied to compute the output ofevery layer of the neural network. In previous year’s sigmoid,hyperbolic tangent or linear activation functions were used in

Fig. 5 Sample of collected tweets

Table 2 Tweet examples withidentified categories SN Tweet Category

1. big change in MP road Infrastructure Development

2. Former dead in village devas Agriculture

3. your hard earned money has been wasted by the previous governments Demonetization

4. They took us back with religion and caste politics Religion

5. This time I want a govt that generate jobs Employment

J. of Data, Inf. and Manag. (2020) 2:1–146

neural networks. Recently, in deep learning softmax or recti-fier linear unit (ReLu) are used.

In the suggested model, we have deployed softmax activa-tion function used for the output of classification neural net-work. The calculation if the softmax function is:

The softmax is calculated with Z as an input vector is asfollows:

σ Zð Þ ¼ eZi

∑kj¼I eZj

; for i ¼ 1…::k and Z ¼ Zi……Zk ð2Þ

3.4 Weighted loss function

Mostly text classification has imbalance multiple target data,resulting in the inaccurate result similar to majority vote class.This problem has been tacked by the weighted loss function,which is a class balancing approach. The loss function for x-i can be evaluated as:

L xið Þ ¼ wi ∑Z

z¼11yi¼zlog piz ð3Þ

Where 1 means indicator function. The input xi has weightwiwhich depends upon ground truth label yi. pi is the output ofprediction layer which is the probability of input xi Classifiedfor class z.

3.5 Early stopping

One of the forms of regularization use to evade overfitting isearly stopping by training the model with a repetitive method.Such an approach improves the performance of data. Thereare many hyperparameters, one of which is how many timesthe full passes of data sets need to be used. Such parametershelp to tune the accuracy of this approach.

4 Traditional approaches

To classify tweets in multiclass, we have used various ma-chine learning approaches. These classifiers are developed

using Python programming language. In this work, we haveused the following well-known traditional classificationtechniques.

4.1 Naïve Bayes classifier

It is a well-known probabilistic classifier and comes under themachine learning technique. This classifier uses Bayes theo-rem of conditional probability (Rane & Kumar, 2018). Thisclassifier passes each tweet and calculates posterior probabil-ity for every class, the final class assignment to the tweet whohas the highest posterior probability. The equation for naïveBayes classification is described as-

PCx

� �¼

PxC

� �:P Cð Þ

P xð Þ ð4Þ

Where.

C specified classx tweet used for classificationP(C)and P(x) prior probabilitiesP C

x

� �posterior probability

Given the Naïve assumption which states that a data pointX = {x1, x2, x3, ….., xi}, the likelihood of each of its feature(independent) occurring in a given class, the equation can berewritten as:

PCx

� �¼ P Cð Þ:Π P

xiC

ð5Þ

Fig. 6 User behavior prediction

Table 3 Parameters with its values

S No. Parameters Values

1 Drop rate 0.4

2 Batch size 512

3 Activation function relu, softmax

4 Regularizer 0.0001

5 Dimension of vector 300

J. of Data, Inf. and Manag. (2020) 2:1–14 7

4.2 SVM classifier

This supervised classification model was developed byVapnik (Cortes & Vapnik, 1995). This algorithm uses a hy-perplane separation strategy. Based on training data, this clas-sifier categorizes test data into an optimal hyperplane. Herewe have used multi-class SVM for tweets classification. Thisis more effective for high dimensional space. When given afinite set of the training set, each of them marked as belongingto one of the available categories, the SVM classifier frames amodel that allocates new example to one of these categories.

4.3 Random forest classifier

Random forests are the kind of ensemble learning method thatrenders predictions by averaging the predictions of multipleindependent base models. Since its creation by Breiman(Breiman, 1996) the random forests as a classifier for regres-sion and classification purpose has been extremely successful.It consists of a group of tree-like classifier where independentrandom vector are dispersed identically, and every tree cast asingle vote for the most favored class. A random vector isformed that is independent of the earlier vector for the samedistribution resulting in the generation of the tree (Breiman,1996). The random forest has certain advantages over another

classifier such as it doesn’t suffer from the problem ofoverfitting (Hastie et al., 2001), the same random forest algo-rithm can be used for classification and regression, and it canbe used for feature engineering, i.e., selecting the most impor-tant features from the available features of the training set.

4.4 Decision tree classifier

It is a simple popularly used for data classification. It has atree-like structure with internal nodes representing the testconditions and leaf nodes are the class categories. In this clas-sifier, each branch indicates the outcome of the test. It followsa top-down approach to represent classification rules. As perthe class availability, a decision tree generates a series of rulesthat are used to identify a class of a test instance. This classifieruses post pruning approaches to handle overfitting (Nithyassik& Nandhini, 2010).

5 User behavior analysis approach in deeplearning framework

User behavior is complex which is difficult to analyze.However, we have proposed the sentiment analysis approachin deep learning framework. The following procedure is statedbelow: -.

5.1 Data collection

Presently, researchers are interested in identifying the user’sbehavior based on tweets (Mäntylä et al., 2018). To performmulticlass classification, we have absorbed tweets of generalassembly elections 2019 of India. Twitter API is used to col-lect tweets using hashtags such as #IndiaElection2019,

Fig. 7 Tweet categorization

Table 4 Confusion matrix for a tweet

Actual Value

Predicted Value Positive Negative

Positive True Positive (TP) False Positive (FP)

Negative False Negative (FN) True Negative (TN)

J. of Data, Inf. and Manag. (2020) 2:1–148

#2019IndiaDecides and so on. Twitter API required consumerkey, consumer secret, access token, and access token secretkeys. Here we have used R language interface to access tweetusing these keys. All the tweets belong to a 1-month post, i.e.,March 2019.

Figure 4 shows the complete process of collectingtweets from Twitter using Twitter API and variouskeys.

The collected tweets are noisy and need to apply pre-processing before performing classification and predic-tion task. A sample of collected tweets has shown inFig. 5.

5.2 Pre-processing of collected tweets

Pre-processing is a necessary step before performing actualexperiments on the collected tweets. Collected tweets arenoisy, skewed, and having a lot of stop words.Therefore it is required to clean all the tweets forperforming classification and prediction tasks. Also,Twitter is unstructured and supports multi-languagepost ing, so that data pre-processing has great

importance. In this work, we have collected onlyEnglish tweets. Pre-processing includes stop word removal,stemming, data filtering and feature extractions (Li et al.,2018).

A. Stop word removalIn this step, we have removed all the commonly

occurring words from each of the tweet. For remov-ing stop words, a predefined list of stop words hasbeen used. Each tweet is compared with the avail-able stop word list and matching words are re-moved from that tweet. These words do not con-tribute to the performance of the model.

B. Punctuation removalThere is a lot of punctuation available in tweets which

are of no importance, all the punctuation symbols suchas.,!, ^,?, etc. are withdrawn in this step.

C. URL removalNext, all the URL’s and hyperlinks are deleted from

the tweets.D. Filtering

Hashtags, RT (retweet symbols) query terms, and spe-cial characters are identified and eliminated.

E. StemmingWe have used porter stemmer (Li et al., 2018) to per-

form stemming on the terms of each tweet. In this process,all the words are substituted to the root word.

F. ConvertingSome text mining approaches are case sensitive, which

may treat “Election” and “election” in a different case. Asa result, we have transformed all the tweets to lowercase.

G. Feature generationFeatures are the exact term which is used by the

Table 5 Performance evaluation measures

MatricName

Definition

Accuracy Accuracy ¼ TPþTNTPþTNþFPþFN

Precision Precision ¼ TPTPþFP

Recall Recall ¼ TPTPþFN

F-Measure F−measure ¼ 2*Precision*RecallPrecisionþRecall

AUC AUC ¼ Recall− FPFPþTN þ 1

� �=2

Table 6 Comparison of precisionand recall in deep learning andtraditional approach

Dataset Deep Learning Traditional Model

Proposed model NB SVM Decision

Tree

Random Forest

Pre. Rec. Pre. Rec. Pre. Rec. Pre. Rec. Pre. Rec.

25 K .97 .97 .93 .93 .91 .89 0.96 0.96 0.95 0.94

50 K .99 .99 .95 0.95 .91 .90 0.98 0.98 0.96 0.96

Table 7 Accuracy and AUCcomparison in deep learning andtraditional approach

Dataset Deep Learning Traditional Model

Proposed model NB SVM Decision Tree Random Forest

Acc. AUC Acc. AUC Acc. AUC Acc. AUC Acc. AUC

25 K .9740 .9880 .9271 .8267 .8920 .8005 .9714 .9577 .9404 .8747

50 K .9870 .9934 .9482 .8020 .9002 .7589 .9824 .9650 .9590 .8496

J. of Data, Inf. and Manag. (2020) 2:1–14 9

classifier to perform classification and user behavior pre-diction. We have used the TF-IDF technique for featurevectorization.

5.3 Identification of tweet category

In pre-processing, all tweets are refined. Next is to identify thecategory of each tweet. For category identification, we have

employed a list of predefined terms for each of the class. Theyare defined according to most similar terms matched in theavailable list. For example, if a tweet contains words like“Employment increase the previous year,” then its label willbe “Employment”. To compute the categories of all the avail-able tweets, we have applied the following algorithm.

Algorithm 1. Class Identification

Above algorithm defines the category of all the tweets. Ifthe category is not defined to any tweet, by default, its cate-gory is zero, which is considered as “other” category. Table 2displays an example of tweets with the defined category.

5.4 A proposed approach for behavior predictionusing deep learning

Multiclass user behavior prediction in deep learning frame-work using election tweets have been structured. In this frame-work, as presented in Fig. 6, there are a series of steps from

tweet collection, pre-processing, training model, overfittingmechanism to user behavior prediction.

The first step of this approach is to collect live streamgeneral election 2019 tweets from Twitter. Twitter API aidsin the absorption of tweets with the usage of hashtags. Aftertweet collection, next comes data pre-processing. It includesstop word removal, URL removal, stemming and so on. Thethird step of class identification. All the refined tweets arelabels in one of the available nine classes using algorithm 1,which is discussed in section 5.3. These classes are related tomajor issues of general elections 2019 of India.

J. of Data, Inf. and Manag. (2020) 2:1–1410

5.5 Tuning parameter

Machine learning and deep learning models need to use var-ious parameters effectively. Every model has a set of criticalparameters. Each parameter in the model is set of defaultvalues that can be changed by the programmer as and whenrequired. Every information from the actual globe is distinctand needs to be worked on differently. Now, if the samemodelis applied to different datasets with the same default valueparameters then a good result will not be generated for allcases uniformly. Therefore, we need to modify the parameterin such a manner that for every specified information-set weget for evaluation can be accomplished by the model. Thisparameter value adjustment method is known as parametertuning (Brown & Mues, 2012). For a specified model, thevalue must be selected wisely and adequately.

Linear classifiers such SVM is implemented with stochas-tic gradient descent (SGD) learning approach. In this ap-proach, the gradient of the loss is estimated for each sampleat a time and the model is updated along the way with decreas-ing strength. The SVM classifier1 is tuned with the parametersloss = ‘hinge’, penalty = ‘l2’, alpha = 1e-3, random_state = 42,max_iter = 5, and tol = None.

Decision tree classifier is implemented with the help ofsklearn.tree.DecisionTreeClassifier2 using default parameterssuch as criterion = ‘gini’, splitter = ‘best’, max_depth = None,m i n_ s amp l e s _ s p l i t = 2 , m i n_ s amp l e s _ l e a f = 1 ,min_weight_fraction_leaf = 0.0, max_features = None,r andom_s t a t e = None , max_ l e a f _node s = None ,min_impurity_decrease = 0.0, min_impurity_split = None,class_weight = None, and presort = False.

Similarly random forest classifier is also implemented withthe help of sklearn.ensemble3 class. In this classifier mainlyused parameters and its values are as n_estimators = 100,max_depth = 100, random_state = 0, and bootstrap = True.

After all data preparation comes to a deep learning ap-proach where firstly parameters are initialized. Table 3 repre-sents the parameters with its values.

In deep learning we have employed 128 and 64 denselayers, “rmsprop” optimizer, “categorical_crossentropy” asloss function. We have deployed early stopping to train itera-tively. Next step is to handle the overfitting by application ofregularized and dropout approach. This proposed approach isapplied to training data to create classification model. Testingdata is used to validate this model. Moreover, the model iscompared with traditional classification approaches to showperformance of the suggested model. In this way, we can

gather knowledge about user behavior according to multiclassconcepts.

The classification and prediction steps of the proposedframework describe various observations on elections 2019of India. It includes behavior analysis from available tweetsin deep learning. Also, this framework shows the major issuesthat were discussed during the election and people also postedtweets regarding these issues on Twitter.

6 Experimental results & discussion

This section highlights the comprehensive experimental testscarried out on the above discussed model. We present resultsobtained for multiclass classification using tweets oflokshabha elections 2019 of India. For this work, we collectedtweets using Twitter API. The real-time tweet dataset isdownloaded using different hashtags which includes#Election2019, #IndianElections2019, #IndiaDecides2019and so on. Here approx. 100 k dataset are downloaded forthe experimental purpose. After collecting tweets, we havecategorized each tweet into exactly one of the nine classesusing algorithm 1. Figure 7 shows the categorization of thedownloaded tweets.

In order to boost the accuracy, we have included the tweetsof all classes other than “0” resulting in the dataset of 100 ktweets. This dataset is segregated into 70% for training and30% for testing to evaluate the effectiveness of classifier.

6.1 Environmental setup

For the implementation of the model we have used the archi-tecture of X86–64, Intel (R) Xeon (R) CPU E5–2630 V4 @2.20 GHz with RAM of 64 GB. We have worked with Keraslibrary on python language for deep learning. Additionally,have installed basic packages like pandas, NumPy, re andmany more, for ease of use of language. We have collected100 k unstructured tweets relating to election issues.

6.2 Performance evaluation

The performance of different classifiers is evaluated by prima-ry key performance indicators that are precision, recall, accu-racy, AUC, and f-measure. Table 4 shows the confusion ma-trix for a tweet. These values will be used in efficiency eval-uations. Table 5 depicts various performance parameters withtheir definitions.

The experimental results for classification tasks are pre-sented in Tables 6 and 7. This experiment includes perfor-mance comparison of different supervised approaches in deeplearning and traditional approach of election 2019 tweets. Theeffectiveness of different supervised approaches is comparedby precision, recall, accuracy, and AUC.

1 https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html2 https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html3 https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html

J. of Data, Inf. and Manag. (2020) 2:1–14 11

6.3 Comparison with the various traditionalapproaches

This segment throws the light on the generated result to drawout conclusions. Precision and recall of the constructed ap-proach in deep learning’s are higher in 50 k dataset (0.99).In deep learning, precision is improved by tuning from drop-out parameter and avoiding overfitting.

Accuracy for 25 k dataset is 97.3% while for 50 k dataset is98.7%. After analysis these outputs, we can state that as datasize increases the accuracy improves due to multiple process-ing layers. This is one of the reasons due to which deep learn-ing approaches are applied on large scale dataset. Figure 8demonstrate the comparison of the proposed deep learningapproach with other traditional approaches.

Area Under Curve (AUC) is generated from precision andrecall which is higher in case of deep learning, so naturallyAUC will be more than traditional approach which is 0.988and 0.993 in 25 k and 50 k data respectively more than .957and .965 of traditional approach. Fig. 9 displays the AUCcomparison graph of the proposed model with various tradi-tional models.

6.4 Discussion

A deep learning approach manages bulk data more effectivelythan the traditional machine learning approach. The proposeddeep learning-based approach to classify election tweetsshows better accuracy results. The traditional decision treepresents higher accuracy as it finds out the best-spilled featurefrom information gain method even though it is less than theproposed approach. SVM approach represents the least favor-able results as it requires more time to train the model which isdirectly dependent on the dataset size. SVM works well withunstructured and semi structured data such as text or tweets.The strength of the SVM is its kernel. Appropriate kernelgives good results for the complex situations. Another advan-tage of SVM is its less overfitting. NB and random forestdepict moderate functionality. Naïve Bayes, being a simplemodel works well with small dataset however; it faces diffi-culty in handling complex and noisy massive real-time tweets.Similarly random forest also reduces overfitting therefore itincreases accuracy. It creates lots of tree and then combinestheir output which increases complexity in the model. It alsotakes longer time to train model.

Fig. 9 AUC comparison of theproposed approach with differenttraditional approaches

Fig. 8 Accuracy comparison ofthe proposed method withdifferent traditional approaches

J. of Data, Inf. and Manag. (2020) 2:1–1412

As deep learning uses multiple hidden layers to train themodel, its prediction accuracy is high because this model iscapable to learn more and more. Traditional classificationmodels work better for small datasets. When data size in-creases, these models’ accuracy goes down. But deeplearning-based models are efficient for larger data sets. Aswe have seen in Tables 6 and 7, accuracy of the deep learningmodels is increasing with the data size. The first dataset con-tains 25 K tweets, and deep learning model has accuracyaround 97%, while when the size of dataset doubles i.e.50 K tweets its accuracy increases up to 99%.

7 Conclusions & future work

Deep learning is a new area of research which is explored forsentiment analysis. In this paper, we have introduced a deeplearning framework with the multiclass approach for user be-havior prediction using tweets of general elections of India2019. We have categorized tweets relating to the problemsor issues of nation under nine broad classes such as agricul-ture, GST, infrastructure and so on. The proposed model iscompared with traditional supervised approaches such as NB,SVM, Decision tree and random forest. The designed ap-proach in deep learning attains good accuracy results of98.70% preceding the traditional methodology. Based on theexperimental results, we can easily predict the behavior of auser which can be useful to analyze people’s sentiment duringelection time. This study provides insight into the issues thatwere most discussed during the election campaign. Such anal-ysis provides an opportunity to amend the policies and strate-gies, to carry out measures to figure out a solution, to trans-form the difficulties into possibility, areas of next action plan,etc.

For future work, we will focus on improving the accuracyin the deep learning framework.We will apply the user behav-ior analysis approach in new domains of the industry. Weexpress the desire to explore ensemble approach.

Compliance with ethical standards

Conflict of interest The authors declare that they have no conflict ofinterest.

References

Alharbi ASM, de Doncker E (May 2019) Twitter sentiment analysis witha deep neural network: an enhanced approach using user behavioralinformation. Cogn Syst Res 54:50–61

Ouyang X, Zhou P, Li CH, Liu L (2015) “Sentiment Analysis UsingConvolutional Neural Network,” in 2015 {IEEE} InternationalC o n f e r e n c e o n C o m p u t e r a n d I n f o r m a t i o nTechnology$\mathsemicolon$ Ubiquitous Computing and

Communications$\mathsemicolon$ Dependable, Autonomic andSecure Computing$\mathsemicolon$ Pervasive Intelligence andComputing

Shuang K, Guo H, Zhang Z, Loo J, Su S (2019) Aword-building methodbased on neural network for text classification. J Exp Theor ArtifIntell 31(3):455–474

Zhu J, Huang C, YangM, FungGPC (2019) Context-based prediction forroad traffic state using trajectory pattern mining and recurrentconvolutional neural networks. Inf. Sci. (Ny) 473:190–201

Kumar SS, Kumar MA, Soman KP (2018) “Identifying Sentiment ofMalayalam Tweets Using Deep Learning,” in Digital Business,Springer International Publishing, pp. 391–408

Mäntylä MV, Graziotin D, Kuutila M (2018) The evolution of sentimentanalysis{\textemdash}A review of research topics, venues, and topcited papers. Comput. Sci. Rev 27:16–32

Cha M, Haddadi H, Benevenuto F, Gummadi KP (2010) Measuring userinfluence in twitter: the million follower fallacy, in Fourth interna-tional AAAI conference on weblogs and social media

TrusovM, Bodapati AV, Bucklin RE (Aug. 2010) Determining influentialusers in internet social networks. J Mark Res 47(4):643–658

Messias J, Schmidt L, Oliveira R, Benevenuto F (2013) You followedmybot! Transforming robots into influential users in Twitter, FirstMonday, vol. 18, no. 7

Andrea ED, Ducange P, Bechini A, Renda A, Marcelloni F (2019)Monitoring the public opinion about the vaccination topic fromtweets analysis. Expert Syst. Appl 116:209–226

Kapočiute-Dzikiene J, Damaševičius R, Woźniak M (2019) SentimentAnalysis of Lithuanian Texts Using Traditional and Deep LearningApproaches. Computers 8(1):4

Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R (2011)“Sentiment analysis of twitter data,” in Proceedings of theWorkshop on Language in Social Media (LSM 2011), pp. 30–38

Kumar A, Sebastian TM (2012) Sentiment analysis on twitter. Int. J.Comput. Sci. Issues 9(4):372

Pak A, Paroubek P (2010) “Twitter as a corpus for sentiment analysis andopinion mining.,” in LREc, 2010, vol. 10, no. 2010, pp. 1320–1326

Martínez-Cámara E, Martín-Valdivia MT, Ureña-López LA, Montejo-Ráez AR (2012) Sentiment analysis in twitter. Nat Lang Eng20(1):1–28

Saif H, He Y, Alani H (2012) Semantic Sentiment Analysis of Twitter, inThe Semantic Web {\textendash} {ISWC} 2012, Springer BerlinHeidelberg, pp. 508–524

Sharma P, Moh TS (2016) “Prediction of Indian election using sentimentanalysis on Hindi Twitter,” in Proceedings - 2016 IEEEInternational Conference on Big Data, Big Data 2016, 2016, pp.1966–1971

Bouazizi M, Ohtsuki T (2017) “A Pattern-Based Approach for Multi-Class Sentiment Analysis in Twitter,” {IEEE} Access, vol. 5, pp.20617–20639

Bouazizi M, Ohtsuki T (2018) “Multi-Class Sentiment Analysis inTwitter: What if Classification is Not the Answer,” {IEEE} Access,vol. 6, pp. 64486–64502

Rosenthal S, Farra N, Nakov P (2017) “{SemEval}-2017 task 4: senti-ment analysis in twitter,” in Proceedings of the 11th internationalworkshop on semantic evaluation ({SemEval}-2017)

Java A, Song X, Finin T, Tseng B (2007) “Why we twitter,” inProceedings of the 9th {WebKDD} and 1st {SNA}-{KDD} 2007workshop on web mining and social network analysis -{WebKDD}/{SNA}-{KDD} 07

Kuncheva Z, Montana G (2015) Community detection in multiplex net-works using locally adaptive random walks,” in Proceedings of the2015 {IEEE}/{ACM} international conference on advances in so-cial networks analysis and mining 2015 - {ASONAM}{\textquotesingle}15

Bizid I, Nayef N, Boursier P, Faiz S, Morcos J (2015) Prominent usersdetection during specific events by learning on- and off-topic

J. of Data, Inf. and Manag. (2020) 2:1–14 13

features of user activities,” in Proceedings of the 2015 {IEEE}/{ACM} international conference on advances in social networksanalysis and mining 2015 - {ASONAM} {\textquotesingle}15

O’Connor B, Balasubramanyan R, Routledge BR, Smith NA (2010)From tweets to polls: linking text sentiment to public opinion timeseries,” in Fourth international AAAI conference on weblogs andsocial media

Achananuparp P, Lim E-P, Jiang J, Hoang T-A (Oct. 2012) Who isretweeting the tweeters? Modeling, originating, and promoting be-haviors in the twitter network. ACM Trans Manag Inf Syst 3(3):1–30

Kassens-Noor E (2012) Twitter as a teaching practice to enhance activeand informal learning in higher education: the case of sustainabletweets. Act Learn High Educ 13(1):9–21

Dhir A, Buragga K, Boreqqah AA (2013) Tweeters on campus: twitter alearning tool in classroom? J UCS 19(5):672–691

Ye L, Xu RF, Xu J (2012) “Emotion prediction of news articles fromreader{\textquotesingle}s perspective based on multi-label classifi-cation,” in 2012 international conference on machine learning andcybernetics

Lin KHY, Yang C, Chen HH (2007) “What emotions do news articlestrigger in their readers?,” in Proceedings of the 30th annual interna-tional {ACM} {SIGIR} conference on research and development ininformation retrieval - {SIGIR} {\textquotesingle}07

Lin KHY, Yang C, Chen HH (2008) “Emotion classification of onlinenews articles from the reader{\textquotesingle}s perspective,” in2008 {IEEE}/{WIC}/{ACM} international conference on web in-telligence and intelligent agent technology

Krawczyk B, McInnes BT, Cano A (2017) Sentiment Classification fromMulti-class Imbalanced Twitter Data Using Binarization,” in LectureNotes in Computer Science, Springer International Publishing, pp.26–37

Zainuddin N, Selamat A, Ibrahim R (2017) Hybrid sentiment classifica-tion on twitter aspect-based sentiment analysis, Appl. Intell

Qin H, Ye X, Zhao Y, Cai X (2018) Hotel Classification Based on OnlineReview Data, in 2018 14th International Conference on NaturalComputation, Fuzzy Systems and Knowledge Discovery ({ICNC}-{FSKD})

Bhoi A, Joshi S (2018) “Various Approaches to Aspect-based SentimentAnalysis,” arXiv Prepr. arXiv1805.01984

Zhang S, Xu X, Pang Y, Han J (2019) “Multi-layer attention based{CNN} for target-dependent sentiment classification,” NeuralProcess. Lett

Stojanovski D, Strezoski G, Madjarov G, Dimitrovski I (2015) “TwitterSentiment Analysis Using Deep Convolutional Neural Network,” inLecture Notes in Computer Science, Springer InternationalPublishing, pp. 726–737

C. Dos Santos and M. Gatti (2014) “Deep convolutional neural networksfor sentiment analysis of short texts,” in Proceedings of COLING2014, the 25th International Conference on ComputationalLinguistics: Technical Papers, 2014, pp. 69–78

Pang B, Lee L (2005) “Seeing stars: Exploiting class relationships forsentiment categorization with respect to rating scales,” inProceedings of the 43rd annual meeting on association for compu-tational linguistics, pp. 115–124

Go A, Bhayani R, Huang L (2009) Twitter sentiment classification usingdistant supervision. CS224N Proj. Report, Stanford 1(12):2009

Manuel K, Indukuri KV, Krishna PR (2010) “Analyzing internet slang forsentiment mining,” in 2010 second Vaagdevi international confer-ence on information Technology for Real World Problems

Vateekul P, Koomsubha T (2016) A study of sentiment analysis usingdeep learning techniques on Thai twitter data, in 2016 13th interna-tional joint conference on computer science and software engineer-ing ({JCSSE})

Hodeghatta UR (2013) “Sentiment analysis of Hollywood movies ontwitter,” in Proceedings of the 2013 {IEEE}/{ACM} internationalconference on advances in social networks analysis and mining -{ASONAM} {\textquotesingle}13

Cabanlit MA, Espinosa KJ (2014) “OptimizingN-gram based text featureselection in sentiment analysis for commercial products in Twitterthrough polarity lexicons,” in {IISA} 2014, The 5th InternationalConference on Information, Intelligence , Systems andApplications, 2014

Huang D-S (2004) A constructive approach for finding arbitrary roots ofpolynomials by neural networks. IEEE Trans Neural Netw 15(2):477–491

Hernández-Blanco A, Herrera-Flores B, Tomás D, Navarro-Colorado B(2019) A systematic review of deep learning approaches to educa-tional data mining,” Complexity, vol, 2019

Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R(2014) Dropout: a simple way to prevent neural networks fromoverfitting. J Mach Learn Res 15(1):1929–1958

Rane A, Kumar A (2018) “Sentiment classification system of twitter datafor {US} airline service analysis,” in 2018 {IEEE} 42nd annualcomputer software and applications conference ({COMPSAC})

Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

Breiman L (Aug. 1996) Bagging predictors. Mach Learn 24(2):123–140T. Hastie, J. Friedman, and R. Tibshirani (2001) “Unsupervised Learning,

” in The Elements of Statistical Learning, Springer New York, pp.437–508

Nithyassik B, Nandhini DEC (2010) Classification techniques in educa-tion domain. Int J Comput Sci Eng 2(5):1647–1684

Li M, Ch’ng E, Chong AYL, See S (2018) Multi-class twitter sentimentclassification with emojis. Ind Manag Data Syst 118(9):1804–1820

Brown I, Mues C (2012) An experimental comparison of classificationalgorithms for imbalanced credit scoring data sets. Expert Syst Appl39(3):3446–3453

Publisher’s note Springer Nature remains neutral with regard tojurisdictional claims in published maps and institutional affiliations.

J. of Data, Inf. and Manag. (2020) 2:1–1414