A

Automatic Document TopicIdentification Using SocialKnowledge Network

Mostafa M. Hassan1, Fakhreddine Karray2 andMohamed S. Kamel21Sandvine Inc., Waterloo, ON, Canada2Department of Electrical and ComputerEngineering, Centre for Pattern Analysis andMachine Intelligence (CPAMI), University ofWaterloo, Waterloo, ON, Canada

Synonyms

Automatic document topic identification; Clus-tering; Ontology; Social knowledge network;Wikipedia

Glossary

ADTI Stands for automatic documenttopic identification

Ontology “A model for describing the world,that consists of a set of types(concepts), properties, andrelationship types” (Garshol 2004)

SKN Stands for social knowledgenetwork

WHO Stands for Wikipedia HierarchicalOntology

TF-IDF A term weighting methodology thatis commonly used in text miningand in information retrieval. Itstands for term frequency-inversedocument frequency

hi5 An online social networkingwebsite

RDF Stands for Resource DescriptionFramework. It is a method ofrepresenting information tofacilitate the data interchange onthe Web

ASR Stands for automatic speechrecognition

NMI Stands for normalized mutualinformation. It is a well-knowndocument clustering performancemeasure

NMF Stands for nonnegative matrixfactorization. Nonnegative matrixfactorization is a family ofalgorithms that tries to factor amatrix X into two matrices Y and Z,with the property that all threematrices have no negative elements

Clustering Is the process of assigning eachinput pattern to a group (cluster),such that each group containssimilar patterns

Taxonomy Is the division of concepts or topicsinto ordered groups or categories

Mohamed S. Kamel: Deceased

# Springer Science+Business Media LLC 2017R. Alhajj, J. Rokne (eds.), Encyclopedia of Social Network Analysis and Mining,DOI 10.1007/978-1-4614-7163-9_352-1

Definition

Document topic identification or indexing is usu-ally used to refer to the task of finding relevanttopics for a set of input documents (Coursey andMihalcea 2009; Medelyan et al. 2008). It is usedin many different real applications, such asimproving retrieval of library documents pertai-ning to a specific topic. It could also be used toimprove the relevancy of search engine results, bycategorizing the search results according to theirgeneral topic and giving users the ability to choosethe domain which is more relevant to their needs.

Overview

Nowadays, social networks are being frequentlyused by 1.73 billion people or more in 2013(European Travel Commission 2013). Differentsocial networks with different levels of complex-ity and popularity have been developed recentlyproviding rich media and knowledge sources.These sources include, but are not limited to,media sharing sources like YouTube and Flickr,microblogging like Twitter, general media such asFacebook and Google+, content tools such as Hi5,blog and journal such as Blogger, connectiontools such as LinkedIn, and authoritative11sources (Korfiatis et al. 2006) such as Wikipedia.Wikipedia can be seen as a social knowledgenetwork (SKN), where the users share theirknowledge together collaboratively to buildsome sort of knowledge repository. It is a freeonline encyclopedia whose contents are writtencollaboratively by a large number of voluntarycontributors around the world. Wikipedia web-pages can be edited freely by any Internet user.This type of SKN leads to a rapid increase of itsgood-quality contents, as any potential mistakesare quickly corrected within the collaborativeenvironment. Wikipedia coverage of topics hasbecome as comprehensive as other well-knownencyclopedias such as Britannica (Giles 2005),with reasonable accuracy.

This paper introduces a novel approach foridentifying document topics using social

knowledge network. In this approach, humanbackground knowledge in the form of SKN isutilized to help in automatically finding the bestmatching topic for input documents. There areseveral applications for automatic documenttopic identification (ADTI). For example, ADTIcan be used to improve the relevancy of searchengine results by categorizing the search resultsaccording to their general topic. It can also giveusers the ability to choose the domain which ismost relevant to their needs. The proposed ADTItechnique extracts background knowledge from ahuman knowledge source, in the form of a SKN,and stores it in a well-structured and organizedform, namely, an ontology. This ontology encom-passes both ontological concepts and the relationsbetween these concepts and is used to infer thesemantic similarity between documents, as well asto identify their topics.

Document topics are among this valuableinformation that needs to be extracted for severalapplications. Recently, many approaches in theliterature employ the use of background knowl-edge to improve performance of document topicidentification. Coursey and Mihalcea (2009) andCoursey et al. (2009) proposed an unsupervisedmethod based on a biased graph centrality algo-rithm, applied to a large knowledge graph builtfrom Wikipedia. They mapped the input docu-ments toWikipedia articles based on the similaritybetween them, and then they used their proposedbiased graph centrality algorithm to find thematching topics. Similar is the work presentedby Schönhofen in (2009), but instead of usingthe Wikipedia full articles’ contents, they usedthe articles’ titles to match the input documentsto the Wikipedia categories. Huynh et al. (2009)suggested an update to the work proposed bySchönhofen in (2009): they added the use of thehyperlinks in Wikipedia in articles’ titles toimprove topic identification.

Janik and Kochut (2008a, b) have used theWikipedia RDF (which is defined in Auer andLehmann (2007)) to create their ontology. Theythen transfer the document text into a graph struc-ture, employing entity matching and relationshipidentification. The categorization is based on

2 Automatic Document Topic Identification Using Social Knowledge Network

measuring the semantic similarity between thecreated graph and the categories defined in theirontology.

Basic Methodology

Extracting an Ontology from a SocialKnowledge NetworkThis section introduces the approach to building aWikipedia Hierarchical Ontology (WHO) fromthe Wikipedia knowledge repository. We use thisontology to utilize the knowledge stored inWikipedia for document representation. We assu-me that each Wikipedia category represents aunique topic. These topics are considered to bethe basic building blocks of the ontology; we referto them as concepts. Wikipedia categories areorganized in a hierarchical manner, so that theroot concepts represent abstract ideas. Recipro-cally, the leaf concepts represent very specificideas. This reflects the world knowledge in differ-ent domains with different level of granularity.Each category (concept) is associated with a col-lection of Wikipedia articles that describe andpresent different ideas related to this concept.Using these articles, we can extract the set ofterms that represent each concept. Furthermore,we associate a weight with each of these terms,which expresses how that term contributes to themeaning of that concept. These weights are cal-culated based on the frequency of occurrence ofthese terms in the articles under that concept. Weconstruct the concept-term mapping matrix M asfollows:

M ¼

tf � icf1, 1 . . . tf � icf1, j . . . tf � icf1, l⋮ ⋱ ⋮ ⋱ ⋮

tf � icfi, 1 . . . tf � icfi, j . . . tf � icfi, l⋮ ⋱ ⋮ ⋱ ⋮

tf � icfn, 1 . . . tf � icfn, j . . . tf � icfn, l

266664

377775,

(1)

where tf-icf i, j is the weight of a term i in theconcept j, n is the total number of concepts that wehave extracted, and l is the total number of termsthat we have found for all the extracted concepts.This describes the basic idea of the creation

process of Wikipedia Hierarchical Ontology. Thedetails of the algorithm have been omitted due tospace limitations. For more details, we refer to ourwork in Hassan (2013).

Automatic Document Topic IdentificationUsing WHOThere are some different tasks in text mining thatfall under document indexing, including docu-ment tagging and keyphrase extraction. Medelyan(2009) has classified these tasks according to twoaspects: first, the source of the terminology thatthe topics are extracted from and, second, thenumber of topics that can be assigned to the doc-uments. She has set three different values for thefirst aspect, which are vocabulary-restricted,document-restricted, and no restriction. Invocabulary-restricted, the source of the topic isusually some sort of background knowledgesuch as a thesaurus or a structured glossary. Indocument-restricted, the source of the topics is theinput documents themselves, where we try toselect the most representative terms from thesedocuments. In no restriction, the topics that areassigned to the documents are selected freely withno restriction to a knowledge source.

The number of the topics aspect has three dif-ferent values, which are very few (main topics),detailed topics, and all possible topics. In maintopics, the number of topics is limited to a smallset of topics (usually less than a 100). In detailedtopics, more specific topics are included, whichmakes the number of topics much bigger (usuallyfrom hundreds to thousands), and usually morethan one topic is assigned to each document. In allpossible topics, the number of topics is limited toall the terms found in the input documents. Thistype is usually called full-text indexing.

In order to complete this classification of tasks,a third aspect would be added to these two aspects,which is the learning paradigm. As is well known,learning paradigms can be classified into threetypes: supervised learning, semi-supervised learn-ing, and unsupervised learning. Table 1 shows alist of possible topic indexing tasks classifiedbased on these three aspects.

Cluster labeling is a complementary task todocument clustering, in order to be a complete

Automatic Document Topic Identification Using Social Knowledge Network 3

topic indexing task. It is the process of selecting arepresentative label (topic) for each clusterobtained from the document clustering process(Popescul and Ungar 2000). The source of theselabels is usually from the terms and/or the phraseswhich the input documents are indexed with. Oneof the approaches used for cluster labeling is touse a feature selection technique, such as mutualinformation and chi-squared feature selection, todifferentiate cluster labeling.

Term assignment, or subject indexing, is theprocess of finding the best representative topicsfor each document. The source of terminology isusually extracted from an external thesaurus,unlike the keyphrase extraction task where themain goal is to extract the most distinct phrasesappearing in the documents. Lastly in documenttagging, or in short tagging, tags can be chosenfreely without any formal guideline. Usually thelast three tasks, term assignment, keyphraseextraction, and tagging, are referred to as docu-ment topic identification. Although they differ inthe source of terminology, they more or less do thesame task, which is assigning each document a setof representative terms/phrases/tags. Also, thesethree tasks have implementations for both learn-ing paradigms.

Automatic document topic identification(ADTI) can be seen as an optimal assignmentproblem where given a set of topic labels,(ℒ = {b1, b2, . . . , bp}), which has beenmarkedas being “of interest,” and a set of input docu-ments, D ¼ d1, d2, . . . , dmf gð Þ , it is required toassign each input document di to one of thesetopics bj.

The word “identification” in ADTI means find-ing the best match between the input document set

and the input topic list. In contrast to otherapproaches, the list of topics is a known entity inour approach, which means that there is no need topredict them (Hassan 2013). To better understandthe difference, let us consider the following exam-ple: assume that we are interested in the topicseconomics, politics, and sports and we have adocument declaring “Barack Obama is the newPresident of the United States.” Usually othertopic indexing approaches will use some form ofbackground knowledge to predict the topic of thisdocument, and they might give the followingtopics as the best matching topics: “US presiden-tial election,” “Presidents of the USA,” and soon. These topics usually will be very specific tothis document. In contrast, our approach will try toidentify the most relevant topic from the list ofgiven topics and will identify it as “politics.”

ADTI can be used in many different real appli-cations, for example, improving retrieval oflibrary documents pertaining to a certain topic. Itcan also be used to improve the relevancy ofsearch engine results, by categorizing searchresults according to their general topic and givingusers the ability to choose the domain which ismore relevant to their needs. It is also needed foran organization like a news publisher or newsaggregators, where they want to automaticallyassign each news article to one of the predefinednews main topics. Similarly, it can be applied fordigital libraries to assign each new article to one ofthe predefined lists of topics. ADTI can also beused to improve the output of automatic speechrecognition (ASR) system by selecting the lan-guage model that is most relevant to the topic ofthe speech input.

Automatic Document Topic Identification Using Social Knowledge Network, Table 1 Topic indexing tasks

Task Source of terminology Number of topics Learning paradigm

Document classification Vocabulary-restricted Main topics only Supervised

Document clustering with cluster labeling Document-restricted Main topics only Unsupervised

Term assignment Vocabulary-restricted Detailed topics Supervised/unsupervised

Keyphrase extraction Document-restricted Detailed topics Supervised/unsupervised

Document tagging Unrestricted Detailed topics Supervised/unsupervised

4 Automatic Document Topic Identification Using Social Knowledge Network

Automatic Document Topic IdentificationMethodologyThe idea of ADTI is to map topics and inputdocuments to the same space and then find theclosest topic to each input document. Here thecommon space between input documents andtopics is the term space. This mapping processcan be split into three different steps. The firststep is to extract the representative concept vectorfor each input topic from the constructed ontol-ogy. The second step is to use the concept taxon-omy inWHO to enrich the topics’ representations.The last step is to use extracted topic vectors toidentify documents’ topics. The following subsec-tion discusses the different ways to extract therepresentative concept vector examined in theproposed approach.

Extracting Representative Concepts for InputTopicsAs mentioned earlier, one of the inputs of theADTI application is a list of topics of interest, towhich we want to classify input documents. In thismodule, we try to find the matching list of con-cepts to these topics. Usually, this process is doneby direct matching of topics’ names and concepts’names. The problem is that sometimes there is nodirect match between some topics and concepts orthe direct matching is not so accurate. For exam-ple, sometimes the given topic of interest is “tech-nology,” but the best matching concept describingthe input document set is “information technol-ogy.” Or as an example of a topic with no directmatching concept, consider “economy.” The bestmatching concept label for this topic is “econom-ics” not “economy.” To resolve this problem,applied manual matching is proposed. In thisapproach, we use the data set provider’s experi-ence about the input document set to find thematching concepts. Given the list of the ontologyconcept labels (ℒ), for each given topic label, westart by searching that list for all available conceptlabels that contain this topic label. The output listfor each topic label is sorted based on the ortho-graphic similarity between the topic label and thelist of concepts’ labels. We pass on these lists ofconcepts’ labels to the data set provider to selectthe best matching concept(s) for each topic, based

on their experience with the input data set. Themain drawback of this approach is that it makesthe whole technique partially manual, as humansare still needed to select the matching concepts.

After identifying the matching concepts eithermanually or automatically, we construct the topic-concept map matrix P. Each row of this matrixrepresents a topic and each column represents aconcept. In other words, if the number of the inputtopics is p and the total number of concepts extra-cted in WHO is n, then P size will be p � n. Eachelement of this matrix, P i, j, is equal to one whenthe concept j is considered a representative con-cept for the topic i, according to the list matchingconcepts extracted in the previous step, and is zerootherwise. Notice that the matrix P is too sparseand n � p.

Enhance Topic Representation by UtilizingOntology TaxonomyIn some cases, this concept-termmatrix P does notsuffice to represent the topic well. This situationoccurs often in abstract topics where the numberof relevant articles in Wikipedia is too small;hence, the representing concept vector will havea very small list of representing terms. For exam-ple, the Wikipedia category “computer science” isonly covered by only four articles. Consequentlythis will affect the identification of the topic. Asmentioned previously, each concept in our extra-cted ontology, WHO, has a conceptual relation-ship to other concepts which are represented in theontology taxonomy, in addition to its representa-tive terms’ vectors. We utilize the hierarchicalstructure of concepts to increase the amount ofinformation that is associated with each topic;this also increases the generality of the topics.This is done by augmenting to the concept-termmapping vector of each topic of interest, the set ofterm mapping vectors associated with each con-cept under the hierarchy of that topic of interest.For example, if the topic of interest is “Biology,”we add term mapping vectors that are associatedwith the concepts “Anatomy,” “Botany,” “Zool-ogy,” etc., to its associated concept-term mappingvector. This includes not only the directlyconnected concepts to this topic but also all thetopics in the hierarchy down to a specific level l.

Automatic Document Topic Identification Using Social Knowledge Network 5

Although the previous augmentation of sub-concepts’ information to the main conceptincreases the amount of information that is asso-ciated with the main topic, it also adds some noise.Noise here means a subset of the information thatis related to the subtopic, but is not related to themain topic. Since the relatedness between themain concept and its subconcepts decreases asthey get farther from it, the quantity of noiseincreases as we go deeper into the hierarchy ofconcepts. To resolve this problem, we introduceda penalty function Pen to penalize the informationcoming from the subconcepts as follows:

Pen ¼ e�L:

The penalty term is a function of the level ofthe subtopic, so that as one goes deeper into thehierarchy, the penalty value increases (Hassan2013).

Identification ApproachThe last step after selecting the list of conceptsthat will represent the given list of topics is utiliz-ing these extracted concepts to identify the topicsof input documents. The nearest centroid app-roach is proposed for identifying documents’topics. This approach is very similar to the nearestcentroid classification approach. The idea of thenearest centroid classification is to create a proto-type, a centroid in this case, for each class, to thenuse this prototype to classify the input documentsby assigning each input document to the closestprototype. Similarly, we define a prototypefor each topic, to use it in identifying inputdocuments’ topics. We use the constructedtopic-concept map matrix P, defined previously,and the WHO concept-term mapping M, definedin (1), to extract the matrix Q that represents thetopics of interest prototypes as follows:

Q ¼ PM: (2)

The size of the output matrix Q is p � l, wherel is the total number of terms in WHO and p is thenumber of the input topics. Notice that each row inthis matrix represents a topic which is a vector of

summation of all its representative concepts’ vec-tors. We can also notice that the size of the matrixQ is much smaller than the original matrix M asp � n, where n is the total number of conceptsin WHO.

The next step is to construct the document-term matrix A from input documents accordingto the conventional VSM representation as shownin (3):

A ¼w11 w12 . . . w1n

w21 w22 . . . w2n

⋮ ⋮ ⋱ ⋮wm1 wm2 . . . wmn

2664

3775: (3)

The size of the output matrix A ism� n, wherem is the total number of input documents and n isthe total number of terms in the input documentset. Then we remove all terms in Awhich are notdefined in Q, as we consider them to be out-of-vocabulary terms and vice versa, and all termsfound in Q and not defined in A are consideredto be out-of-interest terms and are removed fromQ. Then we normalize each vector of Q to be alength of 1. Hence, the new mapping matrix willbe

Q̂ ¼ L�1Q, (4)

where L is a diagonal matrix whose elements arethe lengths of the matrix Q. We can use the fol-lowing equation to calculate the document-topicsimilarity S matrix as follows:

S ¼ AQ̂T, (5)

where each row S i in the matrix S represents thesimilarity between a document i and the list of thetopics of interest. Then we can define the identifi-cation function h(x) as follows:

h xð Þ ¼ argmaxe� 1:p

Sx, e (6)

6 Automatic Document Topic Identification Using Social Knowledge Network

where h(x) is a function that takes an index of adocument x and returns the index of the mostsimilar topic, e.

Experimental Results

The following subsections describe the experi-mental results of applying ADTI using WHO.

Experiment SetupEight different benchmark data sets that fit ADTIrequirements have been selected. These data setsare split into groups or classes, each of whichexplicitly represents a specific topic. Most ofthese data sets have been previously used byZhao et al. (2005) to evaluate the performance ofdifferent document clustering algorithms. Table 2summarizes the properties of these data sets.

In all data sets, terms which appear in only onedocument or do not appear in our ontology,WHO,are removed. Hence, we have n0 in Table 2 torepresent the total number of terms after removingthese terms. Then, the term weights inside docu-ments are normalized according to the TF-IDFweighting. Lastly, all documents are normalizedto represent unit vectors in different directions interm space.

Comparing ADTI to Document ClusteringIn this experiment, ADTI is compared with fourdifferent standard and state-of-the-art clusteringtechniques:

K -means clustering (K -MEANS): The sphericalk-means version is used since the documentsare represented as vectors, and the distancemeasure used is the cosine similarity. Weused the MATLAB implementation of theLloyd’s algorithm (Lloyd 1982). As is wellknown, k-means clustering output depends onthe initial step. Hence, the cluster assignmentsare changed for the different runs. So, weapplied the k-means clustering 10 times. Wereport here the mean and standard deviation ofthese runs.

Hierarchical clustering: Two different linkagemethods are used in hierarchical clustering,

average (HIC-AVG) and complete (HIC-CMP) linkage. We also used the MATLABimplementation for the hierarchical clustering.As the hierarchical clustering does not dependon any initial conditions, there is no need toapply it multiple times.

Spectral clustering (SC): We have used thecosine similarity as the measure of similaritydocuments. Regarding the Laplacian matrixnormalization, we have used the Ng et al.(2002) proposed approach: L = I � D �1/2

SD �1/2. We have used the same k-meansimplementation for clustering. As this methoddepends on k-means approach, the clusterassignments are changed for the differentruns. We applied this algorithm 10 times asfor k-means, and we report the mean and stan-dard deviation of these runs.

NMF clustering (NMF): We have used the Xuet al. (2003) and Xu and Gong (2004) approachfor NMF clustering. As factorization of matri-ces is generally nonunique, NMF is considereda nondeterministic approach. Hence, we app-lied the NMF clustering algorithm 10 timesand we report the mean and standard deviationof these runs.

After applying ADTI and the document clus-tering, the output labels are compared againstground-truth labels to evaluate each method. Inthe case of document clustering, we need to findthe mapping between the clusters’ labels andthe provided ground-truth labels. The Hungarianalgorithm is used to find this matching (Kuhn2005). In the case of ADTI, there is no need tofind the matching as ADTI not only partitions thedata but also provides the topics of these parti-tions. Hence, we can match these labels with theprovided ground-truth labels. Both approacheswere applied to the eight data sets mentionedearlier.

Performance Measures

We have selected some of the most well-knowndocument clustering performance measures.

Automatic Document Topic Identification Using Social Knowledge Network 7

Generally, document clustering has two differentsets of performance measures which are internaland external performance measures. It is mean-ingless to use the clustering internal performancemeasures with ADTI, as the partitioning betweenthe documents is not based on the pairwise simi-larity between documents as in document cluster-ing, and the aim of ADTI is to identify the topicsof documents, neither to minimize the separationbetween classes nor to increase the compactnessof each class.

We chose three external quality measuresto evaluate the performance of the proposedapproach: F-measure, purity, and NMI. F-measureevaluates the output accuracy with regard to agiven ground truth. It is defined as the harmonicmean of the clustering precision and recall. Purityis a measure of the purity of the clusters generated.After mapping each cluster to a class, the purity isdefined as that fraction of documents belonging tothat class, over the total number of documents inthe cluster. Normalizedmutual information (NMI)is a well-known document clustering performancemeasure. It estimates the amount of shared info-rmation between the clusters’ labels and the

categories’ labels. It measures the amount ofinformation that can be obtained from the clusterlabels by observing the category labels. Thehigher the value of these measures, the better theobtained output is. A detailed review of thesemeasures can be found in Hassan (2013). Therunning time, T, of each technique is also mea-sured to compare the efficiency of theseapproaches.

As shown in Fig. 1, ADTI outperforms bothhierarchical clustering techniques for all data setsin terms of F-measures. The standard deviation isshown as error bars with each average value forthe measure. Note that the hierarchical approachesand the proposed approach have no error bars asthey have deterministic outputs.

We can also see that ADTI-CENT3 outper-forms the partitional clustering methods in fivedata sets. In the hitech data set, ADTI-CENT3

has a competitive performance with partitionalclustering methods. For bbc and bbc-sports datasets, we can see that ADTI-CENT3 has a compet-itive performance with NMF and poorer perfor-mance than both spectral clustering and k-means.

Automatic Document Topic Identification Using Social KnowledgeNetwork, Table 2 Summary of data sets usedto evaluate the performance of ADTI: m is the number of documents, n is the total number of terms in all documents, n0 isthe number of used terms where the other terms that are not present in WHO are ignored, and k is the number of topics

ID Source m n n0 Topics k

k1b WebACE 2340 21,839 19,021 Business, entertainment, health, politics, sports, tech 6

wap WebACE 1311 8460 7293 People, television, health, media, art, film, business,culture, music, politics, sports, entertainment, industry,multimedia

14

reviews San JoseMercury(TREC)

4069 36,746 32,921 Food, movie, music, radio, restaurant 5

sports San JoseMercury(TREC)

8580 27,673 24,115 Baseball, basketball, bicycle, boxing, football, golf,hockey

7

hitech San JoseMercury(TREC)

2301 22,498 20,268 Computer, electronics, health, medical, research,technology

6

mm San JoseMercury(TREC)

2521 29,973 27,262 Movie, music 2

bbc BBC News 2225 9636 7992 Business, entertainment, politics, sport, tech 5

bbc-sports

BBC Sport 737 4613 3804 Athletics, cricket, football, rugby, tennis 5

8 Automatic Document Topic Identification Using Social Knowledge Network

Experiments show that recall measure results aremore or less the same performance as F-measure.

In terms of purity, Fig. 2 shows that the pro-posed ADTI approach has broadly similar perfor-mance to partitional clustering methods, exceptwith the mm data set, where ADTI approach out-performs all clustering approaches. We can alsosee that the proposed ADTI approach outperformsboth hierarchical clustering approaches in all datasets.

Figure 3 shows the output performance com-parison with NMImeasure. ADTI-CENT3 outper-forms the hierarchical clustering methods in alldata sets and outperforms all different partitionalclustering techniques in five data sets, while it hasa very competitive performance with the par-titional clustering techniques in second data set,but has poorer performance on the last two datasets.

Figure 4 shows the running time comparison ofthese approaches. We can see that NMF is theslowest approach. As for ADTI approach, it hasnearly the same running time to most clusteringapproaches in almost all data sets.

Key Research Findings

Table 3 shows the overall relative performancemeasures and running time for the different doc-ument clustering methods and ADTI approachwhere relative performance means the ratiobetween the performance of a method and theone of the best performing method for a specificmeasure. The best output performance is shown inbold and the second best is underlined.

As shown in Table 3, the proposed ADTImethod outperforms all the different documentclustering techniques in overall relative perfor-mance measures. In terms of running time,ADTI-CENT3 is better than all clustering exceptthe spectral clustering.

The use of background knowledge to enhancethe performance of text mining has been proposedand widely used in different applications. Thispaper presented a novel approach for automaticdocument identification (ADTI) approach utiliz-ing the background knowledge in the form of anontology. This approach encompasses two mainmodules. The first module is concerned with howto build an organized and structured form ofknowledge, ontology, from a different format of

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0k1b wap reviews sports hitech mm

Data sets

F-measure ↑

bbc-sportbbc

HIC–AVG

SC

HIC–CMP K–MEANS

ADTI–CENT3NMF

Automatic DocumentTopic IdentificationUsing Social KnowledgeNetwork, Fig. 1 Theoutput F-measure of thedifferent documentclustering methods andADTI approach for eightdata sets

Automatic Document Topic Identification Using Social Knowledge Network 9

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0k1b wap reviews sports hitech mm

Data sets

Purity ↑

bbc-sportbbc

HIC–AVG

SC

HIC–CMP K–MEANS

ADTI–CENT3NMF

Automatic DocumentTopic IdentificationUsing Social KnowledgeNetwork, Fig. 2 Theoutput purity of the differentdocument clusteringmethods and ADTIapproach for eight data sets

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0k1b wap reviews sports hitech mm

Data sets

NMI ↑

bbc-sportbbc

HIC–AVG

SC

HIC–CMP K–MEANS

ADTI–CENT3NMF

Automatic DocumentTopic IdentificationUsing Social KnowledgeNetwork, Fig. 3 Theoutput NMI of the differentdocument clusteringmethods and ADTIapproach for eight data sets

10 Automatic Document Topic Identification Using Social Knowledge Network

knowledge repository. The second module detailshow to utilize this knowledge structure, an ontol-ogy, for a newly defined task.

Future Directions for Research

The proposed ADTI approach can be used indifferent applications. One of the potential appli-cations we are planning to study is the applicabil-ity of the proposed approach to improve theoutput of an automatic speech recognition (ASR)system. In order to produce the recognized text, an

ASR system usually needs to be supplied witha language model. The efficiency of the ASRsystem is very much dependent on the accuracyof the supplied language model. So if we couldknow the topic of the speech input, we couldsupply a more relevant language model. Most ofthe time we do not know this kind of informationin advance; therefore, we provide a generic lan-guage model, which leads to a lower accuracy ofthe ASR system. To overcome this problem, wecan provide the output of the ASR system usingthis kind of generic language model to the docu-ment identification system. The document

40

35

30

25

20

15

10

5

0k1b wap reviews sports hitech mm

Data sets

Running time(sec.) ↓

bbc-sportbbc

HIC–AVG

SC

HIC–CMP K–MEANS

ADTI–CENT3NMF

Automatic DocumentTopic IdentificationUsing Social KnowledgeNetwork, Fig. 4 Theoutput running time of thedifferent documentclustering methods andADTI approach for eightdata sets

Automatic Document Topic Identification Using Social Knowledge Network, Table 3 The overall relativeperformance measures for the different document clustering methods and ADTI approach with level 3

Method F-measure Purity NMI Time (s)

HIC-AVG 0.40 � 0.21 0.61 � 0.20 0.43 � 0.37 0.54 � 0.29

HIC-CMP 0.35 � 0.13 0.55 � 0.13 0.24 � 0.19 0.54 � 0.29

K-MEANS 0.79 � 0.17 0.91 � 0.10 0.78 � 0.24 0.56 � 0.23

SC 0.75 � 0.23 0.88 � 0.13 0.73 � 0.30 0.83 � 0.34

NMF 0.72 � 0.19 0.84 � 0.14 0.70 � 0.27 0.09 � 0.09

ADTI-CENT3 0.95 � 0.04 0.94 � 0.05 0.88 � 0.08 0.64 � 0.32

Automatic Document Topic Identification Using Social Knowledge Network 11

identification system will provide in return themost relevant topic using the inaccurate outputof the ASR. Consequently, a more relevant lan-guage model can be supplied to get a more accu-rate result.

Cross-References

▶Analysis and Mining of Tags, (Micro)Blogs,and Virtual Communities

▶Ontology Matching▶Topic Modeling in Online Social Media, UserFeatures, and Social Networks for

▶Web Ontology Language (OWL)▶Wikipedia Knowledge Community Modeling

References

Auer S, Lehmann J (2007) What have Innsbruck andLeipzig in common? Extracting semantics from wikicontent. In: Franconi E, Kifer M, May W (eds) Thesemantic web: research and applications. Springer,Berlin/New York, pp 503–517

Coursey K, Mihalcea R (2009) Topic identification usingWikipedia graph centrality. In: Proceedings of humanlanguage technologies: the 2009 annual conference ofthe North American chapter of the Association forComputational Linguistics, companion volume: shortpapers, Association for Computational Linguistics,Boulder, pp 117–120

Coursey K, Mihalcea R, Moen W (2009) Using encyclo-pedic knowledge for automatic topic identification. In:Proceedings of the thirteenth conference on computa-tional natural language learning, Association for Com-putational Linguistics, Boulder, pp 210–218

European Travel Commission (2013) Social networkingand UGC. http://www.newmediatrendwatch.com/world-overview/137-social-networking-and-ugc, June 2013.Online; Accessed 25 Oct 2013

Garshol L (2004) Metadata? Thesauri? Taxonomies? Topicmaps! Making sense of it all. J Inf Sci 30(4):378

Giles J (2005) Internet encyclopaedias go head to head.Nature 438(7070):900–901

Hassan M (2013) Automatic document topic identificationusing hierarchical ontology extracted from human

background knowledge. PhD dissertation, Universityof Waterloo

Huynh D, Cao T, Pham P, Hoang T (2009) Using hyperlinktexts to improve quality of identifying document topicsbased on Wikipedia. In: International conference onknowledge and systems engineering, 2009 (KSE’09),IEEE, Hanoi, pp 249–254

Janik M, Kochut K (2008a) Training-less Ontology-basedText Categorization. In: workshop on exploitingsemantic annotations in information retrieval (ESAIR2008) at the 30th European Conference on InformationRetrieval, ECIR

Janik M, Kochut K (2008b) Wikipedia in action: ontolog-ical knowledge in text categorization. In: IEEE interna-tional conference on semantic computing, 2008, IEEE,Santa Clara, pp 268–275

Korfiatis NT, Poulos M, Bokos G (2006) Evaluatingauthoritative sources using social networks: an insightfrom Wikipedia. Online Inf Rev 30(3):252–262

Kuhn HW (2005) The Hungarian method for the assign-ment problem. Nav Res Logist 52(1):7–21

Lloyd S (1982) Least squares quantization in PCM. IEEETrans Inf Theory 28(2):129–137

Medelyan O (2009) Human-competitive automatic topicindexing. PhD dissertation, The University of Waikato

Medelyan O,Witten I, Milne D (2008) Topic indexing withWikipedia. In: Proceedings of AAAI workshop onWikipedia and artificial intelligence: an evolving syn-ergy, AAAI, Chicago, pp 19–24

Ng A, Jordan M, Weiss Y et al (2002) On spectral cluster-ing: analysis and an algorithm. Adv Neural Inf ProcessSyst 2:849–856

Popescul A, Ungar LH (2000) Automatic labeling ofdocument clusters. http://citeseer.ist.psu.edu/viewdoc/download? doi:10.1.1.33.141&rep=rep1&type=pdf

Schönhofen P (2009) Identifying document topics usingthe Wikipedia category network. Web Intell Agent Syst7(2):195–207

Xu W, Gong Y (2004) Document clustering by conceptfactorization. In: Proceedings of the 27th annual inter-national ACM SIGIR conference on research anddevelopment in information retrieval, ACM, Sheffield,pp 202–209

Xu W, Liu X, Gong Y (2003) Document clustering basedon non-negative matrix factorization. In: Proceedingsof the 26th annual international ACM SIGIR confer-ence on research and development in informationretrieval, ACM, Toronto, pp 267–273

Zhao Y, Karypis G, Fayyad U (2005) Hierarchical cluster-ing algorithms for document datasets. Data Min KnowlDiscov 10(2):141–168

12 Automatic Document Topic Identification Using Social Knowledge Network