Capturing the semantic structure of documents using ... · Then the pseudo-document’s closeness...

Capturing the semantic structure of documents using summaries inSupplemented Latent Semantic Analysis

KARTHIK KRISHNAMURTHIChrist UniversityComputer Science

BangaloreINDIA

[email protected]

VIJAYAPAL REDDY PANUGANTIGRIET

Computer Science and EngineeringHyderabad

[email protected]

VISHNU VARDHAN BULUSUJNTUH College of Engineering

Information TechnologyKarimnagar

[email protected]

Abstract: Latent Semantic Analysis (LSA) is a mathematical technique that is used to capture the semantic struc-ture of documents based on correlations among textual elements within them. Summaries of documents containwords that actually contribute towards the concepts of documents. In the present work, summaries are used inLSA along with supplementary information such as document category and domain information in the model.This modification is referred as Supplemented Latent Semantic Analysis (SLSA) in this paper. SLSA is used tocapture the semantic structure of documents using summaries of various proportions instead of entire full-lengthdocuments. The performance of SLSA on summaries is empirically evaluated in a document classification appli-cation by comparing the accuracies of classification against plain LSA on full-length documents. It is empiricallyshown that instead of using full-length documents, their summaries can be used to capture the semantic structureof documents.

Key–Words: Dimensionality Reduction, Document Classification, Latent Semantic Analysis, Semantic Structure,Singular Value Decomposition.

1 IntroductionWith the Internet explosion over the recent years,large volumes of unstructured texts in various lan-guages are being added to the world-wide informationrepositories on a daily basis. In the recent years, thisphenomenon is also observed for texts in Indian lan-guages like Hindi, Telugu, Bengali, etc. In general,these languages are low-resource languages in termsof availability of machine translation systems, well-established corpus, natural language processing tools,etc., and thus have become an important area of re-search in the Indian scenario. With the availability ofsuch huge data, the problem of capturing semanticsfrom documents is an important area of interest in theIndian language research community.

In the context of document understanding, La-tent Semantic Analysis (LSA) is a popular methodproposed by Deerwester [1] to capture the semanticstructure of documents based on word co-occurrenceswithin texts. The method gets its name as “Latent” asit mines deeper correlations among words within textsthat are otherwise unseen. The word “Semantic” im-plies that the words in a document help identify thetopics or concepts in the document. LSA is a mathe-matical model that is completely independent of anysort of external sources of semantics like vocabular-

ies, dictionaries, grammar, syntactic parsers, or mor-phologies [2]. Since it is strictly mathematical, it is in-dependent of language and therefore it analyzes wordcombination patterns within texts scripted in any lan-guage. This becomes a motivation for applying themodel on native Indian language texts to analyze pat-terns of word correlations.

After its proposal, LSA was explored under vari-ous areas of research. To state some major work in thefield of text processing, LSA was used by Berry [3] forintelligent information retrieval. Document author-ship was visualized using LSA by Soboroff [4]. Foltz[5] used LSA for measuring textual coherence. Gor-don [6] used it for literature-based discoveries. Wolfe[7] used it for matching readers and texts. Text seg-mentation [8], relationship discovery [9], spam filter-ing [10], essay evaluation [11], etc. used LSA. LSAwas applied to speech processing in speaker recogni-tion systems by Khan [12]. It was also used to providemulti-level information security by Thorleuchter [13].Semantic content detection of video shots [14], facerecognition [15], image retrieval [16], etc. were a fewreal-time applications in image processing that usedLSA.

Two other approaches that have their roots in LSAare Probabilistic Latent Semantic Analysis (PLSA)

WSEAS TRANSACTIONS on COMPUTERSKarthik Krishnamurthi, Vijayapal Reddy Panuganti Griet,

Vishnu Vardhan Bulusu Jntuh

E-ISSN: 2224-2872 314 Volume 14, 2015

and Latent Dirichlet Allocation (LDA). PLSA pro-posed by Hofmann [17], is a technique for documentindexing based on the statistical latent class model forfactor analysis. It defines a generative data model andprovides solid statistical foundation which LSA lacks.LDA proposed by Blei [18], uses the fact that LSAand PLSA work by considering the standard bag-of-words which do not consider word order and docu-ment order, making individual words exchangeable.According to Blei, LDA is a more generative prob-abilistic model compared to PLSA. The basic ideais that documents are represented over random latenttopics and each topic is characterized by a distributionover words. Each document is assumed to be charac-terized by a particular set of topics.

The mathematical approach of LSA has no infor-mation about semantics like word definitions, wordorder, parts-of-speech or grammar rules, etc., yet itis observed to perform quite well. However, by pro-viding supplementary information, LSA’s capabilityto capture word correlations increases. In the presentwork, supplements are included in LSA in two forms– document category and domain information. Thisenhancement is referred as Supplemented Latent Se-mantic Analysis (SLSA) throughout this paper. In or-der to verify the feasibility of the enhancement, plainLSA is used on full-length documents and SLSA isused on their summaries to capture the semantic struc-ture of documents. The resulting semantic structuresare used for document classification. The classifi-cation accuracies of SLSA with summaries is com-pared against plain LSA with full-length documentsfor the two forms of supplements. The experimentsare performed on a Hindi data set by using LSA’s ad-vantage of being language independent. It is empir-ically shown that it is enough to use summaries inLSA along with the supplements instead of using full-length documents to capture the semantic structure ofdocuments.

The rest of the paper is organized as follows. Sec-tion 2 explains the LSA model. Section 3 is a discus-sion on including supplementary information in LSA.Section 4 describes the use of summaries in SLSA.Section 5 is a discussion on the dataset used in theexperiments. Section 6 presents the empirical resultsby comparing SLSA with plain LSA in a documentclassification application for the two forms of supple-ments such as document category and domain infor-mation. Section 7 concludes the paper and presentsthe future scope of work.

2 Latent Semantic AnalysisAt its core, LSA uses Singular Value Decomposition(SVD) followed by dimensionality reduction to cap-ture all correlations latent within documents by mod-eling interrelationships among words so that it cansemantically cluster words and documents that occurin similar contexts. SVD works by taking the con-ventional Vector Space Model (VSM) of text repre-sentation with term frequencies in the input term-by-document matrix. Various other weighting measuresapart from term-frequency also exist. According tothe theorem stated by Baker [19], the input matrix Amnof order m×n is constructed as a product of three ma-trices obtained upon its eigen decomposition:

Amn = UmmSmnVT

nn (1)

where UTU = I, VTV = I; I being an identity matrix,the columns of U and V are orthonormal eigenvectorsof AAT and ATA respectively, and S is a diagonal ma-trix containing the square roots of eigenvalues from Uor V, known as singular values, sorted in descendingorder.

The underlying principle of LSA is that the orig-inal matrix is not perfectly reconstructed. Rather, arepresentation that approximates the original matrix isreconstructed based on reduced number of dimensionsof the original component matrices. Mathematically,the original representation of data in matrix Amn is re-constructed as an approximately equal matrix Akmnfrom the product of three matrices Umk, Skk and andVkn based on just k dimensions of the component ma-trices Umm, Smn and Vnn of the original matrix A. Thediagonal elements of matrix S are non-negative de-scending values. If S is reduced to a k× k order diag-onal matrix Skk, then the first k columns of U and Vform matrices Umk and Vnk respectively. The reducedmodel is:

Akmn = UmkSkkVT

kn (2)

This approximate representation of the original doc-uments after dimensionality reduction reflects all theunderlying word correlations. Word correlations thatoccurred in some context prior to dimensionality re-duction now become more or less frequent, and someword correlations that did not appear at all originallymay now appear significantly or at least fractionally.This lower-dimensional matrix representation of thelinguistic texts is termed as “Semantic structure” or“LSA space” or “Semantic space” in the literature [2].



E-ISSN: 2224-2872 315 Volume 14, 2015

The quality of LSA space directly determines the per-formance of LSA applications. Factors that could af-fect LSA space quality include the kind and size ofcorpus, the dimensions, and the term-weighting mea-sures.

Fixing an optimal dimensionality to be retainedin LSA is an empirical issue. Retaining larger di-mensions reconstructs closer approximations to theoriginal matrix but may span many unessential re-lationships. On the other hand, retaining smallerdimensions saves much of computation but with acompromise on the essential relationships. Typically,the number of dimensions retained should be largeenough to capture the semantic structure in the text,and small enough to omit trivial correlations. Theproper way to make such choices is an open issue inthe factor analytic literature [1].

The semantic space obtained after dimensional-ity reduction through LSA can be used for docu-ment classification. In this context, LSA is viewedfrom a geometrical perspective where words and doc-uments are considered as points in space [1]. Thecombination of SVD and dimensionality reduction es-tablishes a k-dimensional orthogonal semantic spacewhere the words and documents are distributed ac-cording to their common usage patterns. The seman-tic space reflects those words that have been used inthe document to give information about the concepts(the axes) to which the words are closer. Essentially,LSA is a proximity model that spatially groups similarpoints together. As the dimensional space is reduced,related points draw closer to one another. The rela-tive distances between these points in the reduced vec-tor space show the semantic similarity between docu-ments and is used as a basis for document classifica-tion. A test document (a set of words) is mapped as apseudo-document into the semantic space by the pro-cess of “Folding-in” [3]. To fold-in an m×1 test docu-ment vector d into the LSA space of lower dimensionsk, a pseudo-document representation ds based on thespan of the existing term vectors (the rows of Umk) iscalculated as:

ds = dTUmkS-1 (3)

Then the pseudo-document’s closeness with all otherdocuments is measured using any of the standard mea-sures of similarity like Cosine measure, Euclidean dis-tance, etc. The category of the document that is lo-cated in its nearest proximity in space is the categoryof the test document. One of the standard approachesfor document classification like k-Nearest-Neighbor(kNN), Decision Trees, Naive Bayes, Support Vec-

tor Machines (SVM), etc. is applied for classificationpurposes.

In contrast to many other methods of text classi-fication, LSA is categorizes semantically related textsas similar even when they do not share a single term.This is because in the reduced semantic space, thecloseness of documents is determined by the overallpatterns of term usage. So documents are classified assimilar regardless of the precise terms that are used todescribe them. As a result, terms that did not actuallyappear in a document may still end up close to it if thatis consistent with the major patterns of association inthe data.

3 Supplemented Latent SemanticAnalysis

Being purely mathematical, LSA performs quite welleven without relying on any external sources ofsemantics like word definitions, parts-of-speech orgrammar rules, etc. However, when additional infor-mation is added into the process, LSA’s capability tounderstand document semantics improves. Extra in-formation is added to LSA by adding new words ordocuments to the initial term-by-document matrix. Soextra rows or columns get added for the informationthat is intended to be given as supplements to the pro-cess. From the geometrical perspective, the newlyadded supplementary information are new points inthe initial space represented by VSM. With the addi-tion of new words, the correlations that existed be-tween words earlier may now change with respect tothese newly added words. Words that might havenot had any correlation with other words may nowstart getting correlated with them via the newly addedwords. LSA’s ability to capture correlations in thisspace improves.

There are several extensions of LSA that wereempirically shown to perform better for a variety oftasks. Many of these were specifically extended forclassification problems. Relevant prior work is that ofWiemer-Hastings [20] in which surface parsing wasemployed in LSA by replacing pronouns in the textwith their antecedents. The model was evaluated as acognitive model. Zelikovitz [21] used LSA for doc-ument classification by accommodating backgroundknowledge for constructing the semantic space. Thework reported increased accuracy rates in classifica-tion. Serafin [22] suggested that an LSA semanticspace can be built from the co-occurrence of arbitrarytextual features which can be used for dialogue actclassification. Kanejiya [23] attempted to capture thesyntactic context in a shallow manner by enhancingwords in LSA with the parts-of-speech of their imme-



E-ISSN: 2224-2872 316 Volume 14, 2015

diately preceding words to use it an intelligent tutor-ing system. The results reported an increased abil-ity to evaluate more student answers. Rishel [24]achieved a significant improvement in classificationaccuracy of LSA by using part-of-speech tags to aug-ment the term-by-document matrix and then applyingSVD. The results of the work showed that the additionof parts-of-speech tags decrease word ambiguities.

In the present work, extra information is supple-mented to LSA in two forms – document category anddomain information. The model supplemented withthese two forms of supplements is referred as Sup-plemented Latent Semantic Analysis (SLSA) through-out this paper. The category of a document conveyssome information about semantics to a human being.So including it as supplement to LSA provides someamount of benefit to the overall process. The humanknowledge about the category of documents may al-low LSA to develop a better semantic representationof words and documents. When using LSA for doc-ument classification, the labels of categories of thetraining documents which human already knows areadded as supplements (rows) to the initial term-by-document matrix of LSA. For each added label (row),the cells are set to either 1 for the documents corre-sponding to the label or 0 for the rest. LSA may usethis information to form paths of higher-order corre-lations between words and derive a better semanticstructure.

Domain information is provided as supplementsto LSA by including extra documents and in turn ex-tra words other than the existing training set but con-textually similar to the existing training set. So ex-tra rows and columns get added to the initial term-by-document matrix. Specifically when the trainingset is small, the documents in it may not be sufficientto include more number of words that are importantto cover the concepts within a domain. The extradocuments that are added to the training documentsmay contain some extra words related to the conceptswithin the domain but never used in the training set.Such words may provide significant patterns of wordcombinations by forming paths of higher order corre-lations between words in the given domain.

4 Using Summaries in SLSAThe summary of a document gives a brief informa-tion about the document. Just by reading the sum-mary one understands the central idea of the docu-ment. Summaries are either extractive or abstractive.Extractive summaries are generated by picking theimportant sentences of the text and placing them inthe order in which they occur in the text. Abstractive

summaries are generated by writing new sentencesthat capture the main concepts in the text. Most au-tomatic text summarization systems generate extrac-tive summaries as it is difficult to generate abstrac-tive summaries. There exists prior work related tothe use of summaries in categorization which are ofinterest in view of the present work. Ker [25] com-bined word-based frequency and position method toget categorization knowledge from only the title fieldfor text categorization. Ko [26] considered featuresof important sentences for improving text categoriza-tion. Mihalcea [27] used essence of texts to improvedocument classification. Hulth [28] reported an im-provement in text categorization when the full-textrepresentation is combined with the automatically ex-tracted keywords. Recently document classificationwas performed based on the latent topics of importantsentences within documents [29]. Not much of workis done yet using summaries in LSA to obtain seman-tic structure of documents with better conceptual cor-relations.

Summaries of documents are observed to containonly those sentences that highlight the main insightsin a document and thus they contain words that actu-ally contribute towards the concepts of the document.Intuitively if summaries are used in LSA, they im-prove the quality of the semantic structure of docu-ments by removing those sentences which in turn re-moves those words that cannot actually contribute tobuild meaningful correlations. The present work is toconsider extractive summaries in various proportionsinstead of the entire full-length documents in SLSAalong with two forms of supplements – document cat-egory and domain information. The resulting seman-tic space is assessed by using it in a document classifi-cation application. The initial term-by-document ma-trix for SLSA is constructed by taking the weights ofwords appearing only in the summaries of documentsand not their entire full-lengths. This reduces the ini-tial term-by-document matrix to contain only thoseimportant words that contribute solely to the conceptsof that category. The semantic structure that is recon-structed is based upon only those word co-occurrencepatterns that contribute better towards the documentcategory. This high quality semantic space when usedfor document classification would increase the classi-fication performance potentially.

For generating summaries of documents in thepresent work, the LSA-based extractive summary gen-eration method adopted by Krishnamurthi [30] isused. In this method, the matrix VT resulting afterperforming LSA is used to select sentences that be-come part of the extractive summary. The columns ofmatrix VT represent the sentences and the rows rep-resent the concepts. The most important concept in



E-ISSN: 2224-2872 317 Volume 14, 2015

the text is placed in the first row and the row order in-dicates the importance of concepts. The cells of thismatrix gives information about how much a sentencecontributes towards a concept. A higher cell valuemeans the sentence contributes more to the concept.For sentence selection, the first concept is chosen andthe sentence that contributes the most to this conceptis selected as a part of the extractive summary. Thenthe second concept is chosen and in the same way thesentence with the highest contribution to this conceptis selected and added to the summary. This repetitionof choosing a concept and then the sentence that con-tributes the most to that concept is continued until apredefined number of sentences are extracted as a partof the summary.

5 DatasetFor the present work, the large amount of data avail-able on the Internet is explored. The dataset is har-vested from a Hindi language news website. Many on-line news providers like BBC Hindi, Dainik Bhaskar,NDTV Khabar, etc., provide Hindi news articles froma broad range of categories such as science, business,politics, sports, entertainment, education, etc. Thereare many advantages of choosing news articles to cre-ate an in-house Indian language dataset. Firstly, theyare available in abundance and are freely accessible.Secondly, news articles are essayed by journalists withthe aim of highlighting important insights of the newsstory. Such articles have a lot of scope to containnatural co-occurrences of words. These natural co-occurrences provide scope for modeling word corre-lations. Thirdly, the rich linguistic information natu-rally embodied in the Hindi language text allows togather syntactic and lexical knowledge necessary forextracting words and documents that are close to theconcepts grasped by humans.

The chosen dataset contains 900 news articlesdownloaded randomly from the “science”, “sports”and “entertainment” categories of the BBC Hindinews website [31] with 300 articles in each category.Each document was associated with a category labelbased on the categorization of the articles on the BBCwebsite. The documents were further validated for itscategory against its content by a human expert. Fromeach category 50 documents were randomly selectedto be used as supplements to provide domain infor-mation about that category. 50 articles from each cat-egory were randomly selected for performance test-ing of SLSA and the remaining documents of eachcategory were used for training. Table 1 presents thestatistics of the BBC Hindi news dataset.

The in-house dataset may be suspected to be

Table 1: Statistics of the BBC Hindi news datasetDocument attributes ValuesNumber of documents in the dataset 900Number of categories 3Number of documents per category 200Number of documents in training set 600Number of documents in test set 150Number of documents used for provid-ing domain information

150

noisy in nature. However it can be argued that thisdataset subject to proper preprocessing can be usedas a testbed for LSA. During preprocessing of docu-ments in the dataset, initially the corpus was dividedinto individual documents. Then each document wasbroken down to a list of words. Then the punctuations,special characters and numbers were removed. Subse-quently, the stop-words that were used across all thedocuments just as language constructs were removedas they cannot actually infer any meaning. This elimi-nation was based on the stopword list provided by theUniversity of Neuchatel [32]. After this, the dupli-cate occurrences from the remaining word set wereremoved leaving only unique words. These wordswere further stemmed to their root forms because itis the root words of a language that infer meaning ofa document. For stemming, the work of Ramanathan[33] was used, in which suffixes are stripped off ona longest match basis. After all the preprocessing,the dataset contained only unique root words spreadacross multiple documents.

6 Empirical Evaluation of SLSAwith Summaries

In the experiments that are carried out, plain LSAfor full-length documents is the baseline of compar-ison. Extractive summaries of various proportions ofthe documents are used in both training and testingphases of SLSA. The semantic space that is derivedupon dimensionality reduction is used for classifica-tion of Hindi texts. One of the kNN type classifiersi.e. 1-Nearest-Neighbor (1NN) classifier is used forits intuitiveness. This classifier assigns a point (docu-ment) in space to the class of its closest neighbor in thesemantic space. For measuring closeness, Cosine sim-ilarity is used in the empirical evaluations. The accu-racies of classification using plain LSA for full-lengthdocuments (baseline) and SLSA for extractive sum-maries of various proportions are calculated for eachof the two supplements – document category and do-main information. The performance of SLSA is com-



E-ISSN: 2224-2872 318 Volume 14, 2015

pared against the baseline under various dimensionsof the semantic space.

With 600 full-length documents and 10780 wordsin the training set, the initial term-by-document matrixis of order 10780× 600. This matrix is used by plainLSA. Summaries for SLSA are generated by retain-ing 20%, 30%, 40%, 50% and 60% of the full-lengthdocuments resulting in initial term-by-document ma-trices of order 6588 × 600, 7661 × 600, 8376 × 600,8870× 600 and 9366× 600 respectively. Summariesof approximately equal sizes to those in the train-ing sets are generated from 150 documents for test-ing and 150 documents for providing domain infor-mation. Experiments are conducted on each of thesesets. Plain LSA with full-length documents is labeledas LSA and SLSA with summaries of various propor-tions are labeled as SLSA-20, SLSA-30, SLSA-40,SLSA-50 and SLSA-60 in the figures of the follow-ing sub-sections.

6.1 Summaries in SLSA with Document Cat-egory

The category labels of the training documents thatcorrespond to the categorization of documents on theBBC Hindi news website namely “science”, “sports”and “entertainment” are added as supplements (rows)thereby adding 3 rows to the initial matrices. Foreach added label (row), the cells are set to either 1for the documents corresponding to the label or 0 forthe rest. The average accuracy of classifying 150 testdocuments with full-length documents in plain LSAis 87.6%. The classification accuracies across vari-ous dimensions using summaries in various propor-tions including the document category labels are plot-ted against the baseline in Fig. 1 to 5. SLSA withsummaries performs better than the baseline acrossmajority of dimensions of the semantic space. Table 2gives the average classification accuracies obtained inthe experiments. It is observed that there is an over-all increase in performance by 0.8% to 4.2% by usingsummaries in SLSA.

Table 2: Average Classification accuracies with sum-maries in SLSA

Model Accuracy(%) Improvement(%)LSA (Baseline) 87.6 -SLSA-20 88.4 0.8SLSA-30 89.1 1.5SLSA-40 91.2 3.6SLSA-50 91.8 4.2SLSA-60 91.6 4.0

Figure 1: Classification accuracies using SLSA with20% document summaries



6.2 Summaries in SLSA with Domain Infor-mation

For including domain information to the 600 docu-ment summaries in training set, the summaries of ex-tra 150 documents – 50 from each of the categoriesscience, sports and entertainment are included into theinitial term-by-document matrix. This results in in-creasing the order of the initial matrices to 7446×750



E-ISSN: 2224-2872 319 Volume 14, 2015



for summaries of 20%, 8648 × 750 for summaries of30%, 9463×750 for summaries of 40%, 10041×750for summaries of 50% and 10564×750 for summariesof 60% of the full-length documents. For classifica-tion, the summaries of 150 test documents are foldedinto the SLSA semantic space reconstructed alongwith the added domain information and then com-pared with the initial 600 training documents. The av-erage accuracy of classifying the full-length test doc-uments using plain LSA is 87.6%. The comparativeresults of SLSA with summaries against the baselineare shown in Fig. 6 to 10. SLSA with summaries isfound to perform better than the baseline across ma-jority of dimensions of the semantic space. Table 3gives the average classification accuracies obtained inthe experiments. It is observed that there is an over-all increase in performance by 2.1% to 4.8% by usingsummaries in SLSA.

So far very little work is done for text clas-sification with respect to Indian languages due tonon-availability of resources like standard corpus andtools. Text classification tasks for a few Indian lan-guages like Bengali, Punjabi, Assamese and Marathi




are found in the literature. For text classificationon Bengali documents, an n-gram based algorithmwas used by Mansur [34] resulting in 90% classifi-cation accuracy. Nidhi [35] classified Punjabi textdocuments using ontology based classifier. The workgave a classification accuracy of 85%. Sarmah [36]presented an approach for classification of Assamesedocuments using Assamese WordNet. This approach



E-ISSN: 2224-2872 320 Volume 14, 2015



Table 3: Average classification accuracies with sum-maries in SLSA

Model Accuracy(%) Improvement(%)LSA (Baseline) 87.6 -SLSA-20 89.7 2.1SLSA-30 90.2 2.6SLSA-40 91.7 4.1SLSA-50 92.3 4.7SLSA-60 92.4 4.8

gave an accuracy of 90.27% on Assamese documents.The work of Vispute [37] showed that the perfor-mance of a VSM based clustering algorithm is goodfor categorizing Marathi text documents. For Marathidocuments the overall accuracy of the system was91.10%. The present work on Hindi documents per-forms better than the previous techniques in the fieldwith an accuracy of 92.4%. To the best of the authors’knowledge, this work is the first of its kind in Hindi touse LSA for classification.

7 Conclusions and Future Scope

Summaries of documents contain words that actuallycontribute towards the concepts of the document. Inthe present work, summaries are used as inputs toLSA instead of entire full-length documents to cap-ture the semantic structure of documents. Further, themodel is supplemented with extra information in twoforms – document category and domain information.Supplements are added to LSA by adding extra rowsand/or columns to the initial term-by-document ma-trix from where LSA’s processing starts. This en-hancement is referred as Supplemented Latent Se-mantic Analysis (SLSA) in the present work. Thisinput matrix to SLSA results in a high quality seman-tic structure of document summaries which is usedfor classifying Hindi texts. The classification perfor-mances of SLSA on summaries of various proportionsof the full-length documents have been compared withthose of plain LSA on full-length documents for thetwo forms of supplements across various reduced di-mensions of the semantic structure. Considerable im-provements in performance is achieved using extrac-tive summaries in SLSA rather than entire full-lengthdocuments in plain LSA.

The average classification accuracy of LSA us-ing full-length documents is 87.6%. With documentcategory as a supplement in SLSA, the classificationexperiments using summaries of 20, 30, 40, 50 and 60percentages resulted in average classification accura-cies of 88.4%, 89.1%, 91.2%, 91.8% and 91.6% re-spectively. With domain information as a supplementin SLSA, summaries of 20%, 30%, 40%, 50% and60% resulted in average classification accuracies of89.7%, 90.2%, 91.7%, 92.3% and 92.4% respectively.On the whole, it is observed that for various percent-ages of summaries of both training and test documentsas inputs to SLSA, there is an overall improvementin the classification accuracies by 0.8% to 4.8%. Byachieving better classification accuracies using extrac-tive summaries rather than full-length documents, it isconcluded that using summaries to understand docu-ments indeed help in capturing better conceptual cor-relations within texts.

The present work is carried out using term fre-quency as the term weighting measure in the vectorspace model. As an extension to this work, experi-mental evaluations are to be carried out to study theinfluence on the document structure by consideringvarious unsupervised and supervised term weightingmeasures in SLSA along with summaries across dif-ferent supplements in the process.



E-ISSN: 2224-2872 321 Volume 14, 2015

References:

[1] Deerwester, S., Dumais, S., Furnas G. and Lan-dauer T. K. (1990) Indexing by latent semanticanalysis. American Society for Information Sci-ence, 391–407.

[2] Landauer, T. K. and Foltz, P. W. (1998) An Intro-duction to Latent Semantic Analysis. DiscourseProcesses, 259–284.

[3] Berry, M. and Dumais, S. (1995) Using linear al-gebra for intelligent information retrieval. SIAMReview, 573–595.

[4] Soboroff, I., Nicholas, C. K., Kukla, J. M. andEbert, D. S. (1997) Visualizing Document Au-thorship Using n-grams and Latent Semantic In-dexing. Workshop on New Paradigms in Infor-mation Visualization and Manipulation, 43–48.

[5] Foltz, P. W., Kintsch, W. and Landauer, T. K.(1998) The measurement of textual coherencewith Latent Semantic Analysis. Discourse Pro-cesses, 285–308.

[6] Gordon, M. D. and Dumais, S. (1998) Using la-tent semantic indexing for literature based dis-covery. Journal of the American Society for In-formation Science, 674–685.

[7] Wolfe, M. B. and Schreiner, M. E. (1998) Learn-ing from text: Matching readers and text byLatent Semantic Analysis. Discourse Processes,309–336.

[8] Choi, F. Y. Y., Hastings, P. W. and Moore, J. D.(2001) Latent Semantic Analysis for Text Seg-mentation. Proceedings of Empirical Methods inNatural Language Processing, 109–117.

[9] Bradford, R. (2006) Relationship Discovery inLarge Text Collections Using Latent SemanticIndexing. Workshop on Link Analysis, CounterTerrorism and Security, 20–22.

[10] Gansterer, W. and Janecek, A. (2008) Spam Fil-tering Based on Latent Semantic Indexing. Sur-vey of Text Mining: Clustering, Classification,and Retrieval, 165–183.

[11] Botana, G. J. and Leo, J. A. (2010) LatentSemantic Analysis Parameters for Essay Eval-uation using Small-Scale Corpora. Journal ofQuantitative Linguistics, 1–29.

[12] Khan, N. A. and Yegnanarayana, B. (2004) La-tent Semantic Analysis for Speaker Recognition.International Conference of Spoken LanguageProcessing, 2589–2592.

[13] Thorleuchter, D. and Van Den Poel, D. (2012)Improved Multilevel Security with Latent Se-mantic Indexing. Expert Systems with Applica-tions, 13462–13471.

[14] Souvannavong, F. and Merialdo, B. (2004) La-tent semantic indexing for semantic content de-tection of video shots. IEEE Int. Conference onMultimedia and Expo, 1783–1786.

[15] Yang, J., Luo, M. and Jiao, Y. (2013) FaceRecognition Based on Image Latent SemanticAnalysis Model and SVM. International Jour-nal of Signal Processing, Image Processing andPattern Recognition, 101–110.

[16] Pulla, C., Karthik, S. and Jawahar, C. V. (2010)Efficient Semantic Indexing for Image Retrieval.International Conference on Pattern Recogni-tion, 3276–3279.

[17] Hofmann, T. (1999) Probabilistic latent semanticindexing. International ACM SIGIR conferenceon research and development in information re-trieval, 50–57.

[18] Blei, D. M., Ng, A. Y. and Jordan, M. I. (2003)Latent Dirichlet Allocation. Journal of MachineLearning Research, 993–1022.

[19] Baker, K. (2005) Singular Value DecompositionTutorial. Electronic document.

[20] Wiemer-Hastings, P. and Zipitria, I. (2001)Rules for Syntax, Vectors for Semantics. Pro-ceedings of the Annual Conference of the Cog-nitive Science Society, 1112–1117.

[21] Zelikovitz, S. (2001) Using LSI for Text Clas-sification in the Presence of Background Text.ACM International Conference on Informationand Knowledge Management, 113–118.

[22] Serafin R., Eugenio B. D. and Glass M. (2003)Latent semantic analysis for dialogue act classi-fication. North American Chapter of the Associ-ation for Computational Linguistics on HumanLanguage Technology, 94–96.

[23] Kanejiya D., Kumar A. and Prasad S. (2003)Automatic Evaluation of Students’ Answers us-ing Syntactically Enhanced LSA. Workshop onBuilding Educational Applications using Natu-ral Language Processing, 53–60.

[24] Rishel, T., Perkins, A. L. and Yenduri S. (2006)Augmentation of a Term-Document Matrix withPart-of-Speech Tags to Improve Accuracy of La-tent Semantic Analysis. Int. Conference on Ap-plied Computer Science, 573–578.

[25] Ker, S. J. and Chen, J. (2000) A text catego-rization based on summarization technique. ACLworkshop on Recent advances in natural lan-guage processing and information retrieval, 79–83.

[26] Ko, Y., Park, J. and Seo, J. (2004) Improv-ing Text Categorization Using the Importance ofSentences. Information Processing and Manage-ment, 65–79.



E-ISSN: 2224-2872 322 Volume 14, 2015

[27] Mihalcea, R. and Hassan, S. (2005) Using theessence of texts to improve document classifi-cation. Proceedings of the Conference on Re-cent Advances in Natural Language Processing,150–160.

[28] Hulth, A. and Megyesi, B. B. (2006) A study onautomatically extracted keywords in text catego-rization. Int. Conference on Computational Lin-guistics and Annual Meeting of the Associationfor Computational Linguistics, 537–544.

[29] Ogura, Y. and Kobayashi, I. (2013) Text Classi-fication based on the Latent Topics of ImportantSentences extracted by the PageRank Algorithm.ACL Student Research Workshop, 46–51.

[30] Krishnamurthi, K., Panuganti, V. R. and Bulusu,V. V. (2013) An Empirical Evaluation of Di-mensionality Reduction using Latent SemanticAnalysis on Hindi Text. IEEE Int. Conference onAsian Language Processing, 21–24.

[31] http://www.bbc.co.uk/hindi/[32] http://members.unine.ch/jacques.savoy/clef/

hindiST.txt[33] Ramanathan, A. (2003) A Lightweight Stemmer

for Hindi. Workshop of Computational Linguis-tics for South Asian Languages Expanding Syn-ergies with Europe, 42–48.

[34] Mansur, M., Uzzaman, N. and Khan, M. (2006)Analysis of N-Gram Based Text Categoriza-tion for Bangla in a Newspaper Corpus. Centerfor Research on Bangla Language Processing,BRAC University, Dhaka, Bangladesh.

[35] Nidhi and Gupta, V. (2012) Domain Based Clas-sification of Punjabi Text Documents using On-tology and Hybrid Based Approach. Workshopon South and Southeast Asian Natural LanguageProcessing, 109–122.

[36] Sarmah, J., Saharia, N. and Sarma, S.K. (2012)A Novel Approach for Document Classificationusing Assamese WordNet. International GlobalWordnet Conference, 324–329.

[37] Vispute, S.R. and Potey, M.A. (2013) Automatictext categorization of Marathi documents usingclustering technique. International Conferenceon Advanced Computing Technologies, 1–5



E-ISSN: 2224-2872 323 Volume 14, 2015

Date post:	08-Apr-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times