Learning to Classify DocumentsLearning to Classify DocumentsEdwin ZhangEdwin Zhang

Computer Systems Lab 2009-Computer Systems Lab 2009-20102010

IntroductionIntroduction

Classifying documents Will use a Bayesian method and calculate conditional probabilityUse a set of Training Documents Choose a set of features for each categoryCoding in Java

BackgroundBackgroundNaïve Bayes Classifier/Bayesian Methodcomputes the conditional probability p(T|D) for a given document D for every topic Assigns the document D to the topic with the largest conditional probability

http://nltk.googlecode.com/svn/trunk/doc/book/ch06.html

BackgroundBackground

Program has two steps:LearningPrediction

LearningLearning

Will be using training documents

conditional probability

features selection based on how often terms appear in certain documents

http://www.dot.state.mn.us/consult/images/j0341469.jpg

PredictionPrediction

PredictionPredicting what a

unknown document is talking about based on the learning section

http://www.deafsports.co.nz/WebImages/documents.jpg

DevelopmentDevelopment

Created Category, Document, Terms classes– Category class deals with the categories– Document class deals with the documents– Terms class deals with terms that appear in each

document

CategoryCategory

Each category contains an array of documents

My categories started out with tennis and other

Added more categories as my program started working

Document ClassDocument Class

Each document contains an array of terms.

The documents were my training documents

Terms ClassTerms Class

Terms class dealt with all the terms that appeared in the training documents

For each term, an array of counts on the number of times the term appears in documents– Counts for each category

Also, each term is assigned a score– Score = number of times in category A + 1/number of

times in category B + 1 to avoid dividing by 0– Method to calculate the score varied as my program

developed Terms

Development (continued)Development (continued)

Creates an array of categories Reads in all my training documents Stores all the terms that appear in an array of

Terms Sorts the array of terms based on the score for each

category Chose the top 25 terms from the sorted array based

on each category

Development (continued)Development (continued) What I still need to do:

– Test my program's learning and write the prediction part

– Once my program works for two categories, add more categories

http://www.filibeto.org/sun/lib/nonsun/oracle/11.1.0.6.0/B28359_01/text.111/b28303/img/ccapp018.gif

Expected ResultsExpected ResultsThe more training documents, the better the results will likely be In addition, different ways of calculating score will likely produce different results

May play around with thatExpected results

DiscussionDiscussion

Once my program starts running and working correctly, I will discuss the results

I have finished the Learning part of the program, but now I need to do the Prediction part

Works CitedWorks Cited

http://www.nltk.org/book My dad Chai, Kian Ming Adam, Hai Leong Chieu,

and Hwee Tou Ng. ACM Poral. Assocation of Computing Machinery, 2002. Web. 14 Jan. 2010. <http://portal.acm.org/citation.cfm?id=564376.564395&coll=Portal&dl=ACM&CFID=70884224&CFTOKEN=94712991>.

Works Cited (continued)Works Cited (continued)

Eyheramendy, Susana, and David Madigan. "A Flexible Bayesian Generalized Linear Model for Dichotomous Response Data with an Application to Text Categorization." Lecture Notes-Monograph Series 54 (2007): 76-91. JSTOR. Web. 25 Oct. 2009. <http://www.jstor.org/stable/20461460>.

Lavine, Michael, and Mike West. "A Bayesian Method for Classification and Discrimination." Canadian Journal of Statistics 20.4 (1992): 451-461. JSTOR. Web. 14 Jan. 2010. <http://www.jstor.org/>.