Word Sense Disambiguation

MAS.S60Catherine Havasi

Rob Speer

Banks?• The edge of a river– “I fished on the bank of the Mississippi.”

• A financial institution– “Bank of America failed to return my call.”

• The building that houses the financial institution– “The bank burned down last Thursday.”

• A “biological repository”– “I gave blood at the blood bank”.

Word Sense Disambiguation• Most NLP tasks need WSD• “Played a lot of pool last

night… my bank shot is improving!”

• Usually keying to WordNet

“I hit the ball with the bat.”

Types• “All words”– Guess the WN sysnet

• Lexical Subset– A small number of pre-defined words

• Course Word Sense– All words, but more intuitive senses

Types• “All words”– Guess the WN sysnet

• Lexical Subset– A small number of pre-defined words

• Coarse Word Sense– All words, but more intuitive senses

IAA is 75-80% for all words task with WordNet90% for simple binary tasks

What is a Coarse Word Sense?

• How many word senses does the word “bag” have in WordNet?

What is a Coarse Word Sense?

• How many word senses does the word “bag” have in WordNet?– 9 noun senses, 5 verb senses

• Coarse WSD: 6 nouns, 2 verbs• A Coarse WordNet: 6,000 words (Navigli and Litkowski 2006)

• These distinctions are hard even for humans (Snyder and Palmer 2004)

– Fine Grained IAA: 72.5% – Coarse Grained IAA: 86.4%

“Bag”: Noun• 1. A coarse sense containing:

– bag (a flexible container with a single opening)– bag, handbag, pocketbook, purse (a container used for carrying money and

small personal items or accessories)– bag, bagful (the quantity that a bag will hold)– bag, traveling bag, travelling bag, grip, suitcase (a portable rectangular

container for carrying clothes)• 2. bag (the quantity of game taken in a particular period)• 3. base, bag (a place that the runner must touch before scoring)• 4. bag, old bag (an ugly or ill-tempered woman)• 5. udder, bag (mammary gland of bovids (cows and sheep and goats))• 6. cup of tea, bag, dish (an activity that you like or at which you are

superior)

Frequent Ingredients• Open Mind Word Expert• WordNet • eXtended WordNet (XWN)• SemCor 3.0 (“brown1” and “brown2”)• ConceptNet

Semcor

No training set, no problem• Julia Hockenmaier’s “Psudoword” evaluation• Pick two random words– Say, “banana” and “door”

• Combine them together– “BananaDoor”

• Replace all instances of either in your corpora with your new pseudoword

• Evaluate• A bit easier…

The “Flip-flop” Method• Stephen Brown and Jonathan Rose, 1991• Find a single feature or set of features which

disambiguated the words – think the named entity recognizer

An Example

Standard Techniques• Naïve Bayes (notice a trend)– Bag of words– Priors are based on word frequencies

• Unsupervised clustering techniques– Expectation Maximization (EM)– Yarowsky

Yarowsky

(slides from Julia Hockenmaier)

Training Yarowsky

• Created a blend using a large number of resources

• Created an ad hoc category for a word and its surroundings in sentence

• Find which word sense is most similar to category

• Keep the system machinery as general as possible.

Using OMCS

Adding Associations• ConceptNet was included in two forms:– Concept vs. feature matrices– Concept-to-concept associations

• Associations help to represent topic areas• If the document mentions computer-related

words, expect more computer-related word senses

Constructing the Blend

Calculating the Right Sense

“I put my money in the bank”

SemEval Task 7• 14 different systems were submitted in 2007• Baseline: Most frequent sense• Spoiler!: Our system would have placed 4th

• Top three systems:– NUS-PT: parallel corpora with SVM (Chang et al, 2007)

– NUS-ML: Bayesian LDA with specialized features (Chai, et al, 2007)

– LCC-WSD: multiple methods approach with end-to-end system and corpora (Novichi et al, 2007)

Results

Parallel Corpora• IMVHO the “right” way to do it.• Different words have different sense in

different languages• Use parallel corpora to find those instances– Like Euro or UN proceedings

English and Romanian

Gold standards are overrated

• Rada Mihalcea, 2007: “Using Wikipedia for Automatic Word Sense Disambiguation”

Lab: making a simple supervised WSD classifier

• Big thanks to some guy with a blog (Jim Plush)• Training data: Wikipedia articles surrounding

“Apple” (the fruit) and “Apple Inc.”• Test data: hand-classified tweets about apples

and Apple products• Use familiar features + Naïve Bayes to get > 90%

accuracy• Optional: use it with tweetstream to show only

tweets about apples (the fruit)

Slide Thanks• James Pustejovsky, Gerard Bakx, Julie

Hockenmaier• Manning and Schutze