Date post: | 21-Dec-2015 |
Category: |
Documents |
View: | 215 times |
Download: | 1 times |
1
SIMS 290-2: Applied Natural Language Processing
Marti HearstSept 22, 2004
2
Today
Cascaded ChunkingExample of Using Chunking: Word AssociationsEvaluating ChunkingGoing to the next level: Parsing
3
Cascaded Chunking
Goal: create chunks that include other chunksExamples:
PP consists of preposition + NPVP consists of verb followed by PPs or NPs
How to make it work in NLTKThe tutorial is a bit confusing, I attempt to clarify
4
Creating Cascaded Chunkers
Start with a sentence tokenA list of words with parts of speech assignedCreate a fresh one or use one from a corpus
5
Creating Cascaded Chunkers
Create a set of chunk parsersOne for each chunk typeEach one takes as input some kind of list of tokens, and produced as output a NEW list of tokens
– You can decide what this new list is called Examples: NP-CHUNK, PP-CHUNK, VP-CHUNK
– You can also decide what to name each occurrence of the chunk type, as it is assigned to a subset of tokens
Examples: NP, VP, PP
How to match higher-level tags?It just seems to match their string descriptionSo best be certain that their name does not overlap with POS tags too
6
7
8
9
Let’s do some text analysis
Let’s try this on more complex sentencesFirst, read in part of a corpusThen, count how often each word occurs with each POSDetermine some common verbs, choose oneMake a list of sentences containing that verbTest out the chunker on them; examine further
10
11
12
13
Why didn’t this parse work?
14
Why didn’t this parse work?
15
Why d
idn’t
this
pars
e w
ork
?
16
Why didn’t this parse work?
17
Corpus Analysis for Discovery ofWord Associations
Classic paper by Church & Hanks showed how to use a corpus and a shallow parser to find interesting dependencies between words
– Word Association Norms, Mutual Information, and Lexicography, Computational Linguistics, 16(1), 1991
– http://www.research.att.com/~kwc/publications.html
Some cognitive evidence:Word association norms: which word to people say most often after hearing another word
– Given doctor: nurse, sick, health, medicine, hospital…
People respond more quickly to a word if they’ve seen an associated word
– E.g., if you show “bread” they’re faster at recognizing “butter” than “nurse” (vs a nonsense string)
18
Corpus Analysis for Discovery ofWord Associations
Idea: use a corpus to estimate word associationsAssociation ratio: log ( P(x,y) / P(x)P(y) )
The probability of seeing x followed by y vs. the probably of seeing x anywhere times the probability of seeing y anywhereP(x) is how often x appears in the corpusP(x,y) is how often y follows x within w wordsInteresting associations with “doctor”:
– X: honorary Y: doctor– X: doctors Y: dentists– X: doctors Y: nurses– X: doctors Y: treating– X: examined Y:doctor– X: doctors Y: treat
19
Corpus Analysis for Discovery ofWord Associations
Now let’s make use of syntactic information.Look at which words and syntactic forms follow a given verb, to see what kinds of arguments it takesCompute triples of subject-verb-object
Example: nouns that appear as the object of the verb usage of “drink”:
– martinis, cup_water, champagne, beverage, cup_coffee, cognac, beer, cup, coffee, toast, alcohol…
– What can we note about many of these words?
Example: verbs that have “telephone” in their object: – sit_by, disconnect, answer, hang_up, tap, pick_up, return,
be_by, spot, repeat, place, receive, install, be_on
20
Corpus Analysis for Discovery ofWord Associations
The approach has become standardEntire collections available
Dekang Lin’s Dependency Database– Given a word, retrieve words that had dependency
relationship with the input word
Dependency-based Word Similarity– Given a word, retrieve the words that are most similar
to it, based on dependencies
http://www.cs.ualberta.ca/~lindek/demos.htm
21
Example Dependency Database: “sell”
22
Example Dependency-based Similarity: “sell”
23
Homework Assignment
Choose a verb of interestAnalyze the context in which the verb appears
Can use any corpus you like– Can train a tagger and run it on some fresh text
Example: What kinds of arguments does it take?Improve on my chunking rules to get better characterizations
24
Evaluating the Chunker
Why not just use accuracy?Accuracy = #correct/total number
DefinitionsTotal: number of chunks in gold standardGuessed: set of chunks that were labeledCorrect: of the guessed, which were correctMissed: how many correct chunks not guessed?Precision: #correct / #guessedRecall: #correct / #totalF-measure: 2 * (Prec*Recall) / (Prec + Recall)
25
Example
Assume the following numbersTotal: 100Guessed: 120Correct: 80Missed: 20Precision: 80 / 120 = 0.67Recall: 80 / 100 = 0.80F-measure: 2 * (.67*.80) / (.67 + .80) = 0.69
26
Evaluating in NLTKWe have some already chunked text from the Treebank
The code below uses the existing parse to compare against, and to generate Tokens of type word/tag to parse with our own chunker.
Have to add location information so the evaluation code can compare which words have been assigned which labels
27
How to get better accuracy?
Use a full syntactic parserThese days the probabilistic ones work surprisingly well
They are getting faster too.Prof. Dan Klein’s is very good and easy to run
– http://nlp.stanford.edu/downloads/lex-parser.shtml
28
29
30
31
32
Next Week
Shallow Parsing AssignmentDue on Wed Sept 29
Next week:Read paper on end-of-sentence disambiguationPresley and Barbara lecturing on categorizationWe will read the categorization tutorial the following week