Parsing Based on presentations from Chris Manning’s course on Statistical Parsing (Stanford)

transcript

Parsing

Based on presentations from Chris Manning’s course on Statistical Parsing (Stanford)

John hit the ball

Levels of analysis

Level Elements used

Morphology/Lexical Words

POS (morpho-synactic), WSD Words

Shallow syntax parsing Phrases

Full syntax parsing Sentence

NER, MWE Phrases

SRL Parsed trees

Full semantic parsing Parsed trees

Buffalo…

Parsing is a difficult task!

^______^ so excited! #Khaleesi #miniKhaleesi #GoT

Ambiguities

POS tags (e.g., books : a verb or a noun?)

Compositional expression meanings (e.g., he spilled the beans about his past)

Syntactic attachments (V N PP)(e.g., I ate my spaghettis with a fork)

Global semantic ambiguities (e.g., bear left at zoo) Usually,

ambiguities in one layer may be

resolved in upper layers

Ambiguities

Fed raises interest rates 0.5 % in effort to control inflation

Motivation

Parsing may help to resolve ambiguities

Parsing is a step toward understanding the sentence completely

Was shown to improve the results of several NLP applications: MT (Chiang, 2005) Question answering (Hovy et al., 2000) …

Grammar

S NP VP NN interest NP (DT) NN NNS rates NP NN NNS NNS raises NP NNP VBP interest VP V NP VBZ rates …

Minimal grammar on “Fed raises” sentence: 36 parses Simple 10 rule grammar: 592 parses Real-size broad-coverage grammar: millions of parses

Size of grammar

less more

Number of rules

Limits unlikely parses

Butgrammar is not robust

Parses more sentences

Butsentences end up with ever more parses

Statistical parsing

Statistical parsing can help selecting the rules that best fit the input sentence, allowing the grammar to contain more rules

Treebanks( (S

(NP-SBJ (DT The) (NN move))

(VP (VBD followed)

(NP (DT a) (NN round))

(PP (IN of)

(NP (JJ similar) (NNS increases))

(PP (IN by)

(NP (JJ other) (NNS lenders)))

(PP (IN against)

(NP (NNP Arizona) (JJ real) (NN estate) (NNS loans))))))

The Penn Treebank Project (PTB):Arabic, English, Chinese, Persian, French,…

Advantages of treebanks

Reusability of the laborBroad coverageFrequencies and distributional

informationA way to evaluate systems

Types of parsing

Constituency parsing Dependency parsing

Constituency parsing

Constituents are defined based on linguistic rules (phrases)

Constituents are recursive (NP may contain NP as part of its sub-constituents)

Different linguists may define constituents differently…

Dependency parsing

Dependency structure shows which words depend on (modify or are arguments of) which other words

Parsing

We want to run a grammar backwards to find possible structures for a sentence

Parsing can be viewed as a search problem

We can do this bottom-up or top-downWe search by building a search tree which

his distinct from the parse tree

Phrase structure grammars = context-free grammars (CFG)

G = (T, N, S, R)T is set of terminalsN is set of nonterminalsS is the start symbol (one of the

nonterminals)R is rules/productions of the form X ,

where X is a nonterminal and is a sequence of terminals and nonterminals (possibly an empty sequence)

A grammar G generates a language L

Probabilistic or stochastic context-free grammars (PCFGs)

G = (T, N, S, R, P) T is set of terminals N is set of nonterminals S is the start symbol (one of the nonterminals) R is rules/productions of the form X , where X is a

nonterminal and is a sequence of terminals and nonterminals (possibly an empty sequence)

P(R) gives the probability of each rule

Soundness and completeness

A parser is sound if every parse it returns is valid/correct

A parser terminates if it is guaranteed to not go off into an infinite loop

A parser is complete if for any given grammar and sentence, it is sound, produces every valid parse for that sentence, and terminates

(For many purposes, we settle for sound but incomplete parsers: e.g., probabilistic parsers that return a k-best list.)

Top down parsing

Top-down parsing is goal directed

A top-down parser starts with a list of constituents to be built. The top-down parser rewrites the goals in the goal list by matching one against the LHS of the grammar rules, and expanding it with the RHS, attempting to match the sentence to be derived

If a goal can be rewritten in several ways, then there is a choice of which rule to apply (search problem)

Can use depth-first or breadth-first search, and goal ordering

Top down parsing

Disadvantages of top down

A top-down parser will do badly if there are many different rules for the same LHS. Consider if there are 600 rules for S, 599 of which start with NP, but one of which starts with V, and the sentence starts with V

Useless work: expands things that are possible top-down but not there

Repeated work

Bottom up chart parsing

Bottom-up parsing is data directed

The initial goal list of a bottom-up parser is the string to be parsed. If a sequence in the goal list matches the RHS of a rule, then this sequence may be replaced by the LHS of the rule

Parsing is finished when the goal list contains just the start category

If the RHS of several rules match the goal list, then there is a choice of which rule to apply (search problem)

The standard presentation is as shift-reduce parsing

Shift-reduce parsingcats scratch people with claws

cats scratch people with claws SHIFT

N scratch people with claws REDUCE

NP scratch people with claws REDUCE

NP scratch people with claws SHIFT

NP V people with claws REDUCE

NP V people with claws SHIFT

NP V N with clawsREDUCE

NP V NP with clawsREDUCE

NP V NP with claws SHIFT

NP V NP P claws REDUCE

NP V NP P claws SHIFT

NP V NP P N REDUCE

NP V NP P NP REDUCE

NP V NP PP REDUCE

NP VP REDUCE

S REDUCE

Disadvantages of bottom up

Useless work: locally possible, but globally impossible.

Inefficient when there is great lexical ambiguity (grammar-driven control might help here)

Repeated work: anywhere there is common substructure

Parsing as search

Left recursive structures must be found, not predicted

Doing these things doesn't fix the repeated work problem: Both TD and BU parsers can (and frequently do) do

work exponential in the sentence length on NLP problems

Grammar transformations can fix both left-recursion and epsilon productions

Then you parse the same language but with different trees (and fix them post hoc)

Dynamic programming

Rather than doing parsing-as-search, we do parsing as dynamic programming

Examples:CYK (bottom up), Early (top down)

It solves the problem of doing repeated work

Notation

w1n = w1 … wn = the word sequence from 1 to n

wab = the subsequence wa … wb

= the nonterminal Nj dominating wa … wb

We’ll write P(Ni ζj) to mean P(Ni ζj | Ni )

We’ll want to calculate maxt P(t * wab)

Tree and sentence probabilities

P(t) -- The probability of tree is the product of the probabilities of the rules used to generate it

P(w1n) -- The probability of the sentence is the sum of the probabilities of the trees which have that sentence as their yield

P(w1n) = Σj P(w1n, t) where t is a parse of w1n

= Σj P(t)

Phrase structure grammars = context-free grammars (CFG)

G = (T, N, S, R)T is set of terminalsN is set of nonterminalsS is the start symbol (one of the

nonterminals)R is rules/productions of the form X ,

where X is a nonterminal and is a sequence of terminals and nonterminals (possibly an empty sequence)

Chomsky Normal Form (CNF) All rules are of the form X Y Z or X w

A transformation to this form doesn’t change the generative capacity of CFG

With some extra book-keeping in symbol names, you can even reconstruct the same trees with a de-transform Unaries/empties are removed recursively N-ary rules introduce new non-terminals (binarization):

VP V NP PP becomes VP V @VP-V and @VP-V NP PP

In practice it’s a pain Reconstructing n-aries is easy Reconstructing unaries can be trickier

But it makes parsing easier/more efficient

A treebank treeROOT

V NP PP

clawswithpeoplescratch

After binarization

@PP->_P

cats peoplescratch

V NP PP

@VP->_V

@VP->_V_NP

@S->_NP

CYK (Cocke-Younger-Kasami) algorithm

A bottom-up parser using dynamic programming

Assume the PCFG is in Chomsky normal form (CNF)

Maintain |N| nXn tables µ (|N| = number of non-terminals, n = number of input words [length of input sentence])

Fill out the table entries by induction

“Can1 you2 book3 ELAL4 flights5 ?”

w1,1 w1,2 w1,3 w1,4 w1,5

w2,2 w2,3 w2,4 w2,5

w3,3 w3,4 w3,5

w4,4 w4,5

1 2 3 4 5

CYK Base case

–Consider the input strings of length one (i.e., each individual word wi) P(A wi)

–Since the grammar is in CNF: A * wi iff A wi

–So µ[i, i, A] = P(A wi)

CYK Base case

“Can1 you2 book3 ELAL4 flights5 ?”

……

CYK Recursive case

For strings of words of length > 1,A * wij iff there is at least one rule A BCwhere B derives the first k words (between i and i-1 +k ) and C derives the remaining ones (between i+k and j)

(for each non-terminal)Choose the max among all possibilities

i i-1+k i+k j

µ[i, j, A)] = µ [i, i-1 +k, B] *

µ [i+k, j, C] *

P(A BC)

CYK Termination

The max prob parse will be µ [1, n, S]

w1,1 w1,2 w1,3 w1,4 w1,5

w2,2 w2,3 w2,4 w2,5

w3,3 w3,4 w3,5

w4,4 w4,5

Top down: Early algorithmFinds constituents and partial constituents in

inputA B C . D E is partial: only the first half of the A

B C D E

A B C . D E

B C D E

A B C D . E

i j i k

Early algorithm

Proceeds incrementally, left-to-rightBefore it reads word 5, it has already

built all hypotheses that are consistent with first 4 words

Reads word 5 & attaches it to immediately preceding hypotheses. Might yield new constituents that are then attached to hypotheses immediately preceding them …

Use a parse table as we did in CKY, so we can look up anything we’ve discovered so far. “Dynamic programming.”

Example (grammar)

ROOT S

S NP VP NP Papa

NP Det N N caviar

NP NP PP N spoon

VP VP PPV ate

VP V NP P with

PP P NP Det the

0 ROOT . Sinitialize

Remember this stands for (0, ROOT . S)

0 ROOT . S0 S . NP VP

predict the kind of S we are looking for

Remember this stands for (0, S . NP VP)

0 ROOT . S0 S . NP VP0 NP . Det N0 NP . NP PP0 NP . Papa

predict the kind of NP we are looking for(actually we’ll look for 3 kinds: any of the 3 will do)

0 ROOT . S0 S . NP VP0 NP . Det N0 NP . NP PP0 NP . Papa0 Det . the0 Det . a

predict the kind of Det we are looking for (2 kinds)

0 ROOT . S

0 S . NP VP

0 NP . Det N

0 NP . NP PP

0 NP . Papa

0 Det . the

0 Det . a

predict the kind of NP we’re looking for but we were already looking for these sodon’t add duplicate goals! Note that this happenedwhen we were processing a left-recursive rule.

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP0 NP . Det N0 NP . NP PP0 NP . Papa0 Det . the0 Det . a

scan: the desired word is in the input!

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP0 NP . Det N0 NP . NP PP0 NP . Papa0 Det . the0 Det . a scan: failure

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP0 NP . Det N0 NP . NP PP0 NP . Papa0 Det . the0 Det . a

scan: failure

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP 0 S NP . VP0 NP . Det N 0 NP NP . PP0 NP . NP PP0 NP . Papa0 Det . the0 Det . a

attach the newly created NP(which starts at 0) to its customers (incomplete constituents that end at 0and have NP after the dot)

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP 0 S NP . VP0 NP . Det N 0 NP NP . PP0 NP . NP PP 1 VP . V NP0 NP . Papa 1 VP . VP PP0 Det . the0 Det . a

predict

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP 0 S NP . VP0 NP . Det N 0 NP NP . PP0 NP . NP PP 1 VP . V NP0 NP . Papa 1 VP . VP PP0 Det . the 1 PP . P NP0 Det . a

predict

0 Papa 1

0 ROOT . S 0 NP Papa .0 S . NP VP 0 S NP . VP0 NP . Det N 0 NP NP . PP0 NP . NP PP 1 VP . V NP0 NP . Papa 1 VP . VP PP0 Det . the 1 PP . P NP0 Det . a 1 V . ate

predict

0 Papa 1

predict

0 Papa 1

1 P . withpredict

0 Papa 1 ate 2

0 ROOT . S 0 NP Papa . 1 V ate .0 S . NP VP 0 S NP . VP0 NP . Det N 0 NP NP . PP0 NP . NP PP 1 VP . V NP0 NP . Papa 1 VP . VP PP0 Det . the 1 PP . P NP0 Det . a 1 V . ate

1 P . with scan: success!

0 Papa 1 ate 2

0 ROOT . S 0 NP Papa . 1 V ate .0 S . NP VP 0 S NP . VP0 NP . Det N 0 NP NP . PP0 NP . NP PP 1 VP . V NP0 NP . Papa 1 VP . VP PP0 Det . the 1 PP . P NP0 Det . a 1 V . ate

1 P . with

scan: failure

0 Papa 1 ate 2

0 ROOT . S 0 NP Papa . 1 V ate .0 S . NP VP 0 S NP . VP 1 VP V . NP0 NP . Det N 0 NP NP . PP0 NP . NP PP 1 VP . V NP0 NP . Papa 1 VP . VP PP0 Det . the 1 PP . P NP0 Det . a 1 V . ate

1 P . with

attach

0 Papa 1 ate 2

0 ROOT . S 0 NP Papa . 1 V ate .0 S . NP VP 0 S NP . VP 1 VP V . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP0 Det . a 1 V . ate

1 P . with

predict

0 Papa 1 ate 2

0 ROOT . S 0 NP Papa . 1 V ate .0 S . NP VP 0 S NP . VP 1 VP V . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

predict (these next few stepsshould look familiar)

0 Papa 1 ate 2

1 P . with

predict

0 Papa 1 ate 2

1 P . with

scan (this time we fail sincePapa is not the next word)

0 Papa 1 ate 2 the 3

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the .0 S . NP VP 0 S NP . VP 1 VP V . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . withscan: success!

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the .0 S . NP VP 0 S NP . VP 1 VP V . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

0 Papa 1 ate 2 the 3 caviar 4

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

attach

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa0 Det . the 1 PP . P NP 2 Det . the0 Det . a 1 V . ate 2 Det . a

1 P . with

attach(again!)

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP .0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP0 Det . a 1 V . ate 2 Det . a

1 P . with

attach(again!)

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP .0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP0 Det . a 1 V . ate 2 Det . a 4 PP . P NP

1 P . with

1 P . with 0 ROOT S .

attach(again!)

1 P . with 0 ROOT S .

1 P . with 0 ROOT S .4 P . with

0 Papa 1 ate 2 the 3 caviar 4 with 5

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP .0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP0 Det . a 1 V . ate 2 Det . a 4 PP . P NP

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP .0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP0 Det . a 1 V . ate 2 Det . a 4 PP . P NP

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP0 Det . a 1 V . ate 2 Det . a 4 PP . P NP

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 Papa 1 ate 2 the 3 caviar 4 with 5 0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with . 5 Det a .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with . 5 Det a .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP 5 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with . 5 Det a .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP 5 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N 6 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP 6 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6 spoon 7

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with . 5 Det a . 6 N spoon .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP 5 NP Det . N0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N 6 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP 6 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6 spoon 70 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with . 5 Det a . 6 N spoon .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP 5 NP Det . N 5 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N 6 N . caviar0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP 6 N . spoon0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6 spoon 7

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . 4 P with . 5 Det a . 6 N spoon .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 4 PP P . NP 5 NP Det . N 5 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 5 NP . Det N 6 N . caviar 4 PP P NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP . NP PP 6 N . spoon 5 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 5 NP . Papa0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 5 Det . the0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 5 Det . a

0 Papa 1 ate 2 the 3 caviar 4 with a spoon 7

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . … 6 N spoon .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 5 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 4 PP P NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 2 NP NP PP .0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 1 VP VP PP .0 Det . a 1 V . ate 2 Det . a 4 PP . P NP

0 ROOT . S 0 NP Papa . 1 V ate . 2 Det the . 3 N caviar . … 6 N spoon .0 S . NP VP 0 S NP . VP 1 VP V . NP 2 NP Det . N 2 NP Det N . 5 NP Det N .0 NP . Det N 0 NP NP . PP 2 NP . Det N 3 N . caviar 1 VP V NP . 4 PP P NP .0 NP . NP PP 1 VP . V NP 2 NP . NP PP 3 N . spoon 2 NP NP . PP 5 NP NP . PP0 NP . Papa 1 VP . VP PP 2 NP . Papa 0 S NP VP . 2 NP NP PP .0 Det . the 1 PP . P NP 2 Det . the 1 VP VP . PP 1 VP VP PP .0 Det . a 1 V . ate 2 Det . a 4 PP . P NP 7 PP . P NP

1 P . with 0 ROOT S . 1 VP V NP .4 P . with 2 NP NP . PP

0 S NP VP .1 VP VP . PP

0 S NP VP .1 VP VP . PP7 P . with

0 S NP VP .1 VP VP . PP7 P . with0 ROOT S .

Parsing Based on presentations from Chris Manning’s course on Statistical Parsing (Stanford)

Documents