1

Statistical NLPSpring 2007

Lecture 12: Phrase DecodingDan Klein – UC Berkeley

Overview: Extracting Phrases

Sentence-aligned corpus

cat ||| chat ||| 0.9 the cat ||| le chat ||| 0.8dog ||| chien ||| 0.8 house ||| maison ||| 0.6 my house ||| ma maison ||| 0.9language ||| langue ||| 0.9 …

Phrase table(translation model)

Intersected and grown word alignments

Directional word alignments

2

Pharaoh’s Model

[Koehn et al, 2003]

Segmentation Translation Distortion

Pharaoh’s Model

Where do we get these counts?

3

Phrase-Based Decoding

这 7人 中包括 来自 法国 和 俄罗斯 的 宇航 员 .

Decoder design is important: [Koehn et al. 03]

Phrase Scoring

les chatsaiment

lepoisson

cats

like

fresh

fish

.

.frais

.

Learning weights has been tried, several times:

[Marcu and Wong, 02][DeNero et al, 06]… and others

Seems not to work, for a variety of only partially understood reasons

Main issue: big chunks get all the weight, obvious priors don’t help

4

Extracting Phrases

Phrase SizePhrases do help

But they don’t need to be longWhy should this be?

5

Bidirectional Alignment

Alignment Heuristics

6

Sources of Alignments

Lexical Weighting

7

The Pharaoh Decoder

Probabilities at each step include LM and TM

Hypotheis Lattices

8

Pruning

Problem: easy partial analyses are cheaperSolution 1: use beams per foreign subsetSolution 2: estimate forward costs (A*-like)

WSD?Remember when we discussed WSD?

Word-based MT systems rarely have a WSD stepWhy not?

9

What’s Next?Modeling syntax

PCFGs and phrase structureSyntactic parsingGrammar inductionSyntactic language and translation models

Phrase Structure ParsingPhrase structure parsing organizes syntax into constituents or bracketsIn general, this involves nested treesLinguists can, and do, argue about detailsLots of ambiguity

Not the only kind of syntax…

new art critics write reviews with computers

PP

NPNP

N’

NP

VP

S

10

Constituency Tests

How do we know what nodes go in the tree?

Classic constituency tests:Substitution by proform

Question answers

Semantic reference

Dislocation

Cross-linguistic arguments, too

Conflicting TestsConstituency isn’t always clear

Units of transfer:think about ~ penser àtalk about ~ hablar de

Phonological reduction:I will go → I’ll goI want to go → I wanna goa le centre → au centre

La vélocité des ondes sismiques

11

Non-Local PhenomenaDislocation / gapping

Why did the postman think that the neighbors were home?A debate arose which continued until the election.

BindingReference

The IRS audits itselfControl

I want to goI want you to go

Regularity of RulesArgumentationAdjunctionCoordinationX’ Theory

12

PP Attachment

PP Attachment

13

Attachment is a Simplification

I cleaned the dishes from dinner

I cleaned the dishes with detergent

I cleaned the dishes in the sink

Syntactic Ambiguities I

Prepositional phrases:They cooked the beans in the pot on the stove with handles.

Particle vs. preposition:A good pharmacist dispenses with accuracy.The puppy tore up the staircase.

Complement structuresThe tourists objected to the guide that they couldn’t hear.She knows you like the back of her hand.

Gerund vs. participial adjectiveVisiting relatives can be boring.Changing schedules frequently confused passengers.

14

Syntactic Ambiguities IIModifier scope within NPsimpractical design requirementsplastic cup holder

Multiple gap constructionsThe chicken is ready to eat.The contractors are rich enough to sue.

Coordination scope:Small rats and mice can squeeze into holes or cracks in the wall.

Treebank Sentences

15

Human ProcessingGarden pathing:

Ambiguity maintenance