MaxForce: Max-Violation Perceptron and Forced Decoding for...

MaxForce: Max-Violation Perceptron and

Forced Decoding for Scalable MT Training

Heng Yu

Chinese Acad. of Sciences

Liang Huang

CUNY

Haitao Mi

IBM T. J. Watson

0 1 2 3 4 5 6

Bush

held

held talks

talks with

with Sharon

Sharon

Kai Zhao

CUNY

MaxForce: Max-Violation Perceptron and

Forced Decoding for Scalable MT Training

Heng Yu

Chinese Acad. of Sciences

Liang Huang

CUNY

Haitao Mi

IBM T. J. Watson

0 1 2 3 4 5 6

Bush

held

held talks

talks with

with Sharon

Sharon

Scalable Training for MT Finally Made Successful

Kai Zhao

CUNY

Discriminative Training for SMT• discriminative training is dominant in parsing / tagging

• can use arbitrary, overlapping, lexicalized features

• but not very successful yet in machine translation

• most efforts on MT training tune feature weights on the small dev set (~1k sents) not the training set!

• as a result can only use ~10 dense features (MERT)

• or ~10k rather impoverished features (MIRA/PRO)

• Liang et al (2006) train on the training set but failed

2

training set (>100k sentences) dev set (~1k sents)

test set (~1k sents)

Timeline for MT Training

3



MERT (Och ’02)

(dense features)


3



Standard Perceptron (a noble failure) (Liang et al 2006)

MERT (Och ’02)

(dense features)


3



Standard Perceptron (a noble failure) (Liang et al 2006) MIRA

(Watanabe+ ’07)(Chiang+ ’08-’12)

MERT (Och ’02)

(dense features)

(pseudo sparsefeatures)


3





PRO(Hopkins+May ’11)

Regression(Bazrafshan+ ’12)

MERT (Och ’02)

(dense features)



3







HOLS(Flanigan+ ’13)

(sparse features as one dense feature)

MERT (Och ’02)

(dense features)



3







our work (2013): violation-fixing perceptron with truly sparse features



MERT (Och ’02)

(dense features)



3







our work (2013): violation-fixing perceptron with truly sparse features



MERT (Och ’02)

(dense features)

(pseudo sparsefeatures)?

Why previous work fails

• their learning methods are based on exact search

• MT has huge search spaces => severe search errors

• learning algorithms should fix search errors

• full updates (perceptron/MIRA/PRO) can’t fix search errors

• MT involves latent variables (derivations not annotated)

• perceptron/MIRA was not designed for latent variables

• we need better variants for perceptron4

Why our approach works

• use a variant of perceptron tailored for inexact search

• fix search errors in the middle of the search

• “partial updates” instead of “full updates”

• use forced decoding lattice as the target to update to

• use parallelized minibatch to speed up learning

• result: scaled to a large portion of the training data

• 20M sparse features => +2.0 BLEU over MERT/PRO 5

MT as Structured Classification

• with latent variables (hidden derivations)

6

x

ythe man bit the dog

那人咬了狗



6

x


那人咬了狗

...

all gold derivations



6

x


那人咬了狗

...x那人咬了狗




6

x


那人咬了狗

...x

ythe dog bit the man

那人咬了狗

best derivation




6

x


那人咬了狗

...x


那人咬了狗

best derivation

all gold derivations wrong translation



6

x


那人咬了狗

...x


那人咬了狗

best derivation

best goldderivation




6

x


那人咬了狗

...x


那人咬了狗

best derivation

best goldderivation

update: penalize best derivationand reward best gold derivation


--++

Outline

• Motivations

• Phrase-based Translation and Forced Decoding

• Violation-Fixing Perceptron for SMT

• Update Strategies: Early Update and Max-Violation

• Feature Design

• Experiments

7

Phrase-based translation

yu Shalong juxing le huitan

与沙⻰龙举行了会谈

held talks with Sharon

布什Bushi

Bush

yu Shalong juxing le huitanwith Sharon held talks

meetingsSharon heldwithBush

Bushi





布什Bushi

Bush


meetingsSharon heldwith

_ _ _ _ _ _

Bush

Bushi





布什Bushi

Bush



_ _ _ _ _ _

Bush

Bushi

●_ _ _ _ _





布什Bushi

Bush



_ _ _ _ _ _ ●_ _●●●

Bush

Bushi

●_ _ _ _ _





布什Bushi

Bush



_ _ _ _ _ _ ●_ _●●● ●●●●●●

Bush

Bushi

●_ _ _ _ _





布什Bushi

Bush



_ _ _ _ _ _ ●_ _●●● ●●●●●●

Bush

Bushi

●_ _ _ _ _





布什Bushi

Bush



_ _ _ _ _ _ ●_ _●●● ●●●●●●

● ●_●●●

Bush

Bushi

●_ _ _ _ _

Language Model and Beam Search• split each -LM state into many +LM states

9


9

●_ _ _ _ _ Bush


9

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●_ _ _ _ _ Bush


9

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●●●●●● ... Sharon

●●●●●● ... Shalong

●_ _ _ _ _ Bush


9

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●●●●●● ... Sharon

●●●●●● ... Shalong

●_ _ _ _ _ Bush

● ● ● ● ● ●

Forced Decoding

Bushi yu Shalong juxing le huitan Bush held talks with Sharon

• both as data selection (more literal) and oracle derivations

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon

held talks with Sharongold derivation lattice

Forced Decoding

●_ _ _ _ _ Bush



_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


Forced Decoding

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●_ _ _ _ _ Bush



_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


Forced Decoding

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●●●●●● ... Sharon

●●●●●● ... Shalong

●_ _ _ _ _ Bush



_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


Forced Decoding

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●●●●●● ... Sharon

●●●●●● ... Shalong

●_ _ _ _ _ Bush



_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


Forced Decoding

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●●●●●● ... Sharon

●●●●●● ... Shalong

●_ _ _ _ _ Bush



_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


Forced Decoding

●_ _●●● ... talks

●_ _●●● ... talk

●_ _●●● ... meeting

●●●●●● ... Sharon

●●●●●● ... Shalong

●_ _ _ _ _ Bush


one gold derivation


_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


Unreachable Sentences and Prefix

11

Lianheguo

paiqian

50mıng

guanchaiyuan

jiandu

Bolıweiya

huıfumınzhu

zhengzhı

yılaishoucı

quanguo

daxuan

U.N.

sent

50

observers

to

monitor

the

1st

election

since

Bolivia

restored

democracy

5

33

4

1

玻利维亚

恢复民主政治以来首次全国大选联合

国派遣 50

名观察员

监督

• distortion limit causes unreachability (hiero would be better)

• but we can still use reachable prefix-pairs of unreachable pairs

Unreachable Sentences and Prefix

11

Lianheguo

paiqian

50mıng

guanchaiyuan

jiandu

Bolıweiya

huıfumınzhu

zhengzhı

yılaishoucı

quanguo

daxuan

U.N.

sent

50

observers

to

monitor

the

1st

election

since

Bolivia

restored

democracy

5

33

4

1

玻利维亚

恢复民主政治以来首次全国大选联合

国派遣 50

名观察员

监督

• distortion limit causes unreachability (hiero would be better)

• but we can still use reachable prefix-pairs of unreachable pairs

Sentence/Word Reachability Ratio• how many sentences pairs pass forced decoding?

• the ratio drops dramatically as sentences get longer

• prefixes boost coverage

12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10 20 30 40 50 60 70

Ratio

of co

mple

te c

ove

rage

Sentence length

Distortion-unlimitDistortion-limit 6Distortion-limit 4Distortion-limit 2Distortion-limit 0




12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10 20 30 40 50 60 70

Ratio

of co

mple

te c

ove

rage

Sentence length





12

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10 20 30 40 50 60 70

Ratio

of co

mple

te c

ove

rage

Sentence length


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10 20 30 40 50 60 70

Ra

tio o

f co

mp

lete

co

vera

ge

Sentence length

dist-6dist-4dist-2dist-0

Number of Gold Derivations

• exponential in sentence length (on fully reachables)

• these are the “latent variables” in learning

13

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

5 10 15 20 25 30 35 40 45 50

Ave

rage n

um

ber

of deriva

tions

Sentence length

dist-6dist-4dist-2dist-0

Outline

• Background: Phrase-based Translation (Koehn, 2004)

• Forced Decoding

• Violation-Fixing Perceptron for MT Training

• Update strategy

• Feature design

• Experiments

14

Structured Perceptron (Collins 02)

15

x

y=-1y=+1

x

y

update weights

if y ≠ z

w

x zexactinference

binary classification


15

x

y=-1y=+1

x

y

update weights

if y ≠ z

w

x zexactinference


structured classification


15

x


那人咬了狗

x

y=-1y=+1

x

y

update weights

if y ≠ z

w

x zexactinference




• challenges in applying perceptron for MT

• the inference (decoding) is vastly inexact (beam search)

• we know standard perceptron doesn’t work for MT

• intuition: the learner should fix the search error first15

x


那人咬了狗

y

update weights

if y ≠ z

w

x zexactinference

x

y=-1y=+1

x

y

update weights

if y ≠ z

w

x zexactinference

constant# of classes

exponential # of classes




• challenges in applying perceptron for MT

• the inference (decoding) is vastly inexact (beam search)

• we know standard perceptron doesn’t work for MT

• intuition: the learner should fix the search error first15

x


那人咬了狗

y

update weights

if y ≠ z

w

x zexactinference

x

y=-1y=+1

x

y

update weights

if y ≠ z

w

x zexactinference

constant# of classes

exponential # of classes



inexactinference

Search Error: Gold Derivations Pruned

16

_ _ _ _ _ _

0 1 2 3 4 5 6

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


real decoding beam search


16

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




16

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




16

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




16

_ _ _ _ _ _ ● _ _ ● ● ●

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




16

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




16

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon



should fix search errors here!

Fixing Search Error 1: Early Update

17

standard update(no guarantee!)

21

Model


• early update (Collins/Roark’04) when the correct falls off beam

• up to this point the incorrect prefix should score higher

• that’s a “violation” which we want to fix

17


21

Model





17

correct sequencefalls off beam

(pruned)

correct


21

Model





17


(pruned)

correct

incorrect


21

Model





17

earl

y up

date


(pruned)

correct

incorrect

violation guaranteed: incorrect prefix scores higher up to this point


21

Model





• standard perceptron does not guarantee violation

• w/ pruning, the correct seq. might score higher at the end!

• called “invalid” update b/c it doesn’t fix the search error

17

earl

y up

date


(pruned)

correct

incorrect



21

Model

Early Update w/ Latent Variable

18

21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon


• the gold-standard derivations are not annotated

• we treat any reference-producing derivation as good


18

21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




correct


18

21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




correct


18

21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




correct

all correct derivations fall off


18

incorrect

21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




correct



18

earl

y up

date

incorrect


21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




correct



18

earl

y up

date

incorrect


21

Model

_ _ _ _ _ _ ● _ _ _ _ _ ● _ _ ● ● _ ● _ _ ● ● ● ● ● _ ● ● ● ● ● ● ● ● ●

0 1 2 3 4 5 6

Bushheld

talks with Sharon




correct


stop decoding

Fixing Search Error 2: Max-Violation

19

• early update works but learns slowly due to partial updates

• max-violation: use the prefix where violation is maximum

• “worst-mistake” in the search space

• we call these methods “violation-fixing perceptrons” (Huang et al 2012)

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l

standard update is invalid

mod

elw

d�|x|

Early Update vs. Max-Violation

_ _ _ _ _ _

0 1 2 3 4 5 6

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●ea

rly

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


Early-update

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


Early-update

_ _ _ _ _ _ ● _ _ ● ● ●

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


Early-update

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


Early-update

● ● _ ● ● ●

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _● ● ● ● _ ●● _ ● ● ● ●● ● ● _ ● ●

● ● ● _ ● ●

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


Early-update

● ● _ ● ● ●

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _● ● ● ● _ ●● _ ● ● ● ●● ● ● _ ● ●

● ● ● _ ● ● ● ● ● ● ● ●

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|


Early-update

● ● _ ● ● ●

_ _ _ _ _ _

0 1 2 3 4 5 6

● _ _ _ _ _

_ _ ●_ _ __ _ _ _ ● _

_ _ _ _ _ ●● ● _ ● _ _

_ ● ● ● _ __ _ ● ● ●_

_ ● ● ● _ _

● _ _ ● ● _

● _ ● ● _ ●_ ● _ ● ● ●

● _ _ ● ● ●_ ● ● _ _ __ _ ● ●_ _

_ _ ● _ ● _

_ ● _ _ ● _● ● ● ● _ ●● _ ● ● ● ●● ● ● _ ● ●

● ● ● _ ● ●

Max-violation

● ● ● ● ● ●

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|

Latent-Variable Perceptron

early

max

-vi

olat

ion

late

st

full

(standard)

best in the beam

worst in the beamfalls off

the beam biggestviolation

last valid update

correct sequence

invalidupdate!

early

max

-vi

olat

ion

best in the beam

worst in the beam

d�i

d+id+i⇤

d�i⇤d+|x|

dy|x|

std

loca

l


mod

elw

d�|x|

Roadmap of the techniques

22

structured perceptron(Collins, 2002)


22


latent-variable perceptron

(Zettlemoyer and Collins, 2005; Sun et al., 2009)


22




perceptron w/ inexact search

(Collins & Roark, 2004;Huang et al 2012)


22






latent-variable perceptron w/ inexact search

(Yu et al 2013)


22







(Yu et al 2013)

hiero syntactic parsing semantic parsing transliteration

Feature Design

• Dense features:

• standard phrase-based features (Koehn, 2004)

• Sparse Features:

• rule-identification features (unique id for each rule)

• word-edges features

• lexicalized local translation context within a rule

• non-local features

• dependency between consecutive rules

23

WordEdges Features (local)

24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

• the first and last Chinese words in the rule

• the first and last English words in the rule

• the two Chinese words surrounding the rule


24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>




Combo Features:


24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>




Combo Features:


24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>




Combo Features:100010=沙⻰龙|held


24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>






24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>






24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>




Combo Features:100010=沙⻰龙|held010001=举行|talks


24


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>




Combo Features:100010=沙⻰龙|held010001=举行|talks

Lexical backoffs and combos

• Lexical features are often too sparse

• 6 kinds of lexical backoffs with various budgets

• total budget can’t exceed 10 (bilexical)

25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

100010=沙⻰龙|held





25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

P00010=NN|held






25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

P00010=NN|held






25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

P00010=NN|held


010001=举行|talks





25


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

P00010=NN|held

0c0001=举|talks


010001=举行|talks

Non-Local Features (trivial)

26


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>

• two consecutive rule ids (rule bigram model)

• the last two English words and the current rule

• should explore a lot more!


26


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>





26


held a few talks

</s>

r2

布什

Bush

r1

<s>

<s>




Experiments

27

• Date sets

Experiments

27

Scale Language sent. dev tst

SmallCh-En

30knist06 news nist08 news

LargeCh-En


Large Sp-En 170k newstest2012 newtest2013

• Date sets

Experiments

27


SmallCh-En


LargeCh-En



• Date sets

Experiments

27


SmallCh-En


LargeCh-En



• Date sets

Experiments

27

10x dev


SmallCh-En


LargeCh-En



• Date sets

Experiments

27

10x dev 120x dev


SmallCh-En


LargeCh-En



• Date sets

Experiments

27

10x dev 120x dev


SmallCh-En


LargeCh-En



• Date sets

Experiments

27

10x dev 120x dev


SmallCh-En


LargeCh-En



• Date sets

Sp-En sent. word.ratio 55% 43.9%

Experiments

27

10x dev 120x dev


SmallCh-En


LargeCh-En



• Date sets

Sp-En sent. word.ratio 55% 43.9%

31x dev

Perceptron: std, early, and max-violation• standard perceptron (Liang et al’s “bold”) works poorly

• b/c invalid update ratio is very high (search quality is low)

• max-violation converges faster than early update

17

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16 18 20

BLE

U

Number of iteration

MaxForce

MERTearly

local

standard

28




this explains why Liang et al ’06 failedstd ~ “bold”; local ~ “local”

17

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16 18 20

BLE

U

Number of iteration

MaxForce

MERTearly

local

standard

28




50%

60%

70%

80%

90%

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Ratio

beam size

Ratio of invalid updates+non-local feature

(standard perceptron)


17

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16 18 20

BLE

U

Number of iteration

MaxForce

MERTearly

local

standard

28




50%

60%

70%

80%

90%

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Ratio

beam size

Ratio of invalid updates+non-local feature

(standard perceptron)


17

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16 18 20

BLE

U

Number of iteration

MaxForce

MERTearly

local

standard

28

Parallelized Perceptron

29

• mini-batch perceptron (Zhao and Huang, 2013) much faster than iterative parameter mixing (McDonald et al, 2010)

• 6 CPUs => ~4x speedup; 24 CPUs => ~7x speedup

22

23

24

0 0.5 1 1.5 2 2.5 3 3.5 4

BLE

U

Time

MERT PRO-dense

minibatch(24-core)minibatch(6-core)minibatch(1 core)single processor

Time

Internal comparison with different features

• dense: 11 standard features for phrase-based MT

• ruleid: rule identification feature

• word-edges: word-edges features with back-offs

• non-local: non-local features with back-offs

30

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16

BLE

U

Number of iteration

MERT

+non-local+word-edges

+ruleiddense






30

dense: 11 features

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16

BLE

U

Number of iteration

MERT


+ruleiddense






30

ruleid: 0.1%

dense: 11 features

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16

BLE

U

Number of iteration

MERT


+ruleiddense

+0.9 bleu






30

ruleid: 0.1%

wordedges: 99.6%

dense: 11 features

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16

BLE

U

Number of iteration

MERT


+ruleiddense

+0.9 bleu

+2.3






30

ruleid: 0.1%

wordedges: 99.6%

non-local: 0.3%

dense: 11 features

18

19

20

21

22

23

24

25

26

2 4 6 8 10 12 14 16

BLE

U

Number of iteration

MERT


+ruleiddense

+0.9 bleu

+2.3+0.7

External comparison with MERT & PRO

31

• MERT, PRO-dense/medium/sparse all tune on dev-set

• PRO-sparse use the same feature as ours

10

12

14

16

18

20

22

24

26

2 4 6 8 10 12 14 16

BLE

U

Number of iteration

MaxForceMERT

PRO-densePRO-medium

PRO-large

Final Results on FBIS Data

32

• Moses: state-of-the-art phrase-based system in C++

• Cubit: phrase-based system (Huang and Chiang, 2007) in python

• almost identical baseline scores with MERT

• max-violation takes ~47 hours on 24 CPUs (23M features)


32

System Alg. Tune on Features Dev TestMoses

CubitCubitCubitCubitCubit

MERT dev set 11 25.5 22.5


PRO dev set11 25.6 22.6

PRO dev set 3k 26.3 23.0PRO dev set36k 17.7 14.3

MaxForce Train set 23M 27.8 24.5






32





PRO dev set11 25.6 22.6








32





PRO dev set11 25.6 22.6








32





PRO dev set11 25.6 22.6








32





PRO dev set11 25.6 22.6








32





PRO dev set11 25.6 22.6







+2.3


32





PRO dev set11 25.6 22.6







+2.3 +2.0

Results on Spanish-English set

• Data-set: Europarl corpus, 170k sentences

• dev/test set: newtest2012 / 2013 (one-reference only)

• +1 in 1-ref bleu ~ +2 in 4-ref bleu

• bleu improvement is comparable to Chinese w/ 4-refs

33

system algorithm #feat. dev test

Moses Mert 11 27.4 24.4

Cubit MaxForce 21M 28.7 25.5

Sp-En sent. word.Reachable ratio 55% 43.9%

Results on Spanish-English set

• Data-set: Europarl corpus, 170k sentences

• dev/test set: newtest2012 / 2013 (one-reference only)

• +1 in 1-ref bleu ~ +2 in 4-ref bleu

• bleu improvement is comparable to Chinese w/ 4-refs

33

system algorithm #feat. dev test

Moses Mert 11 27.4 24.4

Cubit MaxForce 21M 28.7 25.5

+1.3 +1.1Sp-En sent. word.

Reachable ratio 55% 43.9%

Conclusion• a simple yet effective online learning approach for MT

• scaled to (a large portion of) the training set for the first time

• able to incorporate 20M sparse lexicalized features

• no need to define BLEU+1, or hope/fear derivations

• no learning rate or hyperparameters

• +2.3/+2.0 BLEU points better than MERT/PRO

• the three ingredients that made it work

• violation-fixing perceptron: early-update and max-violation

• forced decoding lattice helps

• minibatch parallelization scales it up to big data34


35



35





35







35







(Yu et al 2013)


35







(Yu et al 2013)



35







(Yu et al 2013)


replacing EM for partially-

observed data

20 years of Statistical MT• word alignment: IBM models (Brown et al 90, 93)

• translation model (choose one from below)

• SCFG (ITG: Wu 95, 97; Hiero: Chiang 05, 07) or STSG (GHKM 04, 06; Liu+ 06; Huang+ 06)

• PBMT (Och+Ney 02; Koehn et al 03)

• evaluation metric: BLEU (Papineni et al 02)

• decoding algorithm: cube pruning (Chiang 07; Huang+Chiang 07)

• training algorithm (choose one from below)

• MERT (Och 03): ~10 dense features on dev set

• MIRA (Chiang et al 08-12) or PRO (Hopkins+May 11): ~10k feats on dev set

• MaxForce: 20M+ feats on training set; +2/+1.5 BLEU over MERT/PRO

• Max-Violation Perceptron with Forced Decoding: fixes search errors

• first successful effort of online large-scale discriminative training for MT

When learning with vastly inexact search, you should use a principled method such as max-violation.

Thank you!

Max-violation

Date post:	25-Jun-2020
Category:	Documents
Upload:	others
View:	11 times
Download:	0 times

MaxForce: Max-Violation Perceptron and Forced Decoding for...

Documents