Post on 10-Apr-2018
transcript
8/8/2019 Bangla Machine Translation
1/47
Master Thesis
English to Bangla Phrase-Based Statistical Machine
Translation
Md. Zahurul Islam2519786
Supervisors:
Dr. Andreas Eisele
PD. Dr. Valia Kordoni
Prof. Hans Uszkoreit
Department of Computational linguistics
Saarland University
Germany
&
Dr. Jrg TiedemannDepartment of Humanities Computing
University of Groningen
The Netherlands
Submitted to
Department of Computational linguistics
atSaarland University
Erasmus Mundus European Masters Program in
Language and Communication Technologies (LCT)
August 2009
8/8/2019 Bangla Machine Translation
2/47
Master Thesis
English to Bangla Phrase-Based Statistical Machine
Translation
Md. Zahurul Islam
1788213
Supervisors:
Dr. Jrg Tiedemann
Department of Humanities Computing
University of Groningen
The Netherlands
&
Dr. Andreas Eisele
DFKI GmbH, Language Technology Lab andDepartment of Computational linguistics
Saarland University
Germany
Submitted to
Faculty of Arts
atUniversity of Groningen
Research Master Linguistics
Erasmus Mundus European Masters Program in
Language and Communication Technologies (LCT)
August 2009
8/8/2019 Bangla Machine Translation
3/47
8/8/2019 Bangla Machine Translation
4/47
8/8/2019 Bangla Machine Translation
5/47
8/8/2019 Bangla Machine Translation
6/47
8/8/2019 Bangla Machine Translation
7/47
8/8/2019 Bangla Machine Translation
8/47
8/8/2019 Bangla Machine Translation
9/47
8/8/2019 Bangla Machine Translation
10/47
8/8/2019 Bangla Machine Translation
11/47
8/8/2019 Bangla Machine Translation
12/47
S > N P+ V P
8/8/2019 Bangla Machine Translation
13/47
!"#$ %' !"(
8/8/2019 Bangla Machine Translation
14/47
)*#+ ,-!
P r (w)
P r (e|w)
P r (j|e)
P r (k|j)
P r (o|k)
8/8/2019 Bangla Machine Translation
15/47
P r (w) .P r (e|w) .P r (j|e) .P r (k|j) .P r (o|k)
P r (w)
8/8/2019 Bangla Machine Translation
16/47
eI1
= e1 . . . ei . . . eI fJ1 = f1 . . . f j . . . f J
fj1
= argminJ,fJ1
Jfj
1
P r
fJ
1 |eI1
.L
fJ1
, fJ
1
L
fJ1
, fJ
1
fJ1
fJ
1
L
fJ1 , fJ
1
=
0 fJ1 = f
J
1
1
= 1
fJ1 , fJ
1
8/8/2019 Bangla Machine Translation
17/47
fJ1
, fJ
1
f
J1 = argmax
P r
fJ1 |eI1
fJ1
eI1
P r
fJ1 |e
I1
P r
fJ1|eI1
=
P r
fJ1
P r
eI1|f
J1
P r
eI1
P r
eI1
eI1
argmax
P r
fJ1 |eI1
= argmax
P r
fJ1
P r
eI1|fJ1
P r
fJ1
P r
eI1|fJ1
P r
eI1|f
J1
P r
fJ1
8/8/2019 Bangla Machine Translation
18/47
fJ1
eI1
e1, e2 . . . eI e
I1
fJ1
fJ1|eI1
fJ1 eI1
d (aj bj1) aj ith ei bj1 i 1th ei1 d (aj bj1) =
|ajbj11|
P r
fJ1 |eI1
=
I,Ji,j=1
(fj , ei) d (aj bj1)
8/8/2019 Bangla Machine Translation
19/47
e = eI
1
f = fJ1
f = argmaxf {P r (f|e)}
f = argmaxf {P r (e|f) .P r (f)}
P r (e|f) P r (f)
P r (e|f) =a
P r (e, a|f)
a a = aI
1= a1 . . . ai . . . aI, ai
{0, . . . J } ai = 0 p (ai|ai1, J) ai aj1 p ((|ai ai1|) |J)
8/8/2019 Bangla Machine Translation
20/47
8/8/2019 Bangla Machine Translation
21/47
8/8/2019 Bangla Machine Translation
22/47
./0#1. 2345,67,893:
)"#;
8/8/2019 Bangla Machine Translation
23/47
%/W .;*/^F %/W
_,&*/F
'PQF _,&*/F
#'/Y )( `F #'/Y &/F .aE*F &/F
.#a .$*. ,-! )*#+ )*#+/! ,b .aE* .aE*b
c*;* deFQ c*/;eFQ )f" gh.) )f"h.)
!"#$ `)i %#$ %$F*
8/8/2019 Bangla Machine Translation
24/47
8/8/2019 Bangla Machine Translation
25/47
8/8/2019 Bangla Machine Translation
26/47
8/8/2019 Bangla Machine Translation
27/47
8/8/2019 Bangla Machine Translation
28/47
8/8/2019 Bangla Machine Translation
29/47
8/8/2019 Bangla Machine Translation
30/47
`;#1 `j^ %&'*F %('W! K#f;*F 'E*ME )E).*
'E*ME )E).* )*fE!*$k^;O*/) `;#1'!"' U!*iU( .*f*Fl #$m K/n*)F g#'UU
`( 'b #D.*/) ;#o' `(#1 #D/./) p#' `)i;F/! -.1* M*b#' M/qr &*/F's
D/b/t
T!E*uE*! -M `1*
;*v'Zw* ;*v'Zw*
-.x#P %F) #;iy$ -.x#P %F)
$z!" . $#FU*. -;*{*#F;* -;*{*#F;*
#|;* |Q.
8/8/2019 Bangla Machine Translation
31/47
K#=. Z' #O;*F. )*', #;,$"'
`;#1 U&F `j/^* %&'*F %('W! K#f;*F 'E*ME )E).*
'E*ME )E).* )*fE!*$k^;O*/) `;#1
'!"' U!*iU(.*f*Fl #$m Z' #O;*F. K/n*)Fg#'UU
`( 'b #D.*/) ;#o' `(#1 #D/./) p#' `)i;F/! -.1* M*b#' M/qr &*/F's
$*$y D$P'ZP D/b/t T!E*uE*! -M -}*^;~ `1* #}q s
)*', #;,$"'
8/8/2019 Bangla Machine Translation
32/47
F)!
8/8/2019 Bangla Machine Translation
33/47
`;#1 `$' #'/P
8/8/2019 Bangla Machine Translation
34/47
`;#1 -U*&&*F `$' #'/P
8/8/2019 Bangla Machine Translation
35/47
8/8/2019 Bangla Machine Translation
36/47
n
8/8/2019 Bangla Machine Translation
37/47
ACC = 1N
Ni=1
{1 ri = ci, 0 }
Top J =1
N
Ni=1
{1 ci,j ci,j = ri, 0 }
LCS(c, r) = 1/2 (length (c) + length (r) editDistance (c, r))
ith
Ri =LCS(ci, ri)
length (ri)Pi =
LCS(ci, ri)
length (ci)Fi = 2
Ri Pi
Ri + Pi
8/8/2019 Bangla Machine Translation
38/47
8/8/2019 Bangla Machine Translation
39/47
8/8/2019 Bangla Machine Translation
40/47
8/8/2019 Bangla Machine Translation
41/47
`;#1 FY*^'* ;/F `;#1 *F F #'/P
8/8/2019 Bangla Machine Translation
42/47
8/8/2019 Bangla Machine Translation
43/47
8/8/2019 Bangla Machine Translation
44/47
8/8/2019 Bangla Machine Translation
45/47
8/8/2019 Bangla Machine Translation
46/47
8/8/2019 Bangla Machine Translation
47/47