TaaS D4.4 Integration with SMT systems with SMT systems.pdf · 2016. 10. 31. · Contract no....

TaaS Terminology as a Service

Project no. 296312

Deliverable D4.4

Integration with SMT systems

Version No. 2.0

31/05/2014

Contract no. 296312

D4.4 V 2.0 Page 2 of 49

Document Information

Deliverable number: D4.4

Deliverable title: Integration with SMT systems

Due date of deliverable: 31/03/2014

Main Author(s): Mārcis Pinnis

Participants: TILDE

Internal reviewer: USFD

Workpackage: WP4

Workpackage title: Terminology services: usage scenarios

Workpackage leader: Kilgray

Dissemination Level: PU

Version: 2.0

Keywords: terminology, SMT, domain adaptation, translation level

integration, training level integration

History of Versions

Version Date Status Name of the

Author

(Partner)

Contributions Description/ Approval

Level

0.1 27/02/

2014

DRAFT Tilde Fishbone The fishbone of the

deliverable drafted with

initial content

0.2 28/03/

2014

DRAFT Tilde Document draft

version

The initial contribution

from Tilde included in

the deliverable.

0.3 29/03/

2014

DRAFT USFD Review of the

deliverable

Review of the

deliverable

1.0 31/03/

2014

FINAL Tilde Updates according to

the review and final

contributions

Preparation of the final

(first) version

Contract no. 296312

D4.4 V 2.0 Page 3 of 49

2.0 31/05/

2014

FINAL Tilde Updates for the

updated version of the

deliverable

Preparation of the

updated version

EXECUTIVE SUMMARY

The Deliverable D4.4 Integration with SMT systems describes scenarios for terminology

(TaaS terminology service) integration in SMT systems, documents the work done on

terminology integration in the LetsMT SMT platform and provides preliminary test results of

the terminology integration experiments within the LetsMT platform through experimental

interfaces. Complete evaluation of the terminology service integration in SMT systems will

be provided as part of the evaluation report in M24.

In the updated version we have revised and updated the following sections: Section 3 “TaaS

API for Machine Translation Systems”, Section 4 “Training Level Integration”, Section 5

“Translation Level Integration”, Section 6 “Conclusions”.

Contract no. 296312

D4.4 V 2.0 Page 4 of 49

Table of Contents

Abbreviations ................................................................................................................... 5 1. Introduction ................................................................................................................. 6 2. Related Work ............................................................................................................... 8

3. TaaS API for Machine Translation Systems .......................................................... 11 4. Training Level Integration of Terminology ............................................................ 13

4.1. Terminology as a Corpus ................................................................................... 14

4.1.1. Test Scenarios and Results ............................................................................ 16

4.2. Translation Model Adaptation ........................................................................... 17

4.2.1. Phrase Table Adaptation ............................................................................... 18 4.2.2. Phrase Filtering ............................................................................................. 20

4.2.3. Test Scenarios and Results ............................................................................ 22

4.3. Language Model Adaptation .............................................................................. 23

4.3.1. In-domain Corpus Acquisition from the Web ............................................... 24

4.3.2. Monolingual Corpora Splitting ..................................................................... 25

4.3.3. Parallel Corpora Splitting .............................................................................. 26 4.3.4. Test Scenarios and Results ............................................................................ 26

5. Translation Level Integration of Terminology ....................................................... 28

5.1. Identification of Terms in Translatable Documents ........................................... 29

5.1.1. TWSC-based Term Identification ................................................................. 29 5.1.2. Fast Term Identification ................................................................................ 31 5.1.3. Valid Phrase-based Term Identification ........................................................ 32

5.2. Acquisition of Translation Equivalents for the Identified Terms ...................... 33

5.2.1. No Translation Equivalent Acquisition ......................................................... 33 5.2.2. Morphological Synthesis-based Translation Equivalent Acquisition ........... 34 5.2.3. Monolingual Corpus-based Translation Equivalent Acquisition .................. 36

5.3. Ranking the Translation Equivalents ................................................................. 37 5.4. Test Scenarios and Results ................................................................................. 38

6. Conclusions ................................................................................................................ 45 References ...................................................................................................................... 46 List of tables ................................................................................................................... 49

Contract no. 296312

D4.4 V 2.0 Page 5 of 49

Abbreviations

Table 1 Abbreviations

Abbreviation Term/definition

ACCURAT Analysis and evaluation of Comparable Corpora for Under

Resourced Areas of machine Translation

API Application Programming Interface

BiTES Bilingual Term Extraction System

BLEU Bilingual Evaluation Understudy

CAT Computer Assisted Translation

DGT-TM Directorate-General for Translation – Translation Memory

FMC Focussed Monolingual Crawler

FST Finite State Transducer

IATE InterActive Terminology for Europe

ID Identifier

IDF Inverse Document Frequency

LISA Localization Industry Standards Association

MERT Minimum Error Rate Training

METEOR Metric for Evaluation of Translation with Explicit ORdering

MT Machine Translation

NE Named Entity

NIST MT evaluation metric from the USA National Institute of

Standards and Technology

POS Part Of Speech

QA Quality Assessment

SMT Statistical Machine Translation

TaaS Terminology as a Service

TBX Term Base eXchange

TER Translation Error Rate

TF*IDF Term Frequency–Inverse Document Frequency

TTC Terminology Extraction, Translation Tools and Comparable

Corpora

TWSC Tilde’s Wrapper system for CollTerm

URL Uniform Resource Locator

XML Extensible Markup Language

Contract no. 296312

D4.4 V 2.0 Page 6 of 49

1. Introduction

Terminology is one of key features that is assessed when performing manual quality

evaluation of translations, for instance, when using the QA Model1 of the Localization

Industry Standards Association (LISA). Many localisation service providers use this model

directly or in adapted forms. The Quality Assessment (QA) form for translations in Tilde,

which is based on the LISA QA Model also requires to evaluate the quality of terminology.

Thus, for statistical machine translation (SMT) systems it is important to provide support for

correct handling of terminology by assuring the two main quality requirements:

1) Terminology has to be used correctly (i.e., if a term glossary is provided, the

translations for terms have to be selected only from the provided glossary).

2) Terminology has to be used consistently (i.e., if a term appears multiple times in a

document, only one translation should be used for the translation of the term).

For machine translation (MT) systems, the first requirement is difficult to achieve,

because the context (or more precisely, the lack of enough context) may not always allow

identifying the correct translations of terms. The second requirement challenges SMT systems

more than rule-based MT systems as the statistics of large amounts of data is difficult to

control if not constrained by means of, e.g., bilingual term glossaries or translation model or

language model domain adaptation techniques. If SMT systems are not developed and

“taught” to understand terminology, ambiguous or unknown contexts in the parallel training

data may result in the selection of incorrect translation hypotheses because of higher

contextual likelihood.

Therefore, the task T4.3 “Terminology for SMT systems” is focussed on the integration

of user tailored terminology glossaries into SMT systems in order to achieve domain

adaptation and produce better quality translations. In this deliverable we describe the

experiments that have been performed and experimental interfaces that have been developed

for the LetsMT (Vasiļjevs et al., 2010) SMT platform in order to allow exploiting TaaS

terminology resources and services. Although being developed for the LetsMT platform, the

experimental interfaces are not only limited to the LetsMT platform and can be reused by

other SMT platforms that are based on the Moses (Koehn et al., 2007) SMT system with

minimal integration efforts (configuration of the Moses Experiment Management System) and

also potentially by other SMT platforms by developing input/output data converters (or

wrappers).

The deliverable documents experimental interfaces for terminology integration in SMT

systems both in SMT training level as well as translation level. The Figure 1 shows the

overall conceptual design of the terminology integration scenarios documented in this

deliverable.

The further sections are structured as follows: section 2 briefly describes related work

and also indicates which methods we have investigated and also improved in the TaaS

project, section 3 describes the TaaS API’s methods that are relevant to SMT systems, section

4 describes the methods for terminology integration in SMT system training level, section 5

describes the methods for terminology integration in SMT system translation level, and the

1 The LISA QA model: http://web.archive.org/web/20080124014404/http://www.lisa.org/products/qamodel/

http://web.archive.org/web/20080124014404/http:/www.lisa.org/products/qamodel/

Contract no. 296312

D4.4 V 2.0 Page 7 of 49

document is concluded in section 6. Sections 4 and 5 briefly describe the methods

implemented in experimental SMT system interfaces and provide test results for a

terminology integration scenario. Note that manual evaluation of the terminology integration

scenarios is out-of-scope of this deliverable; the manual evaluation of the terminology

integration scenarios for more than one language pair will be provided in the deliverable D4.5

“Evaluation Report”.

Figure 1 The conceptual design of terminology integration in SMT systems

Online Terminology Services

Translation

Training

SMT System Training and adaptation

Online Translation Service

Input Text for Translation

Parallel corpus

Monolingual corpus

Bilingual term collections

Monolingual Term

Extraction

Trained SMT

Model

Bilingual Term

Extraction

Translated Text

Contract no. 296312

D4.4 V 2.0 Page 8 of 49

2. Related Work

This section briefly describes related work on terminology integration in SMT systems.

The publication by Skadiņš et al. (2013), which drafts the TaaS project’s terminology

integration in SMT scenarios, has been used as the basis for the related work section in this

deliverable.

There have been numerous research works reporting improvements of translation

quality in terms of automatic machine translation evaluation metrics after integration of terms

and term phrases in SMT systems (by performing domain adaptation of the SMT systems

with the help of terminology) in both SMT system training and translation levels.

Current SMT phrase-based models, including Moses (Koehn et al., 2007), do not handle

terminology translation. Although domain adaptation can be done using additional in-domain

training data (Koehn and Schroeder, 2007), such an approach is very resource intensive as it

requires gathering of the resources (parallel and monolingual corpora) for each individual

domain and for smaller projects or for languages with limited resources this is not an option.

This makes terminology integration with the standard approaches expensive (in terms of time)

and for less resourced languages in many cases also not feasible (due to lack of parallel or

monolingual in-domain corpora). For example, the term „tablet” is ambiguous – it can refer to

a popular consumer electronics product (a tablet computer), a number of sheets of paper

fastened together along one edge (according to WordNet 3.1), a pill used in medicine, and

others. An SMT system would translate this term in every single case according to its

statistical translation and language models. In other words, a term would be translated using

the most probable phrase alignment, which in most cases may not be in the domain specified

by a user.

Researchers have tried to address this issue with different methods that allow

integrating bilingual term collections in the SMT systems in the training level. For instance,

Bouamor et al. (2012) have observed a gain of +0.3 BLEU points for French-English SMT.

They identified terms in parallel corpora (however, monolingually on source and target sides)

with a similar method to the Tilde’s Wrapper System for CollTerm (TWSC; Pinnis et al.,

2012) using linguistically motivated term phrase identification with morpho-syntactic term

phrase patterns. Then they extracted bilingual term pairs by pairing the source and target

terms and integrated them in SMT systems in the training level using three methods: 1) by

simply adding the extracted term phrase pairs to the parallel corpora, which achieved a

significant translation quality improvement (in the TaaS project we have investigated this

scenario and further details can be found in section 4.1), 2) by extending the SMT system’s

phrase table with new entries (this method did not show significant quality improvements),

and 3) by extending the second method further by adding a feature that indicates, which

entries have been newly added as term phrases. The third method showed a significant

translation quality decrease. Pinnis and Skadiņš (2012) have showed that an additional phrase

table feature that identifies phrase translations containing bilingual terminology can

significantly improve the translation quality. As this method addresses morphologically rich

languages and also analyses the whole translation model instead of making minor changes in

the form of an appendix to the translation model, we have investigated this method also in

TaaS (for further details refer to section 4.2). Using additional phrase tables and explicit user-

specified translations of known phrases is a general practice in SMT for different purposes

Contract no. 296312

D4.4 V 2.0 Page 9 of 49

(e.g., Chen and Eisele (2010) use it to create hybrid SMT systems). However, it is not

explicitly used for integrating terminology in SMT systems.

Although not directly related to terminology translation, Nikoulina et al. (2012) have

proposed a framework for integrating Named Entities (NE) within SMT systems by pre-

processing parallel corpora and replacing NEs with placeholders. The SMT system is trained

using the pre-processed parallel and monolingual data. When performing translation, the

translatable content is always pre-processed with the same techniques that replace NEs with

placeholders. After the translation with the SMT system, the placeholders are replaced back

with translations of the NEs using NE-specific translation methods. It was shown that the

introduced model could lead to +2-3 BLEU point improvement over a baseline system for two

different test sets. However, they report results for the translation between languages with

little morphological inflection (i.e., from English to French) and the NEs are translated with 1

to 1 translation equivalents (i.e., using just the canonical forms), which for translation into

morphologically rich languages is not enough to achieve improvement, e.g., as shown for

terminology by Pinnis and Skadiņš (2012). Because NEs are out-of-scope for the TaaS project

and NEs in contexts behave orthogonally to terms (i.e., NEs of the same category often have

common contexts, however terms don’t have a concept for categories and each term can have

different contexts or even can present in contexts that are common for NEs), we did not

pursue to improve this method in the TaaS project.

Another common terminology translation issue is the absence of terms in phrase-based

SMT translation models. The lack of language (terminology) resources causes the “so-called”

missing terminology to be ignored and not translated (i.e., the output is the same as the input).

This issue can be solved if SMT systems provide a runtime integration with existing

terminology databases or terminology collections provided by users. Such research has

already been proposed, for instance, the popular Moses SMT platform allows the pre-

processing of the translatable content during translation by providing possible translation

equivalents for phrases. Carl and Langlais (2002) in their research showed that using

terminology dictionaries in such a way could increase the translation performance for the

English-French language pair. Babych and Hartley (2003) showed that for NE (namely,

organisation names) special “do-not-translate” lists allowed increasing translation quality for

the English-Russian language pair using a similar pre-processing technique that restricts

translation of identified phrases. However, such approaches have been investigated either for

languages with simple morphology or categories of phrases that are rarely translated or even

left untranslated (e.g., many company and organisation names). A recent study in the FP7

project TTC (2013) has shown that for English-Latvian the pre-processing does not yield

positive results for term translation. Hálek et al. (2011) also showed that the translation

performance with on-line pre-processing drops according to BLEU for English-Czech named

entity translation. This proves that the method is not stable when translating into

morphologically rich languages, or the languages with the high level of inflection (e.g., the

Baltic and Slavic languages). For such languages, the task of terminology translation would

also require a morphological synthesiser to be integrated into an SMT system in order to

synthesise the correct inflected word form (or word forms for multiword terms) in case a

morphologically rich language is used as the target language. In the TaaS project, we have

tried to address these issues and our improvements to the pre-processing methods in the form

of experimental interfaces are documented in section 5.

Terminology integration in SMT systems can be also achieved through domain

adaptation with the help of in-domain parallel and monolingual corpora. However, if we focus

Contract no. 296312

D4.4 V 2.0 Page 10 of 49

on building a domain specific SMT engine, pooling together all available data (especially a

significant portion of data that is out of the desired domain) can also lead to negative changes

in quality, since the out-of-domain training data will overwhelm the in-domain data (Koehn

and Schroeder, 2007). Unfortunately, this drawback of domain specific SMT, when only in-

domain data is used, is its failure to capture generalisations relevant to the target language.

This can lead to poor translation quality (Thurmair, 2004). A domain specific SMT engine

needs to capture the generalisations of an engine trained on a large and sufficient supply of

parallel data, yet not lose the crucial domain orientation. It was shown that to achieve this, an

SMT engine can be trained on all available parallel data including out-of-domain data, and

language model training data must be split into in-domain and out-of-domain sets, generating

separate language models (LM) for each of the sets (Koehn and Schroeder, 2007; Lewis et al.,

2010). As shown by Pinnis and Skadiņš (2012) and Pinnis et al. (2013) in-domain comparable

corpora that is collected from the Web using in-domain terminology (as seed terms for corpus

crawling) can also lead to SMT quality improvements when performing language model

adaptation with an in-domain language model and translation model adaptation with

automatically extracted bilingual terminology. In situations when in-domain data for building

a separate domain specific language model are not available, we have also analysed methods

that could be applied on existing parallel or monolingual data in order to acquire an in-domain

corpus with the help of terminology (see section 4.3 for more details). Following the idea that

it is possible to select in-domain sentences automatically from a big general domain corpus

(Moore and Lewis, 2010), we have also performed multiple experiments by splitting

monolingual corpora into in-domain and out-of-domain corpora (for more details refer to

section 4.3.2).

Although SMT domain adaptation has been an active field in the machine translation

research community, the majority of practical SMT applications rely solely on collecting big

amounts of domain specific corpora. Moreover, there are not so many even more advanced

solutions, which would focus on a special handling of terminology. It is assumed that training

data will contain translations with terminology and SMT will learn accurate terminology from

training data. However, it is not usually the case as training data, even if it is in the same

domain, can contain contradicting terminology – industry or corporate specific synonyms in

product- or vendor-biased terminology. Thus, the remainder of this deliverable will present

methods that can be applied in order to achieve SMT system domain adaptation with the help

of in-domain terminology.

Contract no. 296312

D4.4 V 2.0 Page 11 of 49

3. TaaS API for Machine Translation Systems

The TaaS platform provides an API that machine users can use in order to access the

TaaS terminology services. This API is also intended for machine translation (MT) systems.

Before using the API, the MT system developers have to manually submit a request for a

machine user (a key and a password with which the system can authenticate itself within the

TaaS platform). Then, similarly to computer assisted translation (CAT) tools, each user of an

MT system has to be registered in the TaaS platform as a user and request a user key that the

machine user (the MT system) can then use in order to impersonate as the human user

(provided that the user grants rights to the MT system) and access his/her private term

collections. Thus, for access to the TaaS platform each MT system has to have its own key

and password and it has to acquire from a user the user’s key that is linked to the MT

system’s key within the TaaS platform. Further details on the key generation and assignment

tasks can be found in the TaaS deliverable D3.5 “Integrated TaaS System”.

Once an MT system has gained access to the TaaS platform, the TaaS API provides the

following methods that are relevant to the MT systems:

Get a List of Collections - This method retrieves a list of available collections. When

impersonating as a user, this method will return also the user’s private collections. The

method returns only term collection metadata, thus a second method has to be

executed in order to download the actual term collections. The list can be filtered by

adding a language code (or multiple language codes) and a TaaS domain in the

request.

Get a List of Domains – In order to fully benefit from the filtering of term collections

when executing the Get a List of Collections method, the TaaS platform provides also

a method for acquisition of a list of supported domains. This method can be executed

once by the MT systems in order to pre-cache the list of supported domains.

Get a List of Languages – similarly to domains, an MT system can also pre-cache a

list of supported languages. The TaaS platform supports (within the scope of the TaaS

project) 25 languages, thus it is important to know in which languages terminology is

available and is supported in the TaaS platform.

Get a Collection – This method retrieves all term entries of a requested collection.

The request has to contain the required collection’s ID and the output format type. The

result of this method will contain a document with terms from one term collection in

either Term Base eXchange (TBX), Moses (as supported by the LetsMT platform), or

Tab-Separated Values (TSV) formats. The MT systems have to further be able to

parse the output document and extract monolingual terms or bilingual term pairs for

terminology integration into MT systems according to the MT system needs and

functionality.

Term Extraction – This method tags terms in plaintext. This method provides

functionality for TWSC-based Term Identification (see section 5.1.1) and the Fast

Term Identification (see section 5.1.2) that is required for terminology integration in

SMT system translation level. The term extraction method provides functionality for

the first step in the translatable content pre-processing workflow (see section 5).

Contract no. 296312

D4.4 V 2.0 Page 12 of 49

In the further sections, we assume that we have already acquired a bilingual term

collection from the TaaS platform by using the TaaS API and we have extracted a bilingual

term pair list (a bilingual term collection) from the acquired export file.

Contract no. 296312

D4.4 V 2.0 Page 13 of 49

4. Training Level Integration of Terminology

As already noted in the introduction, the terminology integration in SMT systems can

be carried out in two levels. This section describes the work done on terminology integration

in the SMT system training level. From the theoretical point of view, SMT systems in the

translation process try to solve the following problem:

That is, they try to find the target sentence �̂�, which is the most likely translation of the

source sentence 𝑆. Most commonly, the problem (solving the argmax𝑇

(𝑇|𝑆)) is decomposed

using the Bayes theorem into a noisy-channel model (Koehn, 2010) (argmax𝑇

𝑃(𝑆|𝑇)𝑃(𝑇)),

which allows solving two separate problems: 1) we want to identify target language sentences

(hypotheses), which are possible translation equivalents of the source sentence, and 2) we

want to make sure that the translation hypothesis that we generate do really belong to the

target language. The first problem we solve with a translation model that is trained on a large

parallel sentence corpus and the second problem we solve with a language model that is

trained using a large monolingual corpus (often even much larger than the parallel corpus).

The Figure 2 visually depicts an overall training process of a typical SMT system in the

LetsMT platform, broken down in sub-processes. The figure also shows the processes that

combined in workflows train the translation and language models. The terminology

integration methods discussed further in this deliverable will be integrated with means of

experimental interfaces that have to be integrated between those sub-processes. More details

will be given in the respective method’s description.

Because of this subdivision in two separate models, we can integrate terminology (i.e.,

adapt the models in order for them to prefer the required in-domain term translations to out-

of-domain translations) in the two models separately.

Further, the section 4.1 describes a simple method for terminology integration in SMT

systems (for both the translation models and the language models) that is a prerequisite for

more complex terminology integration methods in SMT. The simple method is followed by

separately describing methods applied for terminology integration in SMT system translation

models (section 4.2) and SMT system language models (section 4.3).

Translation

model

Language

model

Constant

�̂� = argmax𝑇

𝑃(𝑇|𝑆) = argmax𝑇

𝑃(𝑆|𝑇)𝑃(𝑇)

𝑃(𝑆)=argmax

𝑇𝑃(𝑆|𝑇)𝑃(𝑇)

Contract no. 296312

D4.4 V 2.0 Page 14 of 49

Figure 2 A typical SMT system training process in the LetsMT platform

4.1. Terminology as a Corpus

The simplest method for terminology integration in SMT systems is to add the in-

domain bilingual term collections to the parallel corpus (or corpora), which is used for

translation model training, and the target language terms to the monolingual corpus (or

corpora), which is used for language model training. This method, although being very

simple, is quite efficient, because it ensures that the terms that are not covered by both the

parallel corpus and the monolingual corpus (i.e., terms that can be considered as out-of-

vocabulary terms) will have at least one translation hypothesis. The conceptual design of this

method is depicted in Figure 3.

A requirement for this method to work is that terminology is added to both the parallel

corpora and the monolingual corpora. Such a requirement is set, because when translating a

sentence, the translation model is responsible for generating translation hypotheses and the

language model is responsible for estimating how well the generated hypotheses represent (or

are likely to belong to) the target language. Therefore, if a term is not present in the parallel

corpus, it cannot be present in the translation hypotheses and if the term is not present in the

monolingual target language’s corpus, the hypotheses containing the term will receive a low

score from the language model.

Translation model

training

Language model

training

Contract no. 296312

D4.4 V 2.0 Page 15 of 49

Figure 3 The conceptual design of the “Terminology as a Corpus” method

However, this method has a limitation. Because terms in the term collections that are

acquired from term banks, e.g., EuroTermBank2, the InterActive Terminology for Europe

(IATE)3, the TaaS platform, etc., are usually stored in their canonical forms (or base forms),

for languages that feature rich morphologies where words can be morphologically inflected,

this method won’t allow the identification of translation equivalents for terms that in contexts

appear in surface forms different from their canonical forms. Nevertheless, this method can be

efficient in the following three scenarios:

When translating from and to languages with little morphological inflection (e.g., from

or to English, German, French, etc.), terms in contexts are often equal to their

canonical forms. Consequently, the recall and the effectiveness of the method is higher

than for morphologically richer languages. E.g., for Latvian, which is a

morphologically rich language, even when translating from English, as shown by

Pinnis and Skadiņš (2012), the method did not show quality improvements when using

a term collection from an authoritative source (the EuroTermBank) because of two

main reasons: 1) in Latvian terms appear in many different surface forms that often are

not equal to the canonical forms, 2) many of the terms in the authoritative data base

were ambiguous, thus the addition of new term pairs causes more statistical

uncertainty for the SMT system. However, it also did not show a quality decrease,

which for the method in general is a positive result.

When acquiring term collections in an automatic process from, e.g., parallel data or

comparable data, the bilingual terms are already stored in surface forms that are

common in different contexts. These bilingual term pairs are better suited as possible

translation hypotheses in different contexts than the canonical forms (for which the

usage in different contexts may be very limited). For bilingual terminology acquisition

from parallel or comparable data, Pinnis (2013) and Aker et al. (2013) have developed

methods that are also integrated in the Bilingual Term Extraction System of the TaaS

2 The EuroTermBank term data base can be accessed at: http://www.eurotermbank.com/. 3 The IATE term data base can be accessed at: http://iate.europa.eu.

http://www.eurotermbank.com/

http://iate.europa.eu/

Contract no. 296312

D4.4 V 2.0 Page 16 of 49

platform (for more details see the deliverable D2.4 Final Bilingual Term Extraction

System). For more details on this scenario see section 4.1.1.

Even if the bilingual terminology is provided by the term databases in a canonical

form, it can still be beneficial in the SMT system training process. More specifically,

by adding the bilingual terminology to the parallel corpus, we indirectly provide the

word alignment processes (e.g., the Giza++ tool in Moses) and further also the phrase

extraction process in the SMT training system with a list of valid term alignments

(single word and multi-word alignments), which can help the word alignment and

phrasal processes to produce word and phrase alignments with a higher precision.

4.1.1. Test Scenarios and Results

We test the terminology integration methods on the English-Latvian language pair and

in the automotive domain (also in the further sections of this deliverable). We use the publicly

available DGT-TM parallel corpus (Steinberger et al., 2012) as the general language corpus.

We used the DGT-TM releases of 2007, 2011 and 2012 in our test scenarios. The total

amount of parallel sentences was 3’159,459 before noise filters (duplicate filters, corrupt

sentence filters, etc.) of LetsMT and 1’954,740 sentence pairs after the filters. The target

language side of the parallel corpus was used for language modelling. After noise filtering,

the monolingual corpora consisted of 1’887,304 sentences. In order to acquire in-domain

terminology we use a small in-domain parallel corpus of 2,617 sentence pairs, which were

randomly split into a tuning set (1745 sentence pairs) and an evaluation set (872 sentence

pairs). The tuning set and the evaluation set are static throughout the whole deliverable. We

extracted bilingual terminology automatically from the tuning set and from comparable

corpora that was collected from the Web (Pinnis and Skadiņš (2012) give more details on

these comparable corpora in the paper). The automatic methods produced 979 term pairs.

These term pairs are non-filtered and may contain noise that is created in the automatic

alignment process, therefore we manually filtered the term pairs in order to remove noise and

also term pairs that are too general and too ambiguous (i.e., the lexical forms may have many

different meanings within in-domain texts). However, note that this is a filtering process and

not a term glossary creation process, where we would create a list of bilingual terms in their

canonical forms. After filtering, we had 845 term pairs in the term collection.

In parallel to the automatic bilingual term extraction and manual filtering, we asked a

professional translator to create a professional term collection from the parallel automotive

domain corpus from which the automatically extracted bilingual terms were acquired. The

translator produced a term collection consisting of 644 term pairs.

Using the publicly available corpus and the tuning data, we created a baseline system

within the LetsMT platform. The automatic evaluation results are given in Table 2. Then, we

added the non-filtered term collection to the parallel and monolingual corpora and re-trained

the system. The results show that there is a significant increase over the baseline system in

translation quality (from 12.68 to 15.51 BLEU points). We also trained a system with the

filtered terms. However it is evident that the results are lower than with the non-filtered terms.

This may be explained with the fact that the automatic alignments were acquired from very

precise in-domain data with respect to the evaluation data and even though the aligned pairs

were noisy and ambiguous, they represented the in-domain data very good. Thus, the noisy

data allows achieving a higher result. Finally, we trained a system using the term collection

created by the professional translator. However the results are lower than with the

Contract no. 296312

D4.4 V 2.0 Page 17 of 49

automatically extracted term collection and the manually post-processed (filtered) term

collection. Adding the professionally created term collection to the parallel and monolingual

corpora did not yield a better result than the automatically extracted term collection, because

of two main reasons: 1) the terms were in their base forms, which when translating into

Latvian often are not the required surface forms, and 2) the professional term collection

contains terms, which in different contexts may be ambiguous and provides just one

translation candidate (e.g., “cover” may be a noun “pārsegs” or a verb “nosegt”, “fill” may be

a noun “uzpilde” or a verb “uzpildīt”/“aizpildīt” depending on the context, etc.). The

automatically extracted term collection may provide multiple translation equivalents for each

term also in different surface forms (as found in the corpus from which the bilingual term

collection was extracted).

Table 2 Terminology as a Corpus test results

Scenario BLEU

(C)

BLEU NIST

(C)

NIST METEOR

(C)

METEOR TER

(C)

TER

Baseline 12.00 12.68 4.1361 4.2644 0.1439 0.1849 0.7893 0.7801

Non-filtered 14.60 15.51 4.4756 4.6301 0.1599 0.2011 0.7660 0.7531

Filtered 13.94 14.76 4.4010 4.5376 0.1580 0.1985 0.7719 0.7604

Professional 12.97 13.62 4.3422 4.4792 0.1513 0.1941 0.7697 0.7586

4.2. Translation Model Adaptation

As described in the previous section, the task of the translation model is to generate

translation hypotheses for source language sentences. Therefore, the goal of terminology

integration in an SMT system’s translation model is to either make the translation model

prefer in-domain translation hypotheses for terms over out-of-domain translation hypotheses

in as many in-domain contexts as possible (i.e., generate in-domain translation hypotheses

with higher translation likelihood scores than out-of-domain translation hypotheses) or to

allow only in-domain translation hypotheses of terms. The conceptual design of the

translation model adaptation methods using bilingual term collections is depicted in Figure 4.

Further, we describe two methods that we have implemented as experimental interfaces

for the LetsMT platform in order to perform translation model adaptation.

Contract no. 296312

D4.4 V 2.0 Page 18 of 49

Figure 4 The conceptual design of the “Translation model adaptation” methods

4.2.1. Phrase Table Adaptation

Following earlier research (Pinnis and Skadiņš, 2012), we implemented an experimental

interface for the LetsMT platform (and thus any SMT platform that uses the Moses SMT

toolkit) that transforms the Moses phrase table of the translation model to an in-domain term-

aware phrase table. We do this by adding a 6th feature to the default five features that are used

in Moses phrase tables. Figure 5 shows that the phrase table adaptation is performed

immediately after a phrase table is created in the SMT system’s training process (the

“consolidate-ttable-halves” process in the LetsMT platform).

The 6th feature receives the following values:

“1” if a phrase on both sides (in both languages) does not contain a term pair from a

bilingual term list. If a phrase contains a term only on one side (in one language), but

not on the other, it receives the value “1” as such situations indicate about possible

out-of-domain (wrong) translation candidates.

“2.718” if a phrase on both sides (in both languages) contains a term pair from the

term list.

In order to find out whether a phrase in the phrase table contains a given term or not, phrases

and terms are stemmed prior to comparison. This allows finding inflected forms of term

phrases even if those are not given in the bilingual term list. The 6th feature identifies phrases

containing in-domain term translations and allows assigning higher translation probabilities to

in-domain translation hypotheses. An example excerpt from an English-Latvian Moses phrase

table with a 6th feature is given in Figure 6.

Contract no. 296312

D4.4 V 2.0 Page 19 of 49

Figure 5 Phrase table adaptation as a step in the translation model training workflow

Figure 6 Example excerpt from an English-Latvian Moses phrase table with a 6th feature

When both the translation model and the language model are created, in a typical SMT

system training workflow the system is tuned, e.g., with Minimum Error Rate Training

(MERT; Bertoldi et al., 2009). The task of tuning is to learn weights for the different features

of an SMT model using a representative of the target domain set of parallel sentences – the

tuning data. The phrase table of the translation model after the adaptation contains 6 features

(one of which identifies whether a phrase pair contains bilingual terminology). In order for

the 6th feature to be productive, the tuning data has to contain the same terminology that was

used to adapt the phrase table, otherwise the tuning process will learn that the 6th feature is

“useless” and assign it a negative weight. Therefore, an important aspect for the phrase table

adaptation method is the selection of tuning data. In the scenarios in section 4.2.3 we have

enriched the bilingual terminology collection also with terms automatically extracted from the

tuning data, thereby ensuring the presence of in-domain terminology in the tuning data.

However, tuning data could be also selected in an automated process from parallel data, e.g.,

by randomly selecting sentence pairs containing 0, 1, 2, etc. bilingual term pairs from the

bilingual term collections in the sentence pairs.

Contract no. 296312

D4.4 V 2.0 Page 20 of 49

4.2.2. Phrase Filtering

When performing phrase table adaptation, we only allow the SMT system to prefer in-

domain translation hypotheses to out-of-domain translation hypotheses. However, sometimes

we could want limiting the term translation hypotheses to only those that are present in a term

glossary and disallow all out-of-domain translations at all. Such a scenario could be beneficial

for the translation of, e.g., named entities (which are not terms, but nonetheless), terms that

have to have a specific translation in domains or use cases that can be considered sensitive

(e.g., in a very sensitive scenario we could disallow racist translations), etc. Thus, a second

method that we have investigated in the TaaS project is phrase filtering.

Figure 7 Phrase filtering as a step in the translation model training workflow

As shown in Figure 7 the phrase filtering step can be performed immediately after

phrase extraction (in the LetsMT platform the phrase extraction process is named “extract-

phrases”). The filtering has to be performed before phrase scoring in order to ensure correct

calculation of translation probability scores.

The term filtering method is both effective (it filters out all wrong translation

hypotheses) and very risky. That is, if a term collection contains ambiguous terms, that is,

phrases that may have multiple meanings and multiple translations also in in-domain texts and

not all translations will be defined in the term collection, then the phrase pairs that contain

translation equivalents, regardless of the fact that they are correct translation hypotheses, will

be filtered out. Let us go through a short example. Imagine that we have an English-Latvian

term collection containing terms from the automotive domain (see Table 3) and we have an

example excerpt from phrases extracted by the LetsMT system’s training process and filtered

Contract no. 296312

D4.4 V 2.0 Page 21 of 49

out by the term filtering process (see Table 4), i.e., the filtering process has decided that the

term pairs are wrong.

For each phrase pair the Table 4 also tells whether the phrase pair is correct, i.e.,

whether such a translation exists regardless the automotive domain constraints and it also

shows whether the phrase pair, if correct, can be considered an in-domain phrase pair

regardless of the term collection. That is, the last column should tell us whether there might

exist term pairs that we have forgot to include in the term collection. The “*” in the last

column indicates that the domain affiliation is ambiguous (meaning, it could and could not

belong to the domain).

Table 3 An example English-Latvian term collection in the automotive domain

English term Latvian term

force spēks

production ražošana

version versija

service apkope

service serviss

rail dzelzceļš

rail sliedes

production ražošana

product produkts

instrument instruments

transmission transmisija

Table 4 An example of English-Latvian phrase pairs that were filtered out by the phrase filtering process

No. English phrase Latvian phrase Correct In-domain?

1 force majeure majeure No -

2 for the production par tās izpildi No -

3 the production tās No -

4 the Dutch version holandiešu tekstā Yes No

5 entry into service nodots ekspluatācijā Yes Yes

6 service pakalpojums Yes Yes *

7 gateway in the rails ieeju kuģu margās Yes No

8 plant protection products augu aizsardzības līdzekļu Yes No

9 products izstrādājumu Yes No *

10 control instruments kontroles ierīces Yes No

11 transmission pārnesumkārbas No -

The Table 4 shows that the filtering step is able to filter out pairs that are incorrect in

terms of phrase boundaries (the examples 1, 2, 3, and 4 in the table). The method also

correctly filters out correct, but out-of-domain phrase pairs. However, it can be seen that the

method is not forgiving if the term collection lacks an important translation equivalent. All

phrases containing the missing translation equivalent are filtered out. Nevertheless, in section

Contract no. 296312

D4.4 V 2.0 Page 22 of 49

4.2.3 we show that the method can be beneficial if applied wisely and bearing in mind the

behaviour of the method; we also show that performing phrase filtering with this method

using automatically extracted term pairs from parallel or comparable corpora without manual

revision is not recommended.


For the testing of translation model adaptation with bilingual terminology, we use

similar data combination as for the testing of the Terminology as a Corpus scenarios (see

section 4.1.1). The difference, however, is that we build two baseline systems and we already

integrate the bilingual terminology in the parallel and monolingual corpora (thus also the

BLEU scores are higher for the baseline systems). Furthermore, we also build the systems

with a second language model – an in-domain language model. The data for the in-domain

language model has been collected from the Web using the Focussed Monolingual Crawler

(FMC). The in-domain monolingual corpus consists of 1’664,403 sentences before the

LetsMT noise filter and 224,639 sentences after noise filtering. As the noise filter removes

also duplicate sentences, there is a large size reduction of the in-domain monolingual corpus.

For both baseline scenarios (with non-filtered and with filtered terms) we separately

adapted the translation models in order to test the translation quality changes. We first tested

the phrase table filtering method in two separate scenarios – source-to-target filtering and

target-to-source filtering. Because in the filtering scenario we remove invalid phrase pairs

with respect to the first language (irrelevant of the translation direction), we get different

filtering results if we consider the source language the first or the target language. The test

results in Table 5 show that for English-Latvian the source-to-target filtering achieves a

higher result (in terms of translation quality). However, it can change between different

language pairs. It is also evident that using filtered terms where we have manually removed

ambiguous terms and too general terms, the translation quality even exceeds the baseline

system’s translation quality. Whereas the non-filtered terms cause valid phrase pairs to be

filtered out from the phrase table. Therefore, the translation quality slightly decreases in

comparison to the baseline. It should be noted that the phrase filtering is a challenging method

that can have beneficial effects, however the term collection has to be very complete (with

non-ambiguous terms or all terms that are ambiguous have to have all the possible translation

equivalents specified in the term collection) in order to achieve a translation quality

improvement. However, as shown by the results when using the filtered terms, translation

quality improvements can be achieved.

Table 5 Test results of terminology integration in SMT system training level – translation model adaptation

Scenario BLEU

(C)

BLEU NIST

(C)

NIST METEOR

(C)

METEOR TER

(C)

TER

Non-filtered terms

Baseline 14.96 15.72 4.5095 4.6825 0.1588 0.2026 0.7660 0.7532

Source-to-target

filtering

14.95 15.68 4.5329 4.6976 0.1609 0.2041 0.7626 0.7507

Target-to-source

filtering

14.34 15.06 4.4613 4.6249 0.1565 0.2005 0.7745 0.7632

6th feature 15.21 15.96 4.5884 4.7566 0.1623 0.2058 0.7636 0.7514

Contract no. 296312

D4.4 V 2.0 Page 23 of 49

Filtered terms

Baseline 13.12 13.87 3.9872 4.1404 0.1385 0.1811 0.7987 0.7874

Source-to-target

filtering 13.42 14.21 4.0753 4.2273 0.1417 0.1839 0.7877 0.7754

Target-to-source

filtering

12.31 12.95 3.8403 3.9850 0.1314 0.1730 0.8070 0.7963

6th feature 13.39 14.1 4.1029 4.2458 0.1434 0.1852 0.7857 0.7737

Finally, the phrase table adaptation with the help of an additional feature in the phrase

table that identifies bilingual terminology in phrase pairs also achieves a translation quality

improvement over the baseline systems for both filtered and non-filtered term collection

scenarios. In addition, it should be noted that this method has not shown a translation quality

decrease in our experiments.

4.3. Language Model Adaptation

The second area of focus after terminology integration in translation models has been

the usage of bilingual terminology in order to perform language model adaptation. The

conceptual design of bilingual terminology integration in SMT system language models is

depicted in Figure 8. The following methods have been investigated in the TaaS project:

Usage of a term collection in order to acquire in-domain monolingual data for

Language model training from the Web (see section 4.3.1).

Usage of a term collection in order to acquire in-domain and out-of-domain

monolingual data from the monolingual corpora that is used for the training of an

SMT system (see section 4.3.2). The in-domain and out-of-domain data are further

used in order to train in-domain and out-of-domain language models for the SMT

system.

Usage of a term collection in order to acquire in-domain and out-of-domain

monolingual data from the parallel corpora that is used for the training of an SMT

system (see section 4.3.3). The in-domain and out-of-domain data are further used in

order to train in-domain and out-of-domain language models for the SMT system.

Contract no. 296312

D4.4 V 2.0 Page 24 of 49

Figure 8 The conceptual design of the “Language model adaptation” methods

4.3.1. In-domain Corpus Acquisition from the Web

Earlier research in the ACCURAT project (Skadiņa et al., 2012) and the TTC project

(TTC Project, 2013) has shown that terminology can be successfully used in order to acquire

an in-domain monolingual corpus from the Web using comparable corpora collection

techniques. In the TaaS project (Pinnis and Skadiņš, 2012) we use the target language terms

of bilingual term collections in order to collect a comparable monolingual corpus with the

Focussed Monolingual Crawler (FMC; Mastropavlos and Papavassiliou, 2011) from the Web.

The FMC requires seed terms as well as a list of Web sites where to start crawling (or Web

domains if crawling should be limited to just the specified Web domains). The bilingual term

collections are very good sources for seed terms, therefore, we use them as seed terms. As the

aligned seed terms may contain also out-of-domain or cross-domain term and NE phrases, we

apply a ranking method based on reference corpus statistics, more precisely, we use the

inverse document frequency (IDF; Spärck Jones, 1972) scores of words from general (broad)

domain corpora (for instance, the whole Wikipedia and current news corpora) to weigh the

specificity of a term pair. We rely on term pairs instead of just the target language terms,

because through empirical analysis we have identified that by doing so we acquire more

specific terms that are better suited as seed terms and the crawled corpora is less noisy (that is,

containing less out-of-domain content because of term ambiguity). We rank each bilingual

term using the following equation:

𝑅(𝑝𝑠𝑟𝑐 , 𝑝𝑡𝑟𝑔) = 𝑚𝑖𝑛(∑ 𝐼𝐷𝐹𝑠𝑟𝑐(𝑝𝑠𝑟𝑐(𝑖))

|𝑝𝑠𝑟𝑐|

𝑖=1

, ∑ 𝐼𝐷𝐹𝑡𝑟𝑔 (𝑝𝑡𝑟𝑔(𝑗))

|𝑝𝑡𝑟𝑔|

𝑗=1

)

where 𝑝𝑠𝑟𝑐 and 𝑝𝑡𝑟𝑔 denote phrases in the source and target languages and 𝐼𝐷𝐹𝑠𝑟𝑐 and 𝐼𝐷𝐹𝑡𝑟𝑔

denote the respective language IDF score functions that return an IDF score for a given token.

The ranking method was selected through a heuristic analysis so that specific in-domain term

and named entity phrases would be ranked higher than broad-domain or cross-domain

Contract no. 296312

D4.4 V 2.0 Page 25 of 49

phrases. The Web domains and URLs are specified manually before executing the corpus

collection.

Using this method we have collected a monolingual corpus in the automotive domain

consisting of 224,639 unique (and filtered with LetsMT corpus filtering methods) sentences

(for more on the comparable corpus refer to Pinnis and Skadiņš, 2012).

4.3.2. Monolingual Corpora Splitting

For language model adaptation in the TaaS project, we have investigated methods that

use terminology in order to acquire in-domain monolingual data with which we can train an

in-domain language model (next to a general language model), thereby achieving language

model adaptation. The idea behind monolingual corpora splitting is that if we already have a

large monolingual corpus, we could use this large corpus and extract from it sentences that we

consider in-domain using a term collection. Because the in-domain sentences contain in-

domain terminology, combined in a corpus they should represent the in-domain texts of the

target language better. However, it should be noted that this method is highly experimental.

By performing just monolingual analysis we might also extract sentence pairs that contain the

term lexical forms, however with different meanings (i.e., if the term lexical forms are

ambiguous). Nevertheless, in the TaaS project we are investigating this method in order to

understand how much can be achieved by the monolingual corpora splitting method.

Each monolingual corpus can be split in two parts – an in-domain part and an out-of-

domain part. When we have just one monolingual corpus, we can easily split it in the two

parts (see Figure 9). However, if we have more than one corpus, we have multiple choices:

We can split both corpora in two parts and train four language models – two in-

domain language models and two out-of-domain language models (see Figure 10).

We can split both corpora in two parts and then combine the in-domain and out-of-

domain parts together so that we would end up having again two corpora – an in-

domain and an out-of-domain corpus (see Figure 11).

We can also create three language models by splitting just one of the corpora or by

splitting all of them, but concatenating back just the in-domain or the out-of-domain

part.

Figure 9 The monolingual corpus splitting method for one target language corpus

Contract no. 296312

D4.4 V 2.0 Page 26 of 49

Figure 10 The monolingual corpus splitting method for two target language corpora (a)

Figure 11 The monolingual corpus splitting method for two target language corpora (b)

4.3.3. Parallel Corpora Splitting

When performing monolingual corpora splitting, we identified that many out-of-domain

sentences were considered in-domain because the term lexical forms of the target language

were ambiguous. A possible method to limit the out-of-domain noise could be to apply a

method similar to the method for phrase table adaptation (see section 4.2.1). That is, if we

identify that a parallel sentence pair contains a bilingual term pair (a term both in the source

sentence and the term’s translation from the term collection in the target sentence) there is a

higher possibility that the target sentence is an in-domain sentence. As this method was not

planned at the beginning, we have not integrated it as an experimental interface. However, we

are considering to examine it in future work in order to compare the results between

monolingual and bilingual corpus splitting.


For the testing of language model adaptation with bilingual terminology, we use similar

data combination as for the testing of the Terminology as a Corpus scenarios (see section

4.1.1). The difference, however, is that we use also an in-domain monolingual corpus that has

been collected from the Web using the Focussed Monolingual Crawler (FMC). The in-domain

monolingual corpus consists of 1’664,403 sentences before the LetsMT noise filter and

224,639 sentences after noise filtering. The noise filter removes also duplicates, therefore,

Contract no. 296312

D4.4 V 2.0 Page 27 of 49

there is such a large size reduction of the in-domain monolingual corpus. The baseline system

has been trained using two language models – the general language model that is based on the

DGT-TM corpus and the in-domain language model that is based on the comparable Web

corpus.

Additionally to the baseline system, we have performed three tests with monolingual

corpora splitting. In the first experiment, we have split the general domain corpus and the in-

domain corpus in two parts. The resulting in-domain parts were joined together in a larger in-

domain corpus, however, the out-of-domain parts were kept separated. Thus, we trained three

language models – one with out-of-domain data (from the DGT-TM corpus), one with

pseudo-out-of-domain data (the out-of-domain part of the initial in-domain corpus), and one

with in-domain data from both initial corpora. For the second experiment we kept all four

parts separated (thus having two in-domain and two out-of-domain language models). For the

third experiment we concatenated the in-domain parts and respectively also the out-of-domain

parts in order to train just two language models.

The Table 6 shows the test results. It is evident in our test results that the system with

two language models, which were based on the two reorganised monolingual corpora)

achieved a significantly higher result than all other systems. However, further analysis is

needed in order to verify that the method works also with different corpora and different term

collections.

The Table 6 also shows that we performed phrase table adaptation for all four scenarios.

The results for all scenarios with the adapted phrase table improved over the system without

phrase table adaptation.

Table 6 Test results of terminology integration in SMT system training level – language model adaptation

Scenario BLEU

(C)

BLEU NIST

(C)

NIST METEOR

(C)

METEOR TER

(C)

TER

Baseline 13.41 14.03 4.0188 4.1510 0.1390 0.1795 0.7991 0.7881

Baseline + 6th feature 13.77 14.43 4.0963 4.2284 0.1424 0.1823 0.7838 0.7735

3 mono corpora 13.79 14.45 4.1979 4.3170 0.1497 0.1903 0.7797 0.7691

3 mono corpora + 6th feature 14.03 14.7 4.2493 4.3825 0.1492 0.1913 0.7753 0.7636

4 mono corpora 13.49 14.3 4.0986 4.2610 0.1418 0.1848 0.7913 0.7778

4 mono corpora + 6th feature 13.91 14.69 4.1278 4.2795 0.1470 0.1881 0.7823 0.7695

2 reorganised mono corpora 14.21 15.06 4.2406 4.3855 0.1501 0.1926 0.7771 0.7667

2 reorganised mono corpora + 6th

feature 15.34 16.24 4.4966 4.6588 0.1603 0.2053 0.7596 0.7470

Contract no. 296312

D4.4 V 2.0 Page 28 of 49

5. Translation Level Integration of Terminology

Terminology integration in SMT system training level as shown by the test results, is

effective and is able to tailor SMT systems to a required domain, however, the training level

integration asks to re-train if not the whole SMT system then at least a significant portion of

the system (e.g., the translation model, the language model, or even both and the systems have

to be also re-tuned in order to adjust weights of the different features used in the SMT

system). For many translation tasks (or projects for localisation service providers) re-training

of a system could also be uneconomical (for instance, if all you need is to translate a five page

document). Furthermore, if we have already trained a relatively good SMT system (let it be a

general domain system or a close-domain system to the domain that is needed), why should

we spend time (which often is comparable to training a new system) on re-training it? We

should instead be able to use the same SMT system, but tailor it to the required domain with

the help of just the right bilingual terminology. Therefore, in this section we will document

methods that we have implemented as part of the TaaS project in order to allow terminology

integration in SMT system translation level.

Figure 12 The conceptual design of terminology integration in the SMT system training level

As the Figure 12 depicts, we use an existing SMT system (in our test scenarios a broad

domain SMT system) and ask the user to provide a bilingual term collection only when

performing the translation of input (source language) data. The terminology integration in the

translation level is ensured by a translatable content pre-processing workflow (see Figure 13)

that uses the bilingual term collection in order to identify terms in the translatable content

(e.g., sentence, paragraph, even a full document) using terminology services that are provided

by the TaaS platform (e.g., with the help of TWSC; Pinnis et al., 2012), annotates the content

with possible translation hypotheses from the bilingual term collection (here also TaaS

terminology services can be used) using XML mark-up4 that complies with the Moses SMT

4 More details on the Moses XML markup can be found on the Moses SMT platform’s home page at:

http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc7.

http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc7

Contract no. 296312

D4.4 V 2.0 Page 29 of 49

platform’s XML mark-up format, assigns also translation confidence scores for each of the

translation hypotheses, and, finally, translates the document taking into account the injected

XML mark-up.

Figure 13 Translatable content pre-processing for terminology integration in SMT system translation level

In the further sub-sections we describe the different processes involved in the

translatable content pre-processing workflow including also specific details on how we ensure

that the methods work also for morphologically rich languages. In section 5.4 we also provide

initial test results.

5.1. Identification of Terms in Translatable Documents

The first task that has to be performed when pre-processing translatable content using a

bilingual term collection is to identify terms in the translatable content. The TaaS platform

through its API provides two methods for terminology identification in documents. The first

method is based on linguistically motivated term candidate identification using TWSC (Pinnis

et al., 2012). The second method is based just on identification of terms based on a term

collection. There are benefits and issues that each of the methods causes. Therefore, as a

compromise we have investigated also a method that performs linguistic analysis (similarly as

in TWSC), but it does not perform statistical analysis that is performed by TWSC. Next, we

will describe the three term identification methods and provide analysis on their beneficial

and negative aspects.

5.1.1. TWSC-based Term Identification

The Tilde’s Wrapper System for CollTerm (TWSC) is a console application that allows

tagging of terms in plaintext documents using state-of-the-art linguistically and statistically

motivated term extraction methods (Pinnis et al., 2012). TWSC identifies terms in documents

in the following steps:

Contract no. 296312

D4.4 V 2.0 Page 30 of 49

First, plaintext documents are tagged with part-of-speech or morpho-syntactic taggers,

which may also include lemmatisation. Lemmatisation is highly recommended for

morphologically rich languages in order to ensure that terms in all inflected forms are

identified by the term tagger.

Then the CollTerm5 tool (developed in the ACCURAT project) is used in order to

extract term candidates in length of one to four tokens from the POS-tagged data. The

term candidates are filtered using morpho-syntactic term phrase patterns that have

been pre-defined for all TaaS languages. The term candidates are ranked using

different co-occurrence measures and TF*IDF scores of the words contained in the

term candidates.

Finally, term candidates are marked in the plaintext document by ensuring that terms

do not overlap, terms with higher confidence scores are preferred, etc.

More details on the language support of TWSC, which has been widened in the TaaS

project in order to cover all TaaS languages, can be found in the TaaS deliverable D2.4 “Final

bilingual term extraction system”. Further details on the methodology behind TWSC can be

found in the paper by Pinnis et al. (2012).

The application of TWSC in the translatable content pre-processing has the following

benefits:

The identified term candidates are linguistically motivated (e.g., we can ensure that

verbs are not translated as nouns, stop-words are not treated as terms, etc.).

The termhood of the identified terms is strengthened by the statistical analysis that is

performed in TWSC (i.e., we can distinguish domain specific terms from general

domain phrases).

For languages, for which lemmatisation support is ensured, TWSC identifies terms

regardless of their surface forms. This is very important for short documents where

terms may occur multiple times, but in different surface forms.

However, TWSC also has several issues, which required us to investigate also different

term identification methods that could be better suited for integration in SMT systems:

In the case if TWSC wrongly identifies for some terms the term unithood (i.e.,

specifies wrong term boundaries) and the correct terms are included in the term

collection, these will not be identified. For instance, imagine that we have in our term

collection the term “crash course” and we need to pre-process the following

translatable content: “A crash course in physics”. As an output TWSC might find the

term “crash”, which according to our term collection would not be correct. Because

we did not find “crash course” with TWSC, but only “crash”, the pre-processing

would not process this example correctly.

Because of the statistical analysis performed by CollTerm, TWSC cannot be executed

on very large (e.g., more than 5MB) plaintext documents. When processing

5 More details on CollTerm can be found in the ACCURAT project’s deliverable D2.6 „Toolkit for multi-level

alignment and information extraction from comparable corpora”, which is available online at:

http://www.accurat-project.eu/uploads/Deliverables/ACCURAT%20D2.6%20Toolkit%20for%20multi-

level%20alignment%20and%20information%20extraction%20from%20comparable%20corpora%20v3.0.pdf.

http://www.accurat-project.eu/uploads/Deliverables/ACCURAT%20D2.6%20Toolkit%20for%20multi-level%20alignment%20and%20information%20extraction%20from%20comparable%20corpora%20v3.0.pdf

http://www.accurat-project.eu/uploads/Deliverables/ACCURAT%20D2.6%20Toolkit%20for%20multi-level%20alignment%20and%20information%20extraction%20from%20comparable%20corpora%20v3.0.pdf

Contract no. 296312

D4.4 V 2.0 Page 31 of 49

documents, CollTerm has to read the whole POS-tagged document into memory and

this can cause out-of-memory exceptions. Of course, 5MB documents are not very

common; however, a stability issue theoretically exists.

For longer documents, the statistical analysis that is performed by CollTerm can be

very time consuming (up to several minutes). As in professional translation speed is

very important, the processing time can cause issues.

Because of the limitations of CollTerm, TWSC is able to identify only terms that

consist of up to four tokens. Longer terms that contain, e.g., conjunctions often cannot

be identified because of this limitation.

5.1.2. Fast Term Identification

In order to address the issues we identified when integrating TWSC into the translatable

content pre-processing workflow, we have investigated a different method for terminology

identification in translatable content. The goal of this method is to: 1) achieve fast processing

of both the translatable content (e.g., real time or close to real time for up to 5MB documents)

and also SMT system training data (e.g., we use this method in order to identify bilingual

terminology in Moses phrase tables when performing translation model adaptation, which is

described in section 4.2), and 2) to ensure that terms from the bilingual term collection would

be identified regardless of their specificity. In this method we treat terms as multi-word

phrases and do not enforce any morpho-syntactic restrictions on the identified phrases.

The Fast Term Identification method performs term identification using a left-to-right

search over stemmed tokens of the translatable content (and stemmed terms from the bilingual

term collection). Term identification prioritises longer terms over shorter terms. Stemming is

performed in order to capture morphological variations of terms, i.e., terms in different

surface forms. To understand the process better, let us look at an example. Table 7 contains an

example English-Latvian term collection. Imagine that we have to pre-process the following

content:

“Do I need a computer mouse pad? For a computer a computer mouse is a vital accessory.

Sometimes you need also a mouse pad.”

Table 7 Example English-Latvian term collection

English Latvian

computer dators

computer mouse datora pele

mouse pad peles paliktnis

mouse pele

After term identification using the Fast Term Identification method, we would identify

the following terms:

“Do I need a [computer mouse] pad? For a [computer] a [computer mouse] is a vital

accessory. Sometimes you need also a [mouse pad].”

Contract no. 296312

D4.4 V 2.0 Page 32 of 49

The example shows that the terms “computer mouse” and “mouse pad” were prioritised

over “computer” and “mouse”, because the Fast Term Identification method found a longer

matching term. However, because of the prioritisation, the method sometimes identifies also

incorrect phrases. For example, the first identified phrase “computer mouse” does not

represent a term from the term collection as the syntactically correct term would have been

“mouse pad”. Because the Fast Term Identification method does not perform any linguistic

analysis it can create such mistakes. In a different example “He planted a tree near the power

plant.” a possible mistake would be, for instance, to identify “planted” as a term if a term

collection would contain “plant” as a term. In this example, the verb “to plant” would be

mistakenly identified as the noun “plant”.

However, this method is very fast (in our experiments it allows processing megabytes of

data in just several seconds) and our test results show that in all test scenarios it outperforms

the TWSC-based method (the test results are given in section 5.4).

5.1.3. Valid Phrase-based Term Identification

In order to provide a compromise between speed and linguistic quality, we have

investigated also a third method for term identification in the translatable content. This

method performs term identification in the following steps:

First, we perform part-of-speech or morpho-syntactic tagging of the translatable

content in order to tokenise the content and enrich it with linguistic information

(morpho-syntactic information and lemmas).

Then we use the morpho-syntactic term phrase patterns from TWSC in order to

identify linguistically valid term phrase candidates. At this point, the identified

phrases may overlap. E.g., in the example from the previous section for the sentence:

“Do I need a computer mouse pad?”, we could identify the following valid term

phrases: “computer mouse pad”, “computer mouse”, “mouse pad”, “computer”,

“mouse”, and “pad”.

Finally, we cross-reference the identified phrases with the bilingual term collection

(we use either the lemma sequences with part of speech categories or stemmed phrases

depending on the linguistic support for a language) and annotate the identified terms

in a left-to-right manner similarly to the Fast Term Identification method. In the

example “Do I need a computer mouse pad?” this method would identify “computer

mouse” as a term, and thus this would be equally wrong as for the fast method.

However, this method effectively deals with the example: “He planted a tree near the

power plant”, from the previous section. As “planted” is a verb, it would not be

marked as a term. Therefore, we would identify only “plant” as a term.

As it is evident, this method is a compromise between relying mostly on the state-of-

the-art term extraction methodology or the bilingual term collection as the only reliable

source. This method performs linguistic filtering of term phrases, which is important in order

to filter out noise and it allows identifying most of terms that can be defined in term

collections (the only restriction is that a valid morpho-syntactic term phrase pattern has to be

defined in the term phrase pattern list).

Contract no. 296312

D4.4 V 2.0 Page 33 of 49

5.2. Acquisition of Translation Equivalents for the Identified Terms

The next pre-processing step after term identification is the acquisition of translation

equivalents for the identified terms. Previous research (Nikoulina et al., 2012; Carl and

Langlais, 2002; Babych and Hartley, 2003, and others) on pre-processing of the translatable

content prior to translation has not given special attention to this question, because the

bilingual term collections already “provide” translation equivalents. The issue that we are

addressing in the TaaS project is that the terms that are provided in the bilingual term

collections are usually in their canonical forms and, as numerous times already stressed, for

morphologically rich languages, the canonical forms may often not be the required surface

forms in the given contexts. Previous research has not seen the need to address these issues,

because of the focus on language pairs that do not require (or require very limited)

morphological generation (e.g., English-French, English-German, etc.).

In the TaaS project, we have investigated several approaches that allow acquiring

surface forms for bilingual terms:

The first method (see section 5.2.1) does not perform morphological generation of

surface forms. This method is applicable when the bilingual term collections have

been acquired, for instance, with automatic bilingual term extraction methods from

parallel corpora (e.g., the non-filtered and filtered term collections used in experiments

for training level integration in section 4). In case of using just canonical forms, this

method should show that translation quality for morphologically rich languages does

not increase over a baseline (it may even decrease).

The second method (see section 5.2.2) uses morphological synthesis and language

dependent surface form generation rules in order to acquire surface forms for terms in

canonical forms. Because of language dependent surface form generation rules and the

language dependent morphological generators, this method has been investigated just

for a limited number of languages (including Latvian and English).

The third method (see section 5.2.3) is language independent. It uses a target language

corpus and identifies surface forms using a similar method to the Fast Term

Identification.

The three methods are described further.

5.2.1. No Translation Equivalent Acquisition

In order to evaluate the impact of canonical forms on the translation quality when

performing translatable content pre-processing, one of the methods for translation candidate

acquisition is to rely only on the translation equivalents from the bilingual term collections.

This method should show whether canonical forms for languages with rich morphology can

lead to quality improvements.

Because bilingual term collections can also be acquired using automatic bilingual

terminology alignment methods, e.g., the MPAligner (Pinnis, 2013) or the USFD Term

Aligner (Aker et al., 2013), the terms can also be already defined in the most common surface

forms found in contexts. Therefore, we investigate also the effectiveness of such term

collections when applied in translatable content pre-processing workflows.

Contract no. 296312

D4.4 V 2.0 Page 34 of 49

5.2.2. Morphological Synthesis-based Translation Equivalent Acquisition

The second method (and the first real method) for term translation equivalent

acquisition is based on morphological synthesis of terms. For each target term from the

bilingual term collection, we perform the following steps in order to acquire a list of term

surface forms:

First, we perform morphological analysis for terms, which do not contain morpho-

syntactic information (morpho-syntactic information is usually included in term

collections that have been automatically aligned using the TaaS Bilingual Term

Extraction System). We acquire for each token of a term a list of possible morpho-

syntactic tags and lemmas. For instance, the Table 8 shows the morphological

information acquired for the Latvian term “tīmekļa lapu”6 (‘Web page’ in English),

using the Tilde’s morphological analyser. The term is not in its canonical form, but in

a surface form (the canonical form would be “tīmekļa lapa”).

Table 8 Morphological information acquired for the Latvian term “tīmekļa lapu”

from the Tilde’s morphological analyser of Latvian

Token Part of Speech Lemma Morpho-syntactic tag

tīmekļa N tīmeklis N-msg---------n-----------l-

lapu N lapa N-fpg---------n-----------l-

lapu N lapa N-fsa---------n-----------l-

Then, based on the morphological analysis we identify all morpho-syntactic term

phrase patterns that may correspond to any sequence of the morpho-syntactic tags of

the term’s tokens. We do this in order to identify the morpho-syntactic structure of the

terms for multi-word terms. Single word terms are usually matched to their part-of-

speech containing morpho-syntactic patterns. For the Latvian term “tīmekļa lapa” (we

continue referring to the term using the canonical form) the only matching term phrase

pattern from the TWSC’s morpho-syntactic term phrase pattern list is

“^N...g.* ^N.*”. The pattern defines a two-word term consisting of two nouns.

The first noun is in a genitive case, but the second noun is allowed to be in any

inflected form.

Next, we identify a morpho-syntactic inflection rule. Each morpho-syntactic pattern

has a manually defined morpho-syntactic inflection rule assigned to it. For the Latvian

term “tīmekļa lapu” the inflection rule is as follows:

“**************************0* ***00*********************0*”. The

morpho-syntactic inflection rule specifies that the first token has to be kept as is (the

only change that can be applied is capitalisation if necessary in contexts) and the

second token can be inflected by changing the number and case of the noun.

6 “Tīmekļa lapa” is an information technology and data processing term that can be found in EuroTermBank:

http://www.eurotermbank.com/search.aspx?text=t%C4%ABmek%C4%BCa%20lapa&langfrom=lv&langto=en

&where=etb%20extres&advanced=false#pos=1.

http://www.eurotermbank.com/search.aspx?text=t%C4%ABmek%C4%BCa%20lapa&langfrom=lv&langto=en&where=etb%20extres&advanced=false#pos=1

http://www.eurotermbank.com/search.aspx?text=t%C4%ABmek%C4%BCa%20lapa&langfrom=lv&langto=en&where=etb%20extres&advanced=false#pos=1

Contract no. 296312

D4.4 V 2.0 Page 35 of 49

Further, we generate all possible inflected forms for each token of the term. Because

we know the lemmas and the parts of speech of each of the tokens, we can use a

morphological synthesiser in order to generate all inflected forms of the tokens. Table

9 shows all acquired forms for the tokens of the term “tīmekļa lapa”.

Table 9 Inflected forms of words “tīmeklis” (web) and “lapa” (page)

using Tilde’s Latvian morphological synthesiser

tīmeklis (noun) lapa (noun)

Inflected

form

Morpho-syntactic tag Inflected

form

Morpho-syntactic tag

tīmekli n0msa000000000n00000000000l0 lapa n0fsn000000000n00000000000l0

tīmekli n0msv000000000n00000000000l0 lapa n0fsv000000000n00000000000l0

tīmeklim n0msd000000000n00000000000l0 lapai n0fsd000000000n00000000000l0

tīmeklis n0msn000000000n00000000000l0 lapas n0fpa000000000n00000000000l0

tīmeklī n0msl000000000n00000000000l0 lapas n0fpn000000000n00000000000l0

tīmeklīt n0msv000000000y00000000000l0 lapas n0fpv000000000n00000000000l0

tīmeklīti n0msa000000000y00000000000l0 lapas n0fsg000000000n00000000000l0

tīmeklīti n0msv000000000y00000000000l0 lapiņ n0fsv000000000y00000000000l0

tīmeklītim n0msd000000000y00000000000l0 lapiņa n0fsn000000000y00000000000l0

tīmeklītis n0msn000000000y00000000000l0 lapiņai n0fsd000000000y00000000000l0

tīmeklītī n0msl000000000y00000000000l0 lapiņas n0fpa000000000y00000000000l0

tīmeklīša n0msg000000000y00000000000l0 lapiņas n0fpn000000000y00000000000l0

tīmeklīši n0mpn000000000y00000000000l0 lapiņas n0fpv000000000y00000000000l0

tīmeklīši n0mpv000000000y00000000000l0 lapiņas n0fsg000000000y00000000000l0

tīmeklīšiem n0mpd000000000y00000000000l0 lapiņu n0fpg000000000y00000000000l0

tīmeklīšos n0mpl000000000y00000000000l0 lapiņu n0fsa000000000y00000000000l0

tīmeklīšu n0mpg000000000y00000000000l0 lapiņā n0fsl000000000y00000000000l0

tīmeklīšus n0mpa000000000y00000000000l0 lapiņām n0fpd000000000y00000000000l0

tīmekļa n0msg000000000n00000000000l0 lapiņās n0fpl000000000y00000000000l0

tīmekļi n0mpn000000000n00000000000l0 lapu n0fpg000000000n00000000000l0

tīmekļi n0mpv000000000n00000000000l0 lapu n0fsa000000000n00000000000l0

tīmekļiem n0mpd000000000n00000000000l0 lapā n0fsl000000000n00000000000l0

tīmekļos n0mpl000000000n00000000000l0 lapām n0fpd000000000n00000000000l0

tīmekļu n0mpg000000000n00000000000l0 lapās n0fpl000000000n00000000000l0

tīmekļus n0mpa000000000n00000000000l0

Once we know the morpho-syntactic inflection rule and the inflected forms of the

term’s tokens, we can generate all possible combinations that are valid according to

the morpho-syntactic inflection rule. All valid combinations for the term “tīmekļa

lapa” are given in Table 10. It is evident that just one inflected form of the first token

qualifies while for the second token multiple inflected forms (in which only the

number and case differs) qualify.

Contract no. 296312

D4.4 V 2.0 Page 36 of 49

Table 10 Valid morpho-syntactic combinations for the term “tīmekļa lapa”

tīmeklis (noun) lapa (noun)

Inflected

form

Morpho-syntactic tag Inflected

form

Morpho-syntactic tag

tīmekli n0msa000000000n00000000000l0 lapa n0fsn000000000n00000000000l0

tīmekli n0msv000000000n00000000000l0 lapa n0fsv000000000n00000000000l0

tīmeklim n0msd000000000n00000000000l0 lapai n0fsd000000000n00000000000l0

tīmeklis n0msn000000000n00000000000l0 lapas n0fpa000000000n00000000000l0

tīmeklī n0msl000000000n00000000000l0 lapas n0fpn000000000n00000000000l0

tīmeklīt n0msv000000000y00000000000l0 lapas n0fpv000000000n00000000000l0

tīmeklīti n0msa000000000y00000000000l0 lapas n0fsg000000000n00000000000l0

tīmeklīti n0msv000000000y00000000000l0 lapiņ n0fsv000000000y00000000000l0

tīmeklītim n0msd000000000y00000000000l0 lapiņa n0fsn000000000y00000000000l0

tīmeklītis n0msn000000000y00000000000l0 lapiņai n0fsd000000000y00000000000l0

tīmeklītī n0msl000000000y00000000000l0 lapiņas n0fpa000000000y00000000000l0

tīmeklīša n0msg000000000y00000000000l0 lapiņas n0fpn000000000y00000000000l0

tīmeklīši n0mpn000000000y00000000000l0 lapiņas n0fpv000000000y00000000000l0

tīmeklīši n0mpv000000000y00000000000l0 lapiņas n0fsg000000000y00000000000l0

tīmeklīšiem n0mpd000000000y00000000000l0 lapiņu n0fpg000000000y00000000000l0

tīmeklīšos n0mpl000000000y00000000000l0 lapiņu n0fsa000000000y00000000000l0

tīmeklīšu n0mpg000000000y00000000000l0 lapiņā n0fsl000000000y00000000000l0

tīmeklīšus n0mpa000000000y00000000000l0 lapiņām n0fpd000000000y00000000000l0

tīmekļa n0msg000000000n00000000000l0 lapiņās n0fpl000000000y00000000000l0

tīmekļi n0mpn000000000n00000000000l0 lapu n0fpg000000000n00000000000l0

tīmekļi n0mpv000000000n00000000000l0 lapu n0fsa000000000n00000000000l0

tīmekļiem n0mpd000000000n00000000000l0 lapā n0fsl000000000n00000000000l0

tīmekļos n0mpl000000000n00000000000l0 lapām n0fpd000000000n00000000000l0

tīmekļu n0mpg000000000n00000000000l0 lapās n0fpl000000000n00000000000l0

tīmekļus n0mpa000000000n00000000000l0

Because this method is very language dependent (it requires a morphological analyser,

morphological synthesiser, term phrase patterns, and term morpho-syntactic inflection rules),

we have investigated it only for a limited number of languages, including Latvian and English

(necessary for our test scenarios).

5.2.3. Monolingual Corpus-based Translation Equivalent Acquisition

For the remaining languages, a language independent method for the acquisition of term

translation equivalents (in different surface forms) has been investigated. In the case if the

second method is not applicable (the language is not supported), we can use a large

monolingual corpus from the target language (or even the monolingual corpus that has been

used in order to train the language model of the SMT system) in order to acquire surface

forms for the given target language terms in a bilingual term collection. This method is based

on the Fast Term Identification method described in section 5.1.2, however, in this scenario

Contract no. 296312

D4.4 V 2.0 Page 37 of 49

we just extract all the different surface forms that we can identify in the monolingual corpus.

Of course, not all surface forms for a term will be found. However, we believe that we will be

able to find at least the most commonly used surface forms with this method.

This method in general is much slower than the previous method, because it involves

searching in a large corpus. However, target language term surface forms can also be pre-

generated when creating (or uploading) the bilingual term collections to the SMT

environment. This would then allow using the pre-generated forms in the SMT translation

level without losing speed (which would mean lose in time and further the translation

performance of a translator).

5.3. Ranking the Translation Equivalents

Now that terms in the translatable content have been identified and the translation

hypotheses have been identified, we still need to assign translation likelihood scores to the

translation hypotheses. It is important to apply the ranking of translation hypotheses, because

not all translation hypotheses are well suited in the observed contexts. In addition, some

translation hypotheses are in general more common than others. In the TaaS project, we are

investigating three methods for translation hypotheses ranking:

The first method assigns equal translation likelihood scores to all translation

hypotheses of a term. We use this method as a baseline method for translation

hypotheses ranking. When assigning equal weights to all translation hypotheses we

actually allow the language model to choose the translation hypotheses. However,

relying simply on the language model means that we lose important statistics that

come from a translation model (e.g., source to target language transfer information)

and we lose important information from the source language’s context as that could

help identifying, which translation hypotheses is more likely.

The second method uses a large monolingual corpus in order to rank translation

hypotheses. We assign for each translation hypotheses its relative frequency between

all the translation hypotheses of a source language term. Only exact match phrases are

counted for the translation hypotheses (because we already may have all the different

surface forms from the previous step as different translation hypotheses). This method

allows assigning higher scores for more common translation hypotheses, however it

does not involve the source language context in the decision making process.

Therefore, there is a need for a third method.

While investigating the second method, we identified the need for a method that

allows capturing the linguistic transfer from the source language to the target language

including the context of the source translatable content. One idea to capture this

transfer is to base it in a morpho-syntactic level. That is, allow prediction of the

surface form morpho-syntactic information of the target language’s term based on the

morpho-syntactic information of the source language’s context around the term. We

base the prediction in a morpho-syntactic level instead of the word level in order to

allow generalisation of the model. However, as this is a new research activity and we

did not plan such a linguistic transfer-based translation equivalent ranking before, we

will not report results in this deliverable. Still, we are interested to perform these

experiments in the nearest future.

Contract no. 296312

D4.4 V 2.0 Page 38 of 49

5.4. Test Scenarios and Results

When integrating terminology in the SMT system translation level, we assume that we

have a general domain SMT system already available. Therefore, SMT system training is not

necessary in the following scenarios. We use the baseline system from the Terminology as a

Corpus experiments (see section 4.1.1) as a baseline system also here.

The test scenarios are split in five dimensions depending on the different methods

investigated in the pre-processing workflow’s sub-processes and depending on the data used

for pre-processing of the translatable content:

The term collection used for pre-processing:

o Non-filtered – raw bilingual term collection automatically extracted from

parallel corpora (tuning data of the SMT system).

o Filtered – the raw bilingual term collection manually revised in order to

remove general language phrases and wrong translations.

o Professional – a bilingual term collection manually created by a professional

translator.

The term identification method:

o TWSC – the TWSC-based Term Identification (see section 5.1.1).

o Fast – the Fast Term Identification (see section 5.1.2).

o Phrase – the Valid Phrase-based Term Identification (see section 5.1.3).

The Translation equivalent acquisition method:

o None – only the translation equivalents that are present in the bilingual term

collection are used as translation equivalents, i.e., we do not generate or

acquire any other translation equivalents (see section 5.2.1).

o FST – the Morphological synthesis-based Translation Equivalent Acquisition

(see section 5.2.2). We name it FST, because we use finite state transducer

based morphological analysers and synthesisers in the translation equivalent

generation process.

o Corpus – the Monolingual Corpus-based Translation Equivalent Acquisition

(see section 5.2.3). For the acquisition of translation equivalents, we used the

general language monolingual corpus of the SMT system.

o Combined – the combination of FST and Corpus-based translation equivalent

generation. Because the FST method does not always produce translation

equivalents (e.g., for unknown to the FST-based morphology words), the

combination of the two methods (one that acquires through generation and one

that acquires through search) we can identify more translation equivalents for

the terms.

The corpus from which surface forms have been extracted for the Corpus and

Combined translation equivalent acquisition methods:

o In-domain corpus – the Web crawled in-domain corpus described in section

4.3.1.

o Broad domain corpus – the DGT-TM monolingual target language corpus.

o In-domain and broad domain corpora combined.

The Translation equivalent ranking method (see section 5.3):

o Equal – every translation equivalent of a source term gets an equal translation

likelihood score assigned.

Contract no. 296312

D4.4 V 2.0 Page 39 of 49

o Simple – we assign translation equivalent translation likelihood scores based

on the translation equivalent relative frequencies in a large monolingual corpus

(the general language corpus in our experiments).

The results are given in three tables below:

Table 11 provides results for pre-processing experiments with the non-filtered term

collection.

Table 12 provides results for pre-processing experiments with the filtered term

collection.

Table 13 provides results for pre-processing experiments with the term collection

created by a professional translator.

Each table provides the results of the baseline system and the different pre-processing

experiments from the remaining 4 test scenario dimensions. Test results for all experiments

are given using NIST (Doddington, 2002), BLEU (Papineni et al., 2002), METEOR (Banerjee

and Lavie, 2005), and TER (Snover et al., 2006) evaluation methods. For each pre-processing

scenario, we provide also a score showing the change over the baseline system, i.e.,

translation quality increase or decrease according to the BLEU metric. Further, we will

analyse each of the dimensions separately in order to identify, which method achieves the

highest evaluation results:

The evaluation results suggest that using the non-filtered automatically extracted term

collection average results in terms of BLEU were lower than using the manually

filtered term collection, which allowed achieving an average increase of 0.25% over

all test scenarios, and significantly lower than using the term collection manually

created by a professional translator, which achieved an average increase of 1.6% over

all test scenarios. The best overall results were achieved using the professional

translator’s created term collection with an average increase of 1.35% over all test

scenarios compared to the results achieved with the filtered term collection.

It is also evident that the Fast Term Identification method significantly outperforms

the linguistically motivated term identification methods (“Phrase” and “TWSC”). This

may be explained with the fact that recall of the Fast Term Identification method is

higher than the recall of the Valid Phrase-based Term Identification method and much

higher than that of the TWSC-based Term Identification method (fully linguistically

and statistically motivated method). This is a very positive result as we can achieve

the highest performance (in terms of speed) and still maintain the best quality.

For translation equivalent acquisition we applied four different methods:

o First, we wanted to see how much we could achieve by using just translation

equivalents from the term collections. For the test scenarios based on the non-

filtered term collection, this achieves the best result. This is because the

automatically extracted term collection already contains terms in their potential

surface forms. By generating additional surface forms we create ambiguity and

the results are lower (however, note that an important step is term translation

equivalent ranking and our experiments applied just basic methods for

ranking). For the filtered and professional term collection (especially the

professional), it is evident that the results by not acquiring additional surface

forms do not achieve the best results, however they are still better than many

of the test scenarios. There are multiple reasons for the relatively good

Contract no. 296312

D4.4 V 2.0 Page 40 of 49

performance. For instance, the term collections contain many multi-word

translation pairs (from over 70% in the non-filtered collection to just below

58% in the professional collection), for which not all words of the multi-word

terms are affected by inflection when generating different surface forms (in

many, but not all, cases, just the head word is inflected). Because the baseline

system’s score is relatively low (just 12.68 BLEU points), translation quality

improves by translating correctly only a part of the multi-word terms. Another

reason is related to search space. Having term translations in their canonical

forms means that in contexts where the canonical forms are required we will

have a 100% precision (in case if terms have only one translation equivalent

specified in the term collection). However, if we generate multiple translation

equivalents in the form of different surface forms of a term, we have a higher

possibility of selecting the wrong translation equivalent if the term translation

equivalent ranking method is not able to assign highly confident translation

likelihood scores.

o Then, we applied the morphological synthesis-based term surface form

acquisition method. This method did not show the best overall results, however

it still exceeded the baseline system’s results in all test scenarios. We believe

that when generating all possible surface forms, we need to be able to rank

(taking the source language’s context into account) the translation equivalents

and maybe even drop the least likely translation equivalents as the high

ambiguity makes it difficult for the SMT system to select the correct form.

o As the third method, we investigated the method for monolingual corpus-based

term translation equivalent acquisition. This method has shown to achieve the

best results. For the filtered and professional term collections, it even

outperforms the scenarios without surface form generation (by 0.54 BLEU

point for the professional term collection). This proves that by generating

surface forms we can achieve a higher translation quality.

o The last method was the combination of morphological synthesis and corpus-

based surface form look-up. However, this method achieved the lowest results.

We believe that the low results are caused by the high ambiguity that results

when combining the two different surface form acquisition methods.

The monolingual corpus-based term translation equivalent acquisition method requires

a monolingual corpus in order to provide surface forms of terms. We investigated

three different corpora. The best results were achieved with the DGT-TM monolingual

target language’s corpus. Experiments with the in-domain corpora achieved the lowest

results. Consequently, the combination of both corpora achieved better results than

using just the in-domain corpus, however, lover results than with the broad domain

corpus. We believe that the lower results of the experiments using the in-domain

corpus are caused by spelling mistakes present in the corpus as well as not sufficient

representativeness of different term surface forms in the corpus.

Finally, we applied two term translation equivalent ranking methods. It is evident that

ranking is a crucial component, because higher scores for more frequent surface forms

in almost all experiments allowed achieving a higher result. This shows that for

morphologically rich languages modelling of correct surface forms is very important

in order to achieve as good results as possible.

Contract no. 296312

D4.4 V 2.0 Page 41 of 49

Table 11 Test results of terminology integration in SMT system translation level using the non-filtered term collection

Pre-processing scenario NIST BLEU METEO R TERNIST

(C)

BLEU

(C)

METEO R

(C)

TER

(C)

Change

over

baseline

Baseline 4.2644 12.68 0.1849 0.7801 4.1361 12.00 0.1439 0.7893

Fast + None + Equal 4.6386 15.34 0.2104 0.7507 4.3407 13.21 0.1619 0.7743 21.0%

Phrase + None + Equal 4.5031 14.22 0.2014 0.7599 4.2256 12.34 0.1531 0.7815 12.1%

TWSC + None + Equal 4.4821 14.01 0.1966 0.7614 4.2864 12.72 0.1526 0.7765 10.5%

Fast + FST + Equal 4.4141 13.46 0.1974 0.7686 4.1513 11.99 0.1511 0.7905 6.2%

Fast + FST + Simple 4.4283 13.63 0.1997 0.7674 4.1634 12.18 0.1532 0.7901 7.5%

Phrase + FST + Equal 4.3811 13.45 0.1931 0.7702 4.1319 12.11 0.1469 0.7901 6.1%

Phrase + FST + Simple 4.3964 13.64 0.1946 0.7693 4.1425 12.30 0.1482 0.7898 7.6%

TWSC + FST + Equal 4.3905 13.41 0.1919 0.7683 4.2142 12.48 0.1488 0.7819 5.8%

TWSC + FST + Simple 4.3899 13.44 0.1923 0.7685 4.2130 12.50 0.1493 0.7822 6.0%

Fast + Combined + Equal 4.3121 12.97 0.1908 0.7785 4.0683 11.65 0.1459 0.7981 2.3%

Fast + Combined + Simple 4.4433 13.53 0.1981 0.7682 4.1709 12.01 0.1509 0.7905 6.7%

Fast + Corpus + Equal 4.5067 14.39 0.2002 0.7630 4.2398 12.63 0.1538 0.7846 13.5%

Fast + Corpus + Simple 4.5935 14.72 0.2047 0.7554 4.2998 12.81 0.1568 0.7798 16.1%

Phrase + Combined + Equal 4.2681 12.78 0.1861 0.7810 4.0369 11.55 0.1413 0.7989 0.8%

Phrase + Combined + Simple 4.3733 13.30 0.1924 0.7714 4.1167 11.89 0.1457 0.7918 4.9%

Phrase + Corpus + Equal 4.4246 13.98 0.1933 0.7682 4.1736 12.39 0.1475 0.7877 10.3%

Phrase + Corpus + Simple 4.4971 14.21 0.1975 0.7614 4.2228 12.46 0.1503 0.7834 12.1%

TWSC + Combined + Equal 4.3580 13.22 0.1892 0.7711 4.1908 12.28 0.1470 0.7837 4.3%

TWSC + Combined + Simple 4.4074 13.50 0.1921 0.7678 4.2243 12.45 0.1488 0.7815 6.5%

TWSC + Corpus + Equal 4.4226 13.78 0.1924 0.7659 4.2471 12.59 0.1502 0.7793 8.7%

TWSC + Corpus + Simple 4.4730 14.15 0.1947 0.7622 4.2790 12.84 0.1514 0.7770 11.6%

























Non-filtered terms and no surface form analysis

Non-filtered terms and in-domain and broad domain monolingual corpus for surface form analysis

Non-filtered terms and broad domain monolingual corpus for surface form analysis

Non-filtered terms and in-domain monolingual corpus for surface form analysis

Non-filtered terms and morphological synthesis based surface form analysis

Contract no. 296312

D4.4 V 2.0 Page 42 of 49

Table 12 Test results of terminology integration in SMT system translation level using the filtered term collection


(C)

BLEU

(C)

METEO R

(C)

TER

(C)

Change

over

baseline

Baseline 4.2644 12.68 0.1849 0.7801 4.1361 12.00 0.1439 0.7893

Fast + None + Equal 4.5860 15.16 0.2053 0.7495 4.3307 13.13 0.1590 0.7693 19.6%


TWSC + None + Equal 4.4686 14.25 0.196788 0.7635 4.2677 12.83 0.15249 0.7791 12.4%

Fast + FST + Equal 4.3777 13.49 0.1917 0.7662 4.1856 12.53 0.1486 0.7811 6.4%

Fast + FST + Simple 4.3946 13.69 0.1939 0.7642 4.2000 12.75 0.1506 0.7799 8.0%

Phrase + FST + Equal 4.3376 13.26 0.1896 0.7710 4.1576 12.39 0.1469 0.7846 4.6%


TWSC + FST + Equal 4.3526 13.22 0.1896 0.7713 4.1982 12.49 0.1478 0.7831 4.3%

TWSC + FST + Simple 4.3520 13.25 0.1902 0.7711 4.1945 12.51 0.1482 0.7835 4.5%





Phrase + Combined + Equal 4.2022 12.54 0.1817 0.7825 4.0454 11.76 0.1406 0.7941 -1.1%
























Phrase + Combined + Equal 4.1918 12.55 0.1812 0.7838 4.0381 11.77 0.1404 0.7949 -1.0%








Filtered terms and no surface form analysis

Filtered terms and in-domain and broad domain monolingual corpus for surface form analysis

Filtered terms and broad domain monolingual corpus for surface form analysis

Filtered terms and in-domain monolingual corpus for surface form analysis

Filtered terms and morphological synthesis based surface form analysis

Contract no. 296312

D4.4 V 2.0 Page 43 of 49

Table 13 Test results of terminology integration in SMT system translation level using the professional term collection


(C)

BLEU

(C)

METEO R

(C)

TER

(C)

Change

over

baseline

Baseline 4.2644 12.68 0.1849 0.7801 4.1361 12.00 0.1439 0.7893

Fast + None + Equal 4.6976 15.55 0.21215 0.7415 4.4487 13.55 0.165387 0.7607 22.6%


TWSC + None + Equal 4.4790 14.12 0.197025 0.7594 4.2899 12.67 0.153513 0.7739 11.4%

Fast + FST + Equal 4.5204 13.67 0.1988 0.7535 4.3398 12.77 0.1552 0.7695 7.8%

Fast + FST + Simple 4.5307 13.66 0.2012 0.7522 4.3480 12.70 0.1575 0.7687 7.7%

Phrase + FST + Equal 4.4546 13.46 0.1955 0.7611 4.2808 12.64 0.1519 0.7762 6.2%


TWSC + FST + Equal 4.3934 13.34 0.1923 0.7654 4.2435 12.57 0.1500 0.7774 5.2%

TWSC + FST + Simple 4.3892 13.27 0.1928 0.7651 4.2364 12.44 0.1506 0.7777 4.7%





































Professional terms and no surface form analysis

Professional terms and in-domain and broad domain monolingual corpus for surface form analysis

Professional terms and broad domain monolingual corpus for surface form analysis

Professional terms and in-domain monolingual corpus for surface form analysis

Professional terms and morphological synthesis based surface form analysis

Contract no. 296312

D4.4 V 2.0 Page 44 of 49

In a summary, the results show that the best results have been achieved with the

combination of the Professional Term Collection, Fast Term Identification, the Corpus-based

Translation Equivalent Acquisition from the Broad Domain Corpus, and the Monolingual

Corpus-based Term Translation Equivalent Ranking – 16.09 BLEU points.

With these experiments we have shown that by using an automatically extracted

bilingual term collection, the translation quality can be improved in a translatable content pre-

processing scenario; in our experiments the highest achieved score was 15.34 BLEU points.

However, an automatically extracted term collection requires that in-domain parallel corpora

(at least 2000 sentence pairs) is available for the extraction of the term collection. Obviously

this is requirement cannot be always satisfied. Therefore, a significant result from our test

scenarios is that by using a term collection created by a professional translator we can achieve

even better results; in our experiments, the highest achieved score was 16.09 BLEU points.

Two very important steps in the translatable content pre-processing scenarios are the

translation equivalent acquisition and translation equivalent ranking. Our experiments have

shown that by acquiring surface forms and preferring higher scores for more frequent surface

forms we can achieve higher translation quality. However, in our experiments we relied only

on the target language in order to predict the correct surface forms. By doing so we don’t

perform a linguistic information transfer from the source language to the target language,

which we believe is important to guess the necessary surface forms of terms in the translated

content. However, this is a task for future work in the field of terminology integration in

SMT.

Contract no. 296312

D4.4 V 2.0 Page 45 of 49

6. Conclusions

In this deliverable, we have reported the work done for terminology integration in SMT

systems. More specifically, we have documented the methods that we have implemented for

terminology integration into SMT systems as experimental interfaces that allows to plug in

terminology into Moses based SMT systems. The interfaces have also been tested within

terminology integration scenarios in the LetsMT platform in two levels of the SMT platform –

the SMT system training level and the SMT system translation level. The deliverable reports

also test results from automatic SMT evaluation methods. The preliminary results show

translation quality improvements for the methods that we have investigated.

Contract no. 296312

D4.4 V 2.0 Page 46 of 49

References

Aker, A., Paramita, M., & Gaizauskas, R. (2013). Extracting bilingual terminologies from

comparable corpora. In Proceedings of the 51st Annual Meeting of the Association for

Computational Linguistics. Sofia, Bulgaria. Retrieved from

http://staffwww.dcs.shef.ac.uk/people/a.aker/papers/acl2013.pdf

Babych, B., & Hartley, A. (2003). Improving machine translation quality with automatic

named entity recognition. Proceedings of the 7th International EAMT workshop on MT

and other Language Technology Tools, Improving MT through other Language

Technology Tools: Resources and Tools for Building MT.

Banerjee, S. and Lavie, A. METEOR: An Automatic Metric for MT Evaluation with

Improved Correlation with Human Judgments. In: Proceedings of the Workshop on

Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43rd

Annual Meeting of the Association of Computational Linguistics (ACL 2005), June

2005, Michigan, USA.

Bertoldi, N., Haddow, B., & Fouet, J.-B. (2009). Improved minimum error rate training in

Moses. The Prague Bulletin of Mathematical Linguistics, 91(1), 7–16.

Bouamor, D., Semmar, N., & Zweigenbaum, P. (2012). Identifying bilingual Multi-Word

Expressions for Statistical Machine Translation. Proceedings of the Eighth

International Conference on Language Resources and Evaluation, pp. 674-679.

Carl, M., & Langlais, P. (2002). An intelligent Terminology Database as a pre-processor for

Statistical Machine Translation. COLING-02 on COMPUTERM 2002: second

international workshop on computational terminology, Vol. 14, pp. 1-7.

Chen, Y., & Eisele, A. (2010). Integrating a Rule-based with a Hierarchical Translation

System. Proceedings of the Seventh International Conference on Language Resources

and Evaluation, Valletta, Malta, pp. 1746-1752.

Doddington, G. Automatic evaluation of machine translation quality using n-gram co-

occurrence statistics. In: Proceedings of the second international conference on Human

Language Technology Research (HLT 2002), 2002, p. 138-145, San Diego, USA.

Hálek, O., Rosa, R., Tamchyna, A., & Bojar, O. (2011). Named entities from Wikipedia for

machine translation. Proceedings of the Conference on Theory and Practice of

Information Technologies (ITAT 2011), pp. 23-30.

Koehn, P. (2010). Statistical Machine Translation. Cambridge University Press.

Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., … Herbst, E.

(2007). Moses: open source toolkit for statistical machine translation. In Proceedings of

the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions

(pp. 177–180). Stroudsburg, PA, USA: Association for Computational Linguistics.

Retrieved from http://dl.acm.org/citation.cfm?id=1557769.1557821

Koehn, P., & Schroeder, J. (2007). Experiments in domain adaptation for statistical machine

translation. Proceedings of the Second Workshop on Statistical Machine Translation,

Prague, Czech Republic, pp. 224-227.

Lewis, W. D., Wendt, C., & Bullock, D. (2010). Achieving Domain Specificity in SMT

without Overt Siloing. Proceedings of the Seventh International Conference on

Language Resources and Evaluation, pp. 2878-2883.

Mastropavlos, N., & Papavassiliou, V. (2011). Automatic acquisition of bilingual language

resources. In Proceedings of the 10th International Conference of Greek Linguistics,

Komotini, Greece.

http://dl.acm.org/citation.cfm?id=1557769.1557821

Contract no. 296312

D4.4 V 2.0 Page 47 of 49

Moore, R. C., & Lewis, W. (2010). Intelligent selection of language model training data.

Proceedings of the ACL 2010 Conference Short Papers, Uppsala, Sweden, pp. 220-224.

Nikoulina, V., Sandor, A., & Dymetman, M. (2012). Hybrid Adaptation of Named Entity

Recognition for Statistical Machine Translation. Proceedings of the Second Workshop

on Applying Machine Learning Techniques to Optimise the Division of Labour in

Hybrid MT (ML4HMT-12), Mumbai, India, pp. 1-16.

Papineni, K., Roukos, S., Ward, T., and Zhu, W. J. BLEU: a method for automatic evaluation

of machine translation. In Proceedings of ACL-2002: 40th Annual meeting of the

Association for Computational Linguistics, 2002, p. 311-318.

Pinnis, M. (2013). Context Independent Term Mapper for European Languages. In

Proceedings of Recent Advances in Natural Language Processing (RANLP 2013) (pp.

562–570). Hissar, Bulgaria.

Pinnis, M., Ljubešić, N., Ştefănescu, D., Skadiņa, I., Tadić, M., & Gornostay, T. (2012). Term

Extraction, Tagging, and Mapping Tools for Under-Resourced Languages. In

Proceedings of the 10th Conference on Terminology and Knowledge Engineering (TKE

2012) (pp. 193–208). Madrid.

Pinnis, M., Skadiņa, I., & Vasiļjevs, A. (2013). Domain Adaptation in Statistical Machine

Translation Using Comparable Corpora: Case Study for English Latvian IT

Localisation. In Proceedings of the 14th International Conference on Intelligent Text

Processing and Computational Linguistics (CICLING 2013) (pp. 224–235). Samos,

Greece: Springer Berlin Heidelberg. doi:10.1007/978-3-642-37256-8_19

Pinnis, M., & Skadiņš, R. (2012). MT Adaptation for Under-Resourced Domains – What

Works and What Not. In A. Tavast, K. Muischnek, & M. Koit (Eds.), Human Language

Technologies – The Baltic Perspective - Proceedings of the Fifth International

Conference Baltic HLT 2012 (Vol. 247, pp. 176–184). Tartu, Estonia: IOS Press.

doi:10.3233/978-1-61499-133-5-176.

Skadiņa, I., Aker, A., Mastropavlos, N., Su, F., Tufiș, D., Verlic, M., … Pinnis, M. (2012).

Collecting and Using Comparable Corpora for Statistical Machine Translation. In N. C.

C. Calzolari, K. Choukri, T. Declerck, M. U. Doğan, B. Maegaard, J. Mariani, … S.

Piperidis (Eds.), Proceedings of the Eight International Conference on Language

Resources and Evaluation (LREC’12) (pp. 438–445). Istanbul, Turkey: European

Language Resources Association (ELRA). Retrieved from http://www.lrec-

conf.org/proceedings/lrec2012/pdf/925_Paper.pdf.

Skadiņš, R., Pinnis, M., Gornostay, T., & Vasiļjevs, A. (2013). Application of Online

Terminology Services in Statistical Machine Translation. In Proceedings of the XIV

Machine Translation Summit (pp. 281–286). Nice, France.

Snover, M., Dorr, B., Schwartz, R., Micciulla, L. and Makhoul, J. A Study of Translation Edit

Rate with Targeted Human Annotation. In: Proceedings of Association for Machine

Translation in the Americas, 2006.

Spärck Jones, K. (1972). A statistical interpretation of term specificity and its application in

retrieval. Journal of Documentation, Vol. 28, pp. 11-21.

Steinberger, R., Eisele, A., Klocek, S., Pilos, S., & Schlter, P. (2012). Dgt-tm: A freely

available translation memory in 22 languages. In Proceedings of the Eight International

Conference on Language Resources and Evaluation (LREC’12) (pp. 454–459).

Thurmair, G. (2004). Comparing rule-based and statistical MT output. Proceedings of the

Workshop on the Amazing Utility of Parallel and Comparable Corpora, Lisbon,

Portugal, pp. 5-9.

Contract no. 296312

D4.4 V 2.0 Page 48 of 49

TTC Project. (2013). Public deliverable D7.3: Evaluation of the impact of TTC on Statistical

MT. TTC Project: Terminology Extraction, Translation Tools and Comparable

Corpora. Retrieved from http://ttc-project.eu/images/stories/TTC_D7.3.pdf, p. 38.

Vasiljevs, A., Gornostay, T., & Skadins, R. (2010). LetsMT!--Online Platform for Sharing

Training Data and Building User Tailored Machine Translation. In Proceedings of the

2010 conference on Human Language Technologies--The Baltic Perspective:

Proceedings of the Fourth International Conference Baltic HLT 2010 (pp. 133–140).

Contract no. 296312

D4.4 V 2.0 Page 49 of 49

List of tables

Table 1 Abbreviations ................................................................................................................ 5 Table 2 Terminology as a Corpus test results .......................................................................... 17 Table 3 An example English-Latvian term collection in the automotive domain ................... 21 Table 4 An example of English-Latvian phrase pairs that were filtered out by the phrase

filtering process ........................................................................................................................ 21

Table 5 Test results of terminology integration in SMT system training level – translation

model adaptation ...................................................................................................................... 22 Table 6 Test results of terminology integration in SMT system training level – language

model adaptation ...................................................................................................................... 27 Table 7 Example English-Latvian term collection ................................................................... 31 Table 8 Morphological information acquired for the Latvian term “tīmekļa lapu” from the

Tilde’s morphological analyser of Latvian .............................................................................. 34

Table 9 Inflected forms of words “tīmeklis” (web) and “lapa” (page) using Tilde’s Latvian

morphological synthesiser ........................................................................................................ 35 Table 10 Valid morpho-syntactic combinations for the term “tīmekļa lapa” .......................... 36

Table 11 Test results of terminology integration in SMT system translation level using the

non-filtered term collection ...................................................................................................... 41


filtered term collection ............................................................................................................. 42


professional term collection ..................................................................................................... 43

Date post:	08-Aug-2021
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times