Machine Translation and its Didactics - eTransFair.eu · •Part of technology course/module or...

Machine Translation and its DidacticsVesna Lušicky & Barbara Heinisch, University of Vienna

• Relevance of machine translation

• Post-editing: Competence

• Settings and integration of machine translation into teaching

• Learning outcomes

• Approaches to teaching machine translation and post-editing

• Typical content

• Challenges

• Case study

• Conclusion: Re-focusing at the crossroads

Vesna Lušicky & Barbara Heinisch, University of Vienna

Overview

• Ubiquity of MT (DePalma et al. 2016; Way 2018)◦ Vertical distribution (industries: technical, medical, legal...)◦ Horizontal distribution (language pairs, professional vs. private contexts)

• Increase in demand for MT post-editing (MTPE) services (Koponen 2016)

• Students are using MT (Koletnik-Korošec 2010; Heinisch & Lušicky 2019)


Relevance of machine translation (MT)

• NMT quality evaluation:◦ Good results (Bojar et al. 2016; Burchardt et al. 2017;

Sennrich et al. 2016)

• Post-editing effort: ◦ Lower overall effort, but conservative results (Bentivogli et

al. 2016; Castilho et al. 2017; Popović 2017; Toral et al. 2018)


Neural machine translation (NMT)

• ISO 18587:2017 Translation services -- Post-editing of machine translation output -- Requirements◦ translation competence, ◦ linguistic and textual competence, ◦ competence in research/information mining, ◦ cultural competence, ◦ technical competence, ◦ and domain competence (ISO 18587 2017: 7)

◦ “a general knowledge of Machine Translation technology and a basic understanding of commonerrors that an MT system makes” (ibid)


Post-editing: Competence

• MTPE vs. translation vs. revision (O‘Brien 2002)

• Similarity in error distribution between NMT and humans (Yamada 2019)

• When to include MT and MTPE?◦ Translation training as prerequisite for MTPE (Yamada 2019)◦ Revision experience as prerequisite for MTPE (Guerberof Arenas & Moorkens 2019)◦ MT as language learning tool (Niño 2009; Briggs 2018)


NMT+PE: Competence

• Part of technology course/module or project management/quality management course/module

• Stand-alone course: MT+MTPE or stand-alone course: MT project management

• Part of revision/translation or other course targeting specific language pair

• Industry programmes (e.g. TAUS)

• Adequate organisational and technological framework◦ Generic MT vs. customized MT, SaaS◦ Other tools (e.g. CAT, QA)


Settings and integration of MT into teaching

• O‘Brien 2002:◦ Theoretical component: knowledge of MT, terminology management, pre-editing and

controlled language, and programming skills◦ Practical component: PE practice with at least two MT engines, terminology management and

coding, controlled language, corpus analysis, and programming (macros).


Development of learning outcomes

• MT evaluation using human and automatic metrics, roles of humans in workflows (Doherty et al. 2012)

• “Conceptualise a translation role that encompasses new tasks such as post-editing and working with MT” (Doherty & Moorkens 2013:123)

• Ethics, payment, collaboration (Doherty & Kenny 2014)

• Risk and quality management in LT/MT process (Canfora & Ottmann 2015; Pym 2015)

• Developing metacognitive capacity: reflecting the deployment of language technologies (Massey & Ehrensberger-Dow 2017)

• MT project management (Guerberof Arenas & Moorkens 2019)


Development of learning outcomes

• Transmissionalist approach vs. transformational approach, learner-oriented activity (Kiraly 2000)

• Situated learning (Kiraly 2000, 2005, 2016)

◦ Project-based learning MTPE (Guerberof Arenas & Moorkens 2019)


Approaches to teaching MT and MTPE

• The basic principles of MT technology

• The types of engines on the market

• Integration between CAT tools and MT systems

• MT output and frequently-occurring errors

• MT evaluation: types, scoring, ranking, error categorisation

• MT engine training and implementation in the localisation/translation workflow (Guerberof Arenas & Moorkens 2019)


Typical content: MT

• Basic concepts (MTPE vs. revision, post-editor profile)

• Controlled language, terminology

• Quality (raw MT output, expected quality, quality of post-edited material)

• MTPE strategies and guidelines (light, full MTPE)

• MTPE effort and productivity (technical, temporal, cognitive)

• MTPE and pricing

• MTPE tools

• Practical MTPE excercises in the language pair

• Monolingual MTPE


Typical content: MTPE

• Mixed language combinations

• Limited revision or editing experience

• Limited translation experience

• Lack of interest in technology

• Different quality expectations

• Relationship between humans and technology

• Perceived post-editing effort vs. actual post-editing effort (Moorkens et al. 2015)

• Insufficient organisational and technological framework


Challenges

• EU Council Presidency Translator◦ A machine translation system that was developed

especially for the Presidency of the Council of the European Union

◦ Part of European Commission’s Connecting Europe Facility eTranslation infrastructure

◦ Combines the CEF eTranslation with custom neuralmachine translation (NMT) engines


Case study

• Training data for customizing MT engines◦ Domain adaptation◦ Text type◦ Parallel data◦ Data corruption◦ Data preparation (sentence splitting, alignment etc.)◦ Data quality


Case study

• Quality estimation

• Evaluation◦ Domain adaptation, text types

• MT error analysis◦ Quality metrics

• Integration with CAT tools

• Post-editing◦ Effort◦ Strategies


Case study

• MT as a process◦ MT customization service (cf. Gaspari et al. 2015)

• Quality management◦ Data management

• Risk management (cf. Nitzke et al. 2019)◦ PE risks (under-, over-editing)

• Legal aspects


Case study: Opportunities

• Predictions:◦ “technical, low-risk, low ambiguity translating and interpreting can be safely delivered with

minimum human intervention” (Katan 2016:377)◦ In eight years: “perform translation about as good as a human who is fluent in both languages

but unskilled at translation, for most types of text, and for most popular languages” (Grace et al. 2018:743)

◦ In two decades: “fully automatic useful translation” (Massardo et al. 2016:11)

• Polarisation: low skill vs. high skill professions (Goos et al. 2014)


At the crossroads

• Translators’ self-concept and identity◦ Current industry: Low autonomy profession (Katan 2011, 2016)◦ Intervention in didactics

• Adaptive expertise (Massey & Ehrensberger-Dow 2017)

• Translation as a strategic, co-creative activity


Intervention and expertise

• Content

• Routines, automaticity

• Low degree of autonomy

• Inquiry, innovation

• Creative problem solving

• High degree of autonomy

• Self-empowerment

• Conceptualisation of the translator’s role


Re-focusing

Vesna Lušicky

University of Vienna

[email protected]

Barbara Heinisch

University of Vienna

[email protected]


Thank you!

Bentivogli, L., Bisazza, A., Cettolo, M. and Federico, M., 2016. Neural versus phrase-based machine translation quality: a case study. Proceedings of Conference on Empirical Methods in Natural Language Processing. EMNLP: Texas, 257–267. Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Yepes, A.J., Koehn, P., Logacheva, V., Monz, C. and Negri, M., 2016, August. Findings of the 2016 conference on machine translation. Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers, 131-198.Briggs, N., 2018. Neural Machine Translation Tools in the Language Learning Classroom: Students' Use, Perceptions, and Analyses. JALT CALL Journal, 14(1), 2-24.

Burchardt, A., Macketanz, V., Dehdari, J., Heigold, G., Peter, J.T. and Williams, P., 2017. A linguistic evaluation of rule-based, phrase-based, and neural MT engines. The Prague Bulletin of Mathematical Linguistics, 108(1), 59-170.Canfora, C. and Ottmann, A., 2015. Risikomanagement für Übersetzungen. trans-kom, 8(2), 314-346.

Castilho, S., Moorkens, J., Gaspari, F., Sennrich, R., Sosoni, V., Georgakopoulou, P., Lohar, P., Way, A., Miceli Barone, A.V. and Gialama, M., 2017. A comparative quality evaluation of PBSMT and NMT using professional translators. MT Summit 2017, 116–131, Nagoya, Japan. Doherty, S., Kenny, D. and Way, A., 2012. Taking statistical machine translation to the student translator. AMTA-2012, San Diego, USA.Doherty, S. & Kenny, D., 2014. The design and evaluation of a statistical machine translation syllabus for translation students. The Interpreter and Translator Trainer, 8(2), 295-315.


References

Doherty, S. & Moorkens, J., 2013. Investigating the experience of translation technology labs: pedagogical implications. Journal of Specialised Translation, 19, 122-136.Gaspari, F.; Almaghout, H. & Doherty, S., 2015. A Survey of Machine Translation Competences: Insights for Translation Technology Educators and Practitioners. Perspectives 23 (3): 333–358.

Goos, M., Manning, A. and Salomons, A., 2014. Explaining job polarization: Routine-biased technological change and offshoring. American Economic Review, 104(8), 2509-2526.Grace, K., Salvatier, J., Dafoe, A., Zhang, B. and Evans, O., 2018. When will AI exceed human performance? Evidence from AI experts. Journal of Artificial Intelligence Research, 62, 729-754.Guerberof Arenas, A. & Moorkens, J., 2019. Machine translation and post-editing training as part of a master’s programme. The Journal of SpecialisedTranslation, 31, 217–238.

Heinisch, B. & Lušicky, V., 2019. User expectations towards machine translation: A case study. Proceedings of the MT Summit XVII: the 17th Machine Translation Summit. EAMT. Hutchins, J., 2005. The First Public Demonstration of Machine Translation: The Georgetown-IBM System, 7th January 1954. Retrieved from: http://www.hutchinsweb.me.uk/GUIBM-2005.pdf (23.06.2019).

ISO 18587:2017 Translation services -- Post-editing of machine translation.


References

Katan, D., 2011. Occupation or profession: A survey of the translators’ world. In Sela-Sheffy, R. & Shlesinger M. (eds.), Profession, identity and status: Translators and interpreters as an occupational group. Amsterdam/Philadelphia: Benjamins, 65–88.Katan, D., 2016. Translation at the cross-roads: Time for the transcreational turn? Perspectives, 24(3), 365–381.Kiraly, D., 2000. A Social Constructivist Approach to Translator Education: Empowerment from Theory to Practice. Manchester: St. Jerome Publishing.

———, ed., 2016. Towards Authentic Experiential Learning in Translator Education. Mainz: Mainz University Press.———., 2005. “Project-Based Learning: A Case for Situated Translation.” Meta: Journal des traducteurs 50 (4): 1098. Koponen, M., 2016. Is Machine Translation Post-Editing Worth the Effort? A Survey of Research into Post-Editing and Effort. The Journal of SpecialisedTranslation 25: 131–148.Korošec Koletnik, M., 2011. Applicability and Challenges of Using Machine Translation in Translator Training. ELOPE: English Language Overseas Perspectives and Enquiries 8 (2): 7–18.Massardo, I., van der Meer, J., Khalilov, M., 2016. Translation technology landscape report. September 2016. De Rijp: TAUS.

Massey, G. and Ehrensberger-Dow, M., 2017. Machine learning: Implications for translator education. Lebende Sprachen, 62(2), 300-312.

Moorkens, J., O’Brien, S., da Silva, I.A., de Lima Fonseca, N.B. and Alves, F., 2015. Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, 29(3-4), 267-284.


References

Niño, A., 2009. Machine translation in foreign language learning: language learners’ and tutors’ perceptions of its advantages and disadvantages. ReCALL, 21(2), 241-258.Nitzke, J., Hansen-Schirra, S. and Canfora, C., 2019. Risk management and post-editing competence. Journal of Specialised Translation, 31, 239-259.O’Brien, S., 2002. Teaching post-editing: a proposal for course content. In 6th EAMT Workshop Teaching Machine Translation, 99-106.

Palma, D.; Pielmeier, H.; Stewart, R. G. & Henderson, S. 2016. The language services market: 2016. Common Sense Advisory.Popović, M., 2017. Comparing language related issues for NMT and PBMT between German and English. The Prague Bulletin of Mathematical Linguistics, 108(1), 209-220.

Pym, A., 2015. Translating as risk management. Journal of Pragmatics 85, 67–80.Sennrich, R., Haddow, B. and Birch, A., 2016. Edinburgh neural machine translation systems for wmt 16. arXiv preprint arXiv:1606.02891.Toral, A., Wieling, M. and Way, A., 2018. Post-editing effort of a novel with statistical and neural machine translation. In Frontiers in Digital Humanities, 5, p.9.Way, A., 2018. Quality Expectations of Machine Translation. In Translation Quality Assessment, 159–178. Springer.Yamada, M., 2018. The impact of Google Neural Machine Translation on Post-editing by student translators. The Journal of Specialised Translation, 31, 87–106.


References

• Slide 1: Flowchart of part of the dictionary lookup procedures (Hutchins 2005, based on Sheridan 1955).

• Slide 4: Press clippings◦ Die Presse, 2018. „Künstliche Intelligenz für österreichisches Deutsch.“ Die Presse print edition, 2018-09-01.◦ Kharpal, A., 2018. „China’s Baidu challenges Google with A.I. that translates languages in real-time.“ CNBC online edition, 2018-10-24.◦ D‘Monte, L., 2018. „The Bible helps AI gain in translation.“ LiveMint online edition, 2019-10-24.

• Slides 14–16: Logo (EU Council Presidency Translator 2018).


Figures

Date post:	05-Apr-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Machine Translation and its Didactics - eTransFair.eu · •Part of technology course/module or...

Documents