Challenging machine translation engines: Some Spanish-English linguistic problems put to the test
DOI:
https://doi.org/10.5007/2175-7968.2023.e85397Keywords:
Machine Translation, Pre-Editing, Post-Editing, Google Translate, Bing, DeepLAbstract
This work is an evaluation of machine translation engines completed in 2018 and 2021, inspired by Isabelle, Cherry & Foster (2017), and Isabelle & Kuhn (2018). The challenge consisted of testing MTs Google Translate and Bing and DeepL in the translation of certain linguistic problems normally found when translating from Spanish into English. The divergences representing a “challenge” to the engines were of morphological and lexical-syntactical types. The absolute winner of the challenge was DeepL, in second place was Bing from Microsoft, and Google was the engine that was the poorest in the management of the linguistic problems. In terms of time, when comparing the engines three years apart, it was found that DeepL was the only one that enhanced its performance by correcting a problem it had before in a test sentence. This was not the case for the other two, on the contrary, their translations were of lower quality. These machines do not seem to be consistent in the manner in which they are improved. These findings may be valuable for translators who may work with these systems as pre or post-editors so that their efforts may be better directed.
References
Austermühl, Frank. Electronic Tools for Translators. London: Routledge, 2014.
Austermühl, Frank & Kortenbruck, Anke. “A Translator’s Sword of Damocles? An Introduction to Machine Translation”. In: Austermühl, Frank. Electronic Tools for Translators. London: Routledge, 2001/2014. p. 153-176.
Banitz, Brita. “Machine Translation: A Critical Look at the Performance of Rule-Based and Statistical Machine Translation”. Cadernos de Tradução, 40(1), p. 54-71, 2020. DOI: https://doi.org/10.5007/2175-7968.2020v40n1p54
Barreiro, Anabela & Ranchhod, Elisabete. “Machine Translation Challenges for Portuguese”. Lingvisticæ Investigationes, 28(1), p. 3-18, 2005. DOI: https://doi.org/10.1075/li.28.1.03bar
Bing Translator. Available at: https://www.bing.com/translator. Accessed in: Nov. 25, 2018 and Dec. 20, 2021.
Bowker, Lynne & Buitrago Ciro, Jairo. Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. Bingley: Emerald Publishing Limited, 2019. DOI: https://doi:10.1108/978-1-78756-721-420191002
DeepL Traductor. Available at: https://www.deepl.com/es/translator. Accessed in: Nov. 25, 2018 and Dec. 20, 2021.
Doan, Coraline. Comparing Encoder-Decoder Architectures for Neural Machine Translation: A Challenge Set Approach. 2021. 274 f. Thesis (Master in Translation Studies) – University of Ottawa, Faculty of Arts, School of Translation and Interpretation, Ottawa, 2021.
Google Traductor. Available at: https://translate.google.com/. Accessed in: Nov. 25, 2018 and Dec. 20, 2021.
Guerberof-Arenas, Ana & Toral, Antonio. “The Impact of Post-Editing and Machine Translation on Creativity and Reading Experience”. Translation Spaces, 9(2), p. 255-282, 2020. DOI: https://doi.org/10.1075/ts.20035.gue
Isabelle, Pierre; Cherry, Colin & Foster, George. “A Challenge Set Approach to Evaluating Machine Translation”. In: Conference on Empirical Methods in Natural Language Processing, 55., 2017, Copenhagen. Proceedings […]. Copenhagen, Denmark: Association for Computational Linguistics, 2017. p. 2486-2496. DOI: https://doi.org/10.18653/v1/D17-1263
Isabelle, Pierre, & Kuhn, Roland. “A Challenge Set for French → English Machine Translation”. arXiv preprint arXiv:1806.02725, 2018. DOI: https://doi.org/10.48550/arXiv.1806.02725
Koehn, Philipp. Neural Machine Translation. New York: Cambridge University Press, 2020. DOI: https://doi.org/10.1017/9781108608480
Konttinen, Kalle; Salmi, Leena & Koponen, Maarit. “Revision and Post-Editing Competences in Translator Education”. In: Koponen, Maarit; Mossop, Brian; Robert, Isabelle S. & Scocchera, Giovanna (Eds.). Translation Revision and Post-Editing: Industry Practices and Cognitive Processes. London: Routledge, 2021. p. 187-202.
Lagoudaki, Elina. “The Value of Machine Translation for the Professional Translator”. In: Conference of the Association for Machine Translation in the Americas, 8., 2008, Waikiki. Proceedings […]. Waikiki, USA: Association for Machine Translation in the Americas, 2008. p. 262-269. Available at: https://aclanthology.org/2008.amta-srw.4.pdf. Accessed in: Feb. 2, 2023.
Marshman, Elizabeth. Evaluating MT. Ottawa: University of Ottawa, 2018. p. 1-20.
Parra Escartín, Carla & Goulet, Marie-Josée. “When the Post-Editor Is Not a Translator”. In: Koponen, Maarit; Mossop, Brian; Robert, Isabelle S. & Scocchera, Giovanna (Eds.). Translation Revision and Post-Editing: Industry Practices and Cognitive Processes. London: Routledge, 2021. p. 89-106.
Rossi, Caroline & Alice Carré. “How to Choose a Suitable Neural Machine Translation Solution: Evaluation of MT Quality”. In: Kenny, Dorothy (Ed.). Machine Translation for Everyone: Empowering Users in the Age of Artificial Inteligence. Berlin: Language Science Press, 2022. p. 51-79. DOI: https://doi.org/10.5281/zenodo.6653406
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Cadernos de Tradução
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright Notice
Authors hold the copyright and grant the journal the right for their articles' first publication, being their works simultaneously licensed under the Creative Commons Attribution License (CC BY), which allows the sharing of such works with its authorship acknowledged and its initial publication in this journal.
Authors are allowed to enter into separate additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or as a book chapter, with an acknowledgment of its initial publication in this journal).
