Challenging machine translation engines: Some Spanish-English linguistic problems put to the test
DOI:
https://doi.org/10.5007/2175-7968.2023.e85397Palavras-chave:
Machine Translation, Pre-Editing, Post-Editing, Google Translate, Bing, DeepLResumo
This work is an evaluation of machine translation engines completed in 2018 and 2021, inspired by Isabelle, Cherry & Foster (2017), and Isabelle & Kuhn (2018). The challenge consisted of testing MTs Google Translate and Bing and DeepL in the translation of certain linguistic problems normally found when translating from Spanish into English. The divergences representing a “challenge” to the engines were of morphological and lexical-syntactical types. The absolute winner of the challenge was DeepL, in second place was Bing from Microsoft, and Google was the engine that was the poorest in the management of the linguistic problems. In terms of time, when comparing the engines three years apart, it was found that DeepL was the only one that enhanced its performance by correcting a problem it had before in a test sentence. This was not the case for the other two, on the contrary, their translations were of lower quality. These machines do not seem to be consistent in the manner in which they are improved. These findings may be valuable for translators who may work with these systems as pre or post-editors so that their efforts may be better directed.
Referências
Austermühl, Frank. Electronic Tools for Translators. London: Routledge, 2014.
Austermühl, Frank & Kortenbruck, Anke. “A Translator’s Sword of Damocles? An Introduction to Machine Translation”. In: Austermühl, Frank. Electronic Tools for Translators. London: Routledge, 2001/2014. p. 153-176.
Banitz, Brita. “Machine Translation: A Critical Look at the Performance of Rule-Based and Statistical Machine Translation”. Cadernos de Tradução, 40(1), p. 54-71, 2020. DOI: https://doi.org/10.5007/2175-7968.2020v40n1p54
Barreiro, Anabela & Ranchhod, Elisabete. “Machine Translation Challenges for Portuguese”. Lingvisticæ Investigationes, 28(1), p. 3-18, 2005. DOI: https://doi.org/10.1075/li.28.1.03bar
Bing Translator. Available at: https://www.bing.com/translator. Accessed in: Nov. 25, 2018 and Dec. 20, 2021.
Bowker, Lynne & Buitrago Ciro, Jairo. Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. Bingley: Emerald Publishing Limited, 2019. DOI: https://doi:10.1108/978-1-78756-721-420191002
DeepL Traductor. Available at: https://www.deepl.com/es/translator. Accessed in: Nov. 25, 2018 and Dec. 20, 2021.
Doan, Coraline. Comparing Encoder-Decoder Architectures for Neural Machine Translation: A Challenge Set Approach. 2021. 274 f. Thesis (Master in Translation Studies) – University of Ottawa, Faculty of Arts, School of Translation and Interpretation, Ottawa, 2021.
Google Traductor. Available at: https://translate.google.com/. Accessed in: Nov. 25, 2018 and Dec. 20, 2021.
Guerberof-Arenas, Ana & Toral, Antonio. “The Impact of Post-Editing and Machine Translation on Creativity and Reading Experience”. Translation Spaces, 9(2), p. 255-282, 2020. DOI: https://doi.org/10.1075/ts.20035.gue
Isabelle, Pierre; Cherry, Colin & Foster, George. “A Challenge Set Approach to Evaluating Machine Translation”. In: Conference on Empirical Methods in Natural Language Processing, 55., 2017, Copenhagen. Proceedings […]. Copenhagen, Denmark: Association for Computational Linguistics, 2017. p. 2486-2496. DOI: https://doi.org/10.18653/v1/D17-1263
Isabelle, Pierre, & Kuhn, Roland. “A Challenge Set for French → English Machine Translation”. arXiv preprint arXiv:1806.02725, 2018. DOI: https://doi.org/10.48550/arXiv.1806.02725
Koehn, Philipp. Neural Machine Translation. New York: Cambridge University Press, 2020. DOI: https://doi.org/10.1017/9781108608480
Konttinen, Kalle; Salmi, Leena & Koponen, Maarit. “Revision and Post-Editing Competences in Translator Education”. In: Koponen, Maarit; Mossop, Brian; Robert, Isabelle S. & Scocchera, Giovanna (Eds.). Translation Revision and Post-Editing: Industry Practices and Cognitive Processes. London: Routledge, 2021. p. 187-202.
Lagoudaki, Elina. “The Value of Machine Translation for the Professional Translator”. In: Conference of the Association for Machine Translation in the Americas, 8., 2008, Waikiki. Proceedings […]. Waikiki, USA: Association for Machine Translation in the Americas, 2008. p. 262-269. Available at: https://aclanthology.org/2008.amta-srw.4.pdf. Accessed in: Feb. 2, 2023.
Marshman, Elizabeth. Evaluating MT. Ottawa: University of Ottawa, 2018. p. 1-20.
Parra Escartín, Carla & Goulet, Marie-Josée. “When the Post-Editor Is Not a Translator”. In: Koponen, Maarit; Mossop, Brian; Robert, Isabelle S. & Scocchera, Giovanna (Eds.). Translation Revision and Post-Editing: Industry Practices and Cognitive Processes. London: Routledge, 2021. p. 89-106.
Rossi, Caroline & Alice Carré. “How to Choose a Suitable Neural Machine Translation Solution: Evaluation of MT Quality”. In: Kenny, Dorothy (Ed.). Machine Translation for Everyone: Empowering Users in the Age of Artificial Inteligence. Berlin: Language Science Press, 2022. p. 51-79. DOI: https://doi.org/10.5281/zenodo.6653406
Downloads
Publicado
Como Citar
Edição
Seção
Licença
Copyright (c) 2023 Cadernos de Tradução
Este trabalho está licenciado sob uma licença Creative Commons Attribution 4.0 International License.
Autores mantêm os direitos autorais e concedem à revista o direito de primeira publicação, com o trabalho simultaneamente licenciado sob a Licença Creative Commons Atribuição 4.0 Internacional (CC BY) que permite o compartilhamento do trabalho com reconhecimento da autoria e publicação inicial nesta revista.
Autores têm autorização para assumir contratos adicionais separadamente, para distribuição não exclusiva da versão do trabalho publicada nesta revista (ex.: publicar em repositório institucional ou como capítulo de livro, com reconhecimento de autoria e publicação inicial nesta revista).
