Evaluating the Italian-English machine translation quality of MWUs in the domain of archaeology

Abstract

Abstract Multiword units (MWUs) represent a challenging and problematic linguistic issue in the field of Natural Language Processing (NLP) due to their idiosyncratic nature. This paper investigates the quality of Neural Machine Translation (NMT) outputs when dealing with MWUs in the domain of archaeology. As a case study, a dataset of 100 MWUs is used as a Gold Standard to evaluate out-of-context and in-context translation outputs from three state-of-the-art NMT systems for the Italian-English language pair: Google Translate, DeepL, and Microsoft Bing Translator. MT outputs are manually evaluated with reference to the Gold Standard, namely out-of-context and in-context human English translations of the selected 100 MWUs. Results show that terminology is still a problematic category for MT quality and that MWUs translation may vary, and sometimes even improve, when further context is provided.