Pretrained character-level language models were recently shown to be competitive with popular subword models across a range of NLP tasks. However, there has been little research on their effectiveness for neural machine translation (NMT). This work performs an extensive comparison across multiple languages and experimental conditions of state-of-the-art character- and subword-level pre-trained models (ByT5 and mT5, respectively) on NMT, showing the effectiveness of character-level modeling in translation, particularly in cases where training data is limited. In our analysis, we show how character models' performance gains are reflected in better translations of orthographically similar words and rare words. While evaluating the importance of source texts in driving model predictions, we highlight ByT5 word-level patterns suggesting an ability to modulate word and character-level information during the translation, providing insights into a potential weakness of character-level modeling. We conclude by assessing the efficiency tradeoff of character models, suggesting their usage in non-time-critical scenarios to boost translation quality.

翻译：预训练的字符级语言模型近期被证明在多种自然语言处理任务中与流行的子词模型具有竞争力。然而，关于它们在神经机器翻译（NMT）中的有效性研究仍较为有限。本研究针对当前最先进的字符级与子词级预训练模型（分别为ByT5和mT5），在多种语言和实验条件下对NMT性能进行了广泛比较，揭示了字符级建模在翻译中的有效性，尤其在训练数据有限的场景中表现突出。在分析中，我们展示了字符模型的性能提升如何反映于同形近词和罕见词的更准确翻译。通过评估源文本对模型预测的重要性，我们强调了ByT5的词级模式特征，表明其能够在翻译过程中调节词级与字符级信息，这为字符级建模的潜在弱点提供了见解。最后，我们评估了字符模型的效率权衡，建议在非实时场景中使用它们以提升翻译质量。