Pretrained large character-level language models have been recently revitalized and shown to be competitive with subword models across a range of NLP tasks. However, there has not been any research showing their effectiveness in neural machine translation (NMT). This work performs an extensive comparison across multiple languages and experimental conditions of state-of-the-art character- and subword-level pre-trained models (ByT5 and mT5, respectively) on NMT, and shows that the former not only are effective in translation, but frequently outperform subword models, particularly in cases where training data is limited. The only drawback of character models appears to be their inefficiency (at least 4 times slower to train and for inference). Further analysis indicates that character models are capable of implicitly translating on the word or subword level, thereby nullifying a major potential weakness of operating on the character level.

翻译：近期，预训练的大规模字符级语言模型得以复兴，并在多种自然语言处理任务中展现出与子词模型竞争的能力。然而，尚未有研究证明其在神经机器翻译（NMT）中的有效性。本文对最新字符级和子词级预训练模型（分别为ByT5和mT5）在NMT中的表现进行了跨多种语言和实验条件的全面比较，结果表明：字符级模型不仅有效，而且在翻译中常优于子词模型，尤其在训练数据有限的情况下表现突出。字符模型的唯一缺陷在于其低效性（训练和推理速度至少慢4倍）。进一步分析表明，字符模型能够隐性地在词级或子词级层面进行翻译，从而消除了在字符级操作的一大潜在弱点。