The integration of language models for neural machine translation has been extensively studied in the past. It has been shown that an external language model, trained on additional target-side monolingual data, can help improve translation quality. However, there has always been the assumption that the translation model also learns an implicit target-side language model during training, which interferes with the external language model at decoding time. Recently, some works on automatic speech recognition have demonstrated that, if the implicit language model is neutralized in decoding, further improvements can be gained when integrating an external language model. In this work, we transfer this concept to the task of machine translation and compare with the most prominent way of including additional monolingual data - namely back-translation. We find that accounting for the implicit language model significantly boosts the performance of language model fusion, although this approach is still outperformed by back-translation.

翻译：语言模型在神经机器翻译中的集成在过去已被广泛研究。研究表明，在额外的目标端单语数据上训练的外部语言模型有助于提升翻译质量。然而，一直存在一个假设：翻译模型在训练过程中也会隐式学习一个目标端语言模型，这会在解码时与外部语言模型产生干扰。近年来，自动语音识别领域的一些工作表明，如果在解码过程中中和掉隐式语言模型，集成外部语言模型可以带来进一步的改进。本研究将这一概念迁移到机器翻译任务中，并与最著名的引入额外单语数据的方法——即回译——进行对比。我们发现，考虑隐式语言模型显著提升了语言模型融合的性能，尽管这种方法仍然不如回译方法。