Autoregressive (AR) and Non-autoregressive (NAR) models are two types of generative models for Neural Machine Translation (NMT). AR models predict tokens in a word-by-word manner and can effectively capture the distribution of real translations. NAR models predict tokens by extracting bidirectional contextual information which can improve the inference speed but they suffer from performance degradation. Previous works utilized AR models to enhance NAR models by reducing the training data's complexity or incorporating the global information into AR models by virtue of NAR models. However, those investigated methods only take advantage of the contextual information of a single type of model while neglecting the diversity in the contextual information that can be provided by different types of models. In this paper, we propose a novel generic collaborative learning method, DCMCL, where AR and NAR models are treated as collaborators instead of teachers and students. To hierarchically leverage the bilateral contextual information, token-level mutual learning and sequence-level contrastive learning are adopted between AR and NAR models. Extensive experiments on four widely used benchmarks show that the proposed DCMCL method can simultaneously improve both AR and NAR models with up to 1.38 and 2.98 BLEU scores respectively, and can also outperform the current best-unified model with up to 0.97 BLEU scores for both AR and NAR decoding.

翻译：自回归（AR）与非自回归（NAR）模型是神经机器翻译（NMT）中两大生成模型类型。AR模型以逐词方式预测词元，能有效捕捉真实翻译的分布；NAR模型则通过提取双向语境信息来预测词元，虽能提升推理速度，但存在性能下降问题。以往研究利用AR模型增强NAR模型，主要通过降低训练数据复杂度实现，或借助NAR模型将全局信息融入AR模型。然而，这些方法仅利用单一类型模型的语境信息，忽视了不同模型类型所能提供的语境信息多样性。本文提出一种新颖的通用协作学习方法DCMCL，将AR与NAR模型视为协作者而非师生关系。为分层利用双侧语境信息，在AR与NAR模型之间引入词元级互学习和序列级对比学习。在四个广泛使用的基准数据集上进行的大量实验表明，所提出的DCMCL方法能同时提升AR与NAR模型性能，分别带来最高1.38和2.98个BLEU值的提升；同时，在AR与NAR解码场景下，该方法相比当前最优统一模型可实现高达0.97个BLEU值的提升。