Back-translation (BT) is an effective semi-supervised learning framework in neural machine translation (NMT). A pre-trained NMT model translates monolingual sentences and makes synthetic bilingual sentence pairs for the training of the other NMT model, and vice versa. Understanding the two NMT models as inference and generation models, respectively, the training method of variational auto-encoder (VAE) was applied in previous works, which is a mainstream framework of generative models. However, the discrete property of translated sentences prevents gradient information from flowing between the two NMT models. In this paper, we propose the categorical reparameterization trick (CRT) that makes NMT models generate differentiable sentences so that the VAE's training framework can work in an end-to-end fashion. Our BT experiment conducted on a WMT benchmark dataset demonstrates the superiority of our proposed CRT compared to the Gumbel-softmax trick, which is a popular reparameterization method for categorical variable. Moreover, our experiments conducted on multiple WMT benchmark datasets demonstrate that our proposed end-to-end training framework is effective in terms of BLEU scores not only compared to its counterpart baseline which is not trained in an end-to-end fashion, but also compared to other previous BT works. The code is available at the web.

翻译：回译（Back-translation，BT）是神经机器翻译（Neural Machine Translation，NMT）中一种有效的半监督学习框架。一个预训练的NMT模型翻译单语句子，生成合成的双语语句对用于训练另一个NMT模型，反之亦然。将这两个NMT模型分别理解为推断模型和生成模型，先前的研究应用了变分自编码器（Variational Auto-Encoder，VAE）的训练方法，这是生成模型的主流框架。然而，翻译句子的离散特性阻碍了梯度信息在两个NMT模型之间流动。本文中，我们提出了分类重参数化技巧（Categorical Reparameterization Trick，CRT），它使NMT模型能够生成可微分的句子，从而使VAE的训练框架能够以端到端的方式工作。我们在WMT基准数据集上进行的回译实验证明了我们提出的CRT相较于Gumbel-softmax技巧（一种流行的分类变量重参数化方法）的优越性。此外，我们在多个WMT基准数据集上进行的实验表明，我们提出的端到端训练框架在BLEU分数方面是有效的，不仅相较于非端到端训练的对应基线模型，也相较于其他先前的回译工作。代码可在网上获取。