Back-translation is an effective semi-supervised learning framework in neural machine translation (NMT). A pre-trained NMT model translates monolingual sentences and makes synthetic bilingual sentence pairs for the training of the other NMT model, and vice versa. Understanding the two NMT models as inference and generation models, respectively, previous works applied the training framework of variational auto-encoder (VAE). However, the discrete property of translated sentences prevents gradient information from flowing between the two NMT models. In this paper, we propose a categorical reparameterization trick that makes NMT models generate differentiable sentences so that the VAE's training framework can work in the end-to-end fashion. Our experiments demonstrate that our method effectively trains the NMT models and achieves better BLEU scores than the previous baseline on the datasets of the WMT translation task.

翻译：反向翻译是神经机器翻译（NMT）中一种有效的半监督学习框架。预训练的NMT模型翻译单语句子，生成合成双语语句对，用于训练另一个NMT模型，反之亦然。将这两个NMT模型分别视为推理模型和生成模型后，先前的工作采用了变分自编码器（VAE）的训练框架。然而，翻译句子的离散特性阻碍了梯度信息在两个NMT模型之间的流动。本文提出一种分类重参数化技巧，使NMT模型能够生成可微分的句子，从而使VAE的训练框架能够以端到端的方式工作。我们的实验表明，该方法有效训练了NMT模型，并在WMT翻译任务的数据集上取得了比先前基线方法更优的BLEU分数。