Non-autoregressive (NAR) language models are known for their low latency in neural machine translation (NMT). However, a performance gap exists between NAR and autoregressive models due to the large decoding space and difficulty in capturing dependency between target words accurately. Compounding this, preparing appropriate training data for NAR models is a non-trivial task, often exacerbating exposure bias. To address these challenges, we apply reinforcement learning (RL) to Levenshtein Transformer, a representative edit-based NAR model, demonstrating that RL with self-generated data can enhance the performance of edit-based NAR models. We explore two RL approaches: stepwise reward maximization and episodic reward maximization. We discuss the respective pros and cons of these two approaches and empirically verify them. Moreover, we experimentally investigate the impact of temperature setting on performance, confirming the importance of proper temperature setting for NAR models' training.

翻译：非自回归（NAR）语言模型以其在神经机器翻译（NMT）中的低延迟而闻名。然而，由于解码空间庞大且难以准确捕获目标词之间的依赖关系，NAR模型与自回归模型之间存在性能差距。此外，为NAR模型准备合适的训练数据是一项不平凡的任务，通常会加剧曝光偏差。为应对这些挑战，我们将强化学习（RL）应用于Levenshtein Transformer这一代表性的基于编辑的NAR模型，证明利用自生成数据的RL可以提升基于编辑的NAR模型的性能。我们探索了两种RL方法：逐步奖励最大化和回合奖励最大化。我们讨论了这两种方法各自的优缺点并进行了实证验证。此外，我们通过实验研究了温度设置对性能的影响，证实了合适的温度设置对NAR模型训练的重要性。