Non-autoregressive Transformer is a promising text generation model. However, current non-autoregressive models still fall behind their autoregressive counterparts in translation quality. We attribute this accuracy gap to the lack of dependency modeling among decoder inputs. In this paper, we propose CNAT, which learns implicitly categorical codes as latent variables into the non-autoregressive decoding. The interaction among these categorical codes remedies the missing dependencies and improves the model capacity. Experiment results show that our model achieves comparable or better performance in machine translation tasks, compared with several strong baselines.

翻译：非偏向变异器是一种充满希望的文本生成模型。 但是,目前非偏向模型在翻译质量方面仍然落后于其自动递减模型。 我们把这一准确性差距归因于在解码器投入中缺乏依赖性模型。 在本文中,我们建议CNAT将隐含的绝对代码作为潜在变量学习到非偏向解码中。这些绝对代码之间的相互作用纠正了缺失的依赖性并改进了模型能力。实验结果显示,与几个强有力的基线相比,我们的模型在机器翻译任务中取得了类似或更好的业绩。