In real-world systems, scaling has been critical for improving the translation quality in autoregressive translation (AT), which however has not been well studied for non-autoregressive translation (NAT). In this work, we bridge the gap by systematically studying the impact of scaling on NAT behaviors. Extensive experiments on six WMT benchmarks over two advanced NAT models show that scaling can alleviate the commonly-cited weaknesses of NAT models, resulting in better translation performance. To reduce the side-effect of scaling on decoding speed, we empirically investigate the impact of NAT encoder and decoder on the translation performance. Experimental results on the large-scale WMT20 En-De show that the asymmetric architecture (e.g. bigger encoder and smaller decoder) can achieve comparable performance with the scaling model, while maintaining the superiority of decoding speed with standard NAT models. To this end, we establish a new benchmark by validating scaled NAT models on the scaled dataset, which can be regarded as a strong baseline for future works. We release code and system outputs at https://github.com/DeepLearnXMU/Scaling4NAT.

翻译：在现实系统中，规模扩展对于提升自回归翻译（AT）的质量至关重要，然而这一因素在非自回归翻译（NAT）中尚未得到充分研究。本文通过系统研究规模扩展对NAT行为的影响，填补了这一空白。针对两种先进NAT模型在六个WMT基准上的大量实验表明，规模扩展可以缓解NAT模型常见的弱点，从而获得更好的翻译性能。为减少规模扩展对解码速度的副作用，我们通过实验探究了NAT编码器和解码器对翻译性能的影响。在大规模WMT20英-德数据集上的实验结果显示，非对称架构（例如更大的编码器和更小的解码器）能够在保持标准NAT模型解码速度优势的同时，取得与规模扩展模型相当的性能。最终，我们在扩展数据集上验证了规模化的NAT模型，建立了一个新的基准，可作为未来工作的有力基线。我们已在https://github.com/DeepLearnXMU/Scaling4NAT 开源代码和系统输出。