Autoregressive decoding limits the efficiency of transformers for Machine Translation (MT). The community proposed specific network architectures and learning-based methods to solve this issue, which are expensive and require changes to the MT model, trading inference speed at the cost of the translation quality. In this paper, we propose to address the problem from the point of view of decoding algorithms, as a less explored but rather compelling direction. We propose to reframe the standard greedy autoregressive decoding of MT with a parallel formulation leveraging Jacobi and Gauss-Seidel fixed-point iteration methods for fast inference. This formulation allows to speed up existing models without training or modifications while retaining translation quality. We present three parallel decoding algorithms and test them on different languages and models showing how the parallelization introduces a speedup up to 38% w.r.t. the standard autoregressive decoding and nearly 2x when scaling the method on parallel resources. Finally, we introduce a decoding dependency graph visualizer (DDGviz) that let us see how the model has learned the conditional dependence between tokens and inspect the decoding procedure.

翻译：自回归解码限制了Transformer在机器翻译（MT）中的效率。学界提出了特定的网络架构和基于学习的方法来解决这一问题，但这些方法成本高昂且需要修改MT模型，以牺牲翻译质量为代价换取推理速度。本文从一个较少探索但颇具前景的方向——解码算法——出发来应对该问题。我们提出将标准贪婪自回归解码的MT过程重构为并行形式，利用雅可比和高斯-赛德尔不动点迭代方法实现快速推理。这种形式无需训练或模型修改即可加速现有模型，同时保持翻译质量。我们提出了三种并行解码算法，在不同语言和模型上进行了测试，结果表明，与标准自回归解码相比，并行化实现了最高38%的加速比，当在并行资源上扩展该方法时，加速比接近2倍。最后，我们引入了解码依赖关系图可视化工具（DDGviz），该工具可揭示模型如何学习词元间的条件依赖关系，并帮助检查解码过程。