Parallel decoding methods such as Jacobi decoding show promise for more efficient LLM inference as it breaks the sequential nature of the LLM decoding process and transforms it into parallelizable computation. However, in practice, it achieves little speedup compared to traditional autoregressive (AR) decoding, primarily because Jacobi decoding seldom accurately predicts more than one token in a single fixed-point iteration step. To address this, we develop a new approach aimed at realizing fast convergence from any state to the fixed point on a Jacobi trajectory. This is accomplished by refining the target LLM to consistently predict the fixed point given any state as input. Extensive experiments demonstrate the effectiveness of our method, showing 2.4$\times$ to 3.4$\times$ improvements in generation speed while preserving generation quality across both domain-specific and open-domain benchmarks.

翻译：并行解码方法（如Jacobi解码）有望提升大语言模型（LLM）的推理效率，因其打破了LLM解码过程的顺序性，将其转化为可并行化计算。然而实际应用中，相较于传统自回归（AR）解码，该方法仅能实现微小的加速提升，主要原因是Jacobi解码在单次不动点迭代步骤中极少能准确预测一个以上的token。为解决此问题，我们提出了一种新方法，旨在实现从任意状态到Jacobi轨迹上不动点的快速收敛。该方法通过优化目标LLM，使其在给定任意状态作为输入时能一致地预测不动点。大量实验证明了我们方法的有效性，在领域特定和开放域基准测试中，生成速度提升了2.4倍至3.4倍，同时保持了生成质量。