Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token generation rates. However, currently available open-source dLLMs often generate at much lower rates, typically decoding only a single token at every denoising timestep in order to maximize output quality. We present Spiffy, a speculative decoding algorithm that accelerates dLLM inference by $\mathbf{2.8{-}3.1\times}$ while provably preserving the model's output distribution. This work addresses the unique challenges involved in applying ideas from speculative decoding of AR-LLMs to the dLLM setting. Spiffy proposes draft states by leveraging the dLLM's distribution itself in an auto-speculative manner. This approach is efficient and effective, and eliminates the overheads of training and running an independent draft model. To structure the candidate draft states, we propose a novel directed draft graph which is uniquely designed to take advantage of the bidirectional, block-wise nature of dLLM generation and can be verified in parallel by the dLLM. To further optimize the structure of these draft graphs, we introduce an efficient, offline calibration algorithm that procedurally determines high-quality graph configurations. These optimized draft graphs, enabling increased acceptance rates, lead to a significant boost in the overall speedup achieved by the system. Crucially, Spiffy is also complementary to other recent innovations in improving dLLM generation speeds such as KV-caching and multi-token unmasking. We demonstrate that when combined with such parallel decoding algorithms, Spiffy is able to effectively multiply the benefits of these methods leading to total speedups of up to $\mathbf{7.9\times}$.

翻译：扩散大语言模型（dLLMs）最近作为一种强大的替代方案出现，相较于自回归大语言模型（AR-LLMs），其具备显著更高的令牌生成速率潜力。然而，当前可用的开源dLLMs通常以低得多的速率生成，通常在每个去噪时间步仅解码单个令牌，以最大化输出质量。我们提出了Spiffy，一种推测解码算法，可将dLLM推理加速$\mathbf{2.8{-}3.1\times}$，同时可证明地保持模型的输出分布。这项工作解决了将AR-LLMs推测解码思想应用于dLLM场景所面临的独特挑战。Spiffy通过以自推测方式利用dLLM自身的分布来提出草稿状态。这种方法高效且有效，并消除了训练和运行独立草稿模型的开销。为了构建候选草稿状态，我们提出了一种新颖的有向草稿图，其独特设计旨在利用dLLM生成的双向、分块特性，并且可以由dLLM并行验证。为了进一步优化这些草稿图的结构，我们引入了一种高效的离线校准算法，该算法通过程序化方式确定高质量的图配置。这些优化的草稿图能够提高接受率，从而显著提升系统实现的整体加速效果。至关重要的是，Spiffy也与近期其他提升dLLM生成速度的创新技术（如KV缓存和多令牌解掩码）互补。我们证明，当与这类并行解码算法结合时，Spiffy能够有效地倍增这些方法的优势，实现高达$\mathbf{7.9\times}$的总加速。