Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel. However, state-of-the-art block-wise dLLMs rely on a "remasking" mechanism that decodes only the most confident tokens and discards the rest, effectively wasting computation. We demonstrate that recycling computation from the discarded tokens is beneficial, as these tokens retain contextual information useful for subsequent decoding iterations. In light of this, we propose Residual Context Diffusion (RCD), a module that converts these discarded token representations into contextual residuals and injects them back for the next denoising step. RCD uses a decoupled two-stage training pipeline to bypass the memory bottlenecks associated with backpropagation. We validate our method on both long CoT reasoning (SDAR) and short CoT instruction following (LLaDA) models. We demonstrate that a standard dLLM can be efficiently converted to the RCD paradigm with merely ~1 billion tokens. RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead across a wide range of benchmarks. Notably, on the most challenging AIME tasks, RCD nearly doubles baseline accuracy and attains up to 4-5x fewer denoising steps at equivalent accuracy levels.

翻译：扩散大语言模型（dLLMs）作为一种有前景的替代方案，因其能够并行解码多个令牌而受到关注。然而，当前最先进的分块式dLLMs依赖于一种"重掩码"机制，该机制仅解码置信度最高的令牌并丢弃其余令牌，这实质上造成了计算资源的浪费。我们证明，回收这些被丢弃令牌的计算是有益的，因为这些令牌保留了有助于后续解码迭代的上下文信息。基于此，我们提出了残差上下文扩散（RCD）模块，该模块将这些被丢弃的令牌表示转换为上下文残差，并将其重新注入到下一个去噪步骤中。RCD采用解耦的两阶段训练流程，以规避反向传播相关的内存瓶颈。我们在长链思维推理（SDAR）和短链指令跟随（LLaDA）模型上验证了我们的方法。实验表明，标准dLLM仅需约10亿令牌即可高效转换为RCD范式。在广泛的基准测试中，RCD以极小的额外计算开销，持续将前沿dLLMs的准确率提升5-10个百分点。值得注意的是，在最具挑战性的AIME任务上，RCD几乎将基线准确率提升了一倍，并在同等准确率水平下实现了高达4-5倍的去噪步骤减少。