Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length. On LongBench and InfiniteBench, Prefilling-dLLM achieves state-of-the-art quality among dLLM acceleration methods, and an attention kernel that parallelizes decoding over the non-contiguously cached chunk KV yields 9.1--28.0x speedup at 8K--32K contexts. We further show that beginning-of-sequence tokens prepended to each chunk act as periodic attention anchors that eliminate the lost-in-the-middle phenomenon. Code is available at https://github.com/menik1126/Prefilling-dLLM.

翻译：扩散大语言模型（dLLMs）在每个去噪步骤中重新编码整个前缀，导致计算量与上下文长度呈平方关系增长，这使得长上下文场景下的计算开销难以承受。我们提出Prefilling-dLLM——一种针对dLLMs的免训练预填充-解码分离框架，该框架将前缀划分为N个块，一次性缓存其键值（KV）表示，并利用块内令牌稀疏性选择最相关的top-K个块进行解码，表明稀疏预填充在性能上可媲美密集注意力，同时将每步复杂度从全序列长度的平方级降低至仅解码长度的平方级。在LongBench和InfiniteBench基准上，Prefilling-dLLM在dLLM加速方法中达到最优质量，且通过并行化非连续缓存块KV的解码注意力核，在8K–32K上下文下实现9.1–28.0倍加速。我们进一步证明，附加在每个块前的序列起始令牌（BOS）作为周期性注意力锚点，消除了“中间迷失”现象。代码已开源：https://github.com/menik1126/Prefilling-dLLM。