Discrete diffusion language models generate text by iteratively denoising an entire response in parallel. At each step, they predict tentative tokens for every masked position, committing the confident predictions to the output and discarding the unconfident ones. We show that the discarded tokens are in fact a useful lookahead signal for retrieval-augmented generation: even low-confidence tokens often surface salient entities early in the denoising trajectory, enabling retrieval of stronger evidence before the output is finalized. We exploit this through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses these lookahead tokens to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms current training-free diffusion and autoregressive retrieval baselines at up to $8\times$ higher throughput.

翻译：离散扩散语言模型通过并行迭代去噪整个响应来生成文本。每一步，它们为每个掩码位置预测候选词，将高置信度预测提交为输出，并丢弃低置信度预测。我们发现，被丢弃的标记实际上为检索增强生成提供了有用的前瞻信号：即使在去噪轨迹早期，低置信度标记也常能浮现关键实体，从而在输出最终确定前检索到更强证据。我们通过扩散语言模型的自增强检索（SARDI）来利用这一特性，这是一个动态RAG框架，在去噪过程中使用这些前瞻标记引导检索。SARDI无需训练、与检索器无关，且适用于任何具备推理能力的离散扩散语言模型。在五项多跳问答基准测试中，SARDI以高达8倍吞吐量超越了当前无训练扩散模型与自回归检索基线。