PromptReps showed that an autoregressive language model can be used directly as a retriever by prompting it to generate dense and sparse representations of a query or passage. Extending this to multiple representatives is inefficient for autoregressive models, since tokens must be generated sequentially, and prior multi-token variants did not reliably improve over single-token decoding. We show that the bottleneck is sequential generation, not the multi-token idea itself. DiffRetriever is a representative-token retriever for diffusion language models: it appends K masked positions to the prompt and reads all K in a single bidirectional forward pass. Across in-domain and out-of-domain evaluation, multi-token DiffRetriever substantially improves over single-token on every diffusion backbone we test, while autoregressive multi-token is flat or negative and pays a latency cost that scales with K where diffusion does not. After supervised fine-tuning, DiffRetriever on Dream is the strongest BEIR-7 retriever in our comparison, ahead of PromptReps, the encoder-style DiffEmbed baseline on the same diffusion backbones, and the contrastively fine-tuned single-vector RepLLaMA. A per-query oracle on the frozen base model exceeds contrastive fine-tuning at the same fixed budget, pointing to adaptive budget selection as future work. Code is available at https://github.com/ielab/diffretriever.

翻译：PromptReps 研究表明，自回归语言模型可通过提示生成查询或段落的稠密/稀疏表征，直接作为检索器使用。将该方法扩展至多表征场景对自回归模型而言效率低下——由于令牌必须顺序生成，且先前的多令牌变体未能可靠地超越单令牌解码。我们证明性能瓶颈在于顺序生成机制，而非多令牌思想本身。DiffRetriever 是一种面向扩散语言模型的表征令牌检索器：它在提示后附加 K 个掩码位置，通过单次双向前向传播同时读取全部 K 个令牌。在域内与跨域评估中，多令牌 DiffRetriever 在所有测试的扩散骨干网络上均显著优于单令牌版本，而自回归多令牌方案要么性能持平甚至下降，且其延迟成本随 K 值线性增长（扩散模型无此问题）。经监督微调后，基于 Dream 的 DiffRetriever 成为我们对比实验中最强的 BEIR-7 检索器，超越了 PromptReps、相同扩散骨干网络上的编码器式 DiffEmbed 基线，以及对比微调的单向量 RepLLaMA。在冻结基模型上使用逐查询 oracle 方法，可在相同固定预算下超越对比微调性能，这为自适应预算选择指明了未来研究方向。代码已开源至 https://github.com/ielab/diffretriever。