This paper shows how diffusion language models (DLMs) can be used as effective and efficient retrievers. Existing DLM-based retrievers (e.g., DiffEmbed) follow BERT-style encoding, representing each query or passage as a single mean-pooled vector. This ignores how DLMs are trained to generate responses through masked-position prediction under bidirectional attention, a capability that can provide stronger retrieval signals. We propose DiffRetriever, which uses the DLM's native masked-position prediction directly for retrieval. For each query or passage, DiffRetriever appends one or more masked positions, using the outputs as retrieval representations in a single forward pass. With one masked position, single-representation DiffRetriever already improves over DiffEmbed on the same backbones. DiffRetriever also naturally extends to multi-representation retrieval: DLMs process multiple masked positions jointly, enabling ColBERT-style fine-grained matching with little additional encoding latency. In autoregressive LLM retrievers, the same multi-representation strategy requires sequential decoding and therefore incurs much higher latency. DiffRetriever obtains the strongest aggregate effectiveness within our matched comparison, outperforming DiffEmbed, PromptReps, and RepLLaMA. Masked-position counts selected on training data transfer well across datasets, while per-query variation suggests headroom for adaptive allocation. Code is available at https://github.com/ielab/diffretriever.

翻译：本文展示了扩散语言模型（DLMs）如何被用作高效且有效的检索器。现有基于DLM的检索器（如DiffEmbed）遵循BERT式编码，将每个查询或段落表示为单一的均值池化向量。这种方法忽略了DLM通过双向注意力机制下的掩码位置预测来生成响应的能力——而该能力可提供更强的检索信号。我们提出DiffRetriever，它直接利用DLM的原生掩码位置预测能力进行检索。对于每个查询或段落，DiffRetriever附加一个或多个掩码位置，并通过单次前向传播将输出结果作为检索表示。当使用单个掩码位置时，单一表示的DiffRetriever在相同骨干网络上已优于DiffEmbed。DiffRetriever还自然扩展至多表示检索：DLM联合处理多个掩码位置，从而实现ColBERT式的细粒度匹配，且编码延迟增加极小。在自回归大语言模型检索器中，相同的多表示策略需要顺序解码，因此会产生更高的延迟。在我们的对比实验中，DiffRetriever获得了最强的综合效能，优于DiffEmbed、PromptReps和RepLLaMA。基于训练数据选择的掩码位置数量在不同数据集间具有良好的迁移性，而逐查询的差异表明自适应分配尚有提升空间。代码已开源至https://github.com/ielab/diffretriever。