Recent advances in large language models (LLMs) have inspired new paradigms for document reranking. While this paradigm better exploits the reasoning and contextual understanding capabilities of LLMs, most existing LLM-based rerankers rely on autoregressive generation, which limits their efficiency and flexibility. In particular, token-by-token decoding incurs high latency, while the fixed left-to-right generation order causes early prediction errors to propagate and is difficult to revise. To address these limitations, we explore the use of diffusion language models (dLLMs) for document reranking and propose DiffuRank, a reranking framework built upon dLLMs. Unlike autoregressive models, dLLMs support more flexible decoding and generation processes that are not constrained to a left-to-right order, and enable parallel decoding, which may lead to improved efficiency and controllability. Specifically, we investigate three reranking strategies based on dLLMs: (1) a pointwise approach that uses dLLMs to estimate the relevance of each query-document pair; (2) a logit-based listwise approach that prompts dLLMs to jointly assess the relevance of multiple documents and derives ranking lists directly from model logits; and (3) a permutation-based listwise approach that adapts the canonical decoding process of dLLMs to the reranking tasks. For each approach, we design corresponding training methods to fully exploit the advantages of dLLMs. We evaluate both zero-shot and fine-tuned reranking performance on multiple benchmarks. Experimental results show that dLLMs achieve performance comparable to, and in some cases exceeding, that of autoregressive LLMs with similar model sizes. These findings demonstrate the promise of diffusion-based language models as a compelling alternative to autoregressive architectures for document reranking.

翻译：近年来，大语言模型（LLMs）的进展为文档重排任务带来了新的范式。尽管这一范式能更好地利用LLMs的推理与上下文理解能力，但现有基于LLM的重排方法大多依赖于自回归生成，这限制了其效率与灵活性。具体而言，逐词解码会导致较高的延迟，而固定的从左到右生成顺序会使早期预测错误传播且难以修正。为应对这些局限，我们探索利用扩散语言模型（dLLMs）进行文档重排，并提出DiffuRank——一个基于dLLMs构建的重排框架。与自回归模型不同，dLLMs支持更灵活的解码与生成过程，不受限于从左到右的顺序，并能实现并行解码，从而可能提升效率与可控性。具体而言，我们研究了三种基于dLLMs的重排策略：（1）点式方法，使用dLLMs估计每个查询-文档对的相关性；（2）基于对数几率的列表式方法，提示dLLMs联合评估多个文档的相关性，并直接从模型对数几率推导排序列表；（3）基于排列的列表式方法，将dLLMs的标准解码过程适配至重排任务。针对每种方法，我们设计了相应的训练方式以充分发挥dLLMs的优势。我们在多个基准上评估了零样本与微调后的重排性能。实验结果表明，在模型规模相近的情况下，dLLMs取得了与自回归LLMs相当、甚至在某些情况下更优的性能。这些发现证明了基于扩散的语言模型作为自回归架构的有力替代方案，在文档重排任务中具有广阔前景。