Large Language Models (LLMs) have demonstrated superior performance in listwise passage reranking task. However, directly applying them to rank long-form documents introduces both effectiveness and efficiency issues due to the substantially increased context length. To address this challenge, we propose a pointwise summarization model SumRank, aligned with downstream listwise reranking, to compress long-form documents into concise rank-aligned summaries before the final listwise reranking stage. To obtain our summarization model SumRank, we introduce a three-stage training pipeline comprising cold-start Supervised Fine-Tuning (SFT), specialized RL data construction, and rank-driven alignment via Reinforcement Learning. This paradigm aligns the SumRank with downstream ranking objectives to preserve relevance signals. We conduct extensive experiments on five benchmark datasets from the TREC Deep Learning tracks (TREC DL 19-23). Results show that our lightweight SumRank model achieves state-of-the-art (SOTA) ranking performance while significantly improving efficiency by reducing both summarization overhead and reranking complexity.
翻译:摘要:大语言模型(LLMs)在列表式段落重排任务中展现出卓越性能。然而,直接将其应用于长文档排序时,由于上下文长度显著增加,会引发有效性和效率问题。为应对这一挑战,我们提出基于逐点压缩的摘要模型SumRank,该模型与下游列表重排任务对齐,能够在最终列表重排阶段前将长文档压缩为简洁且对齐排序的摘要。为训练SumRank模型,我们引入三阶段训练流程:冷启动监督微调(SFT)、专用强化学习数据构建,以及基于强化学习的排序驱动对齐。该范式使SumRank与下游排序目标对齐以保留相关性信号。我们在TREC深度学习赛道(TREC DL 19-23)的五个基准数据集上进行广泛实验。结果表明,轻量级SumRank模型在实现最先进(SOTA)排序性能的同时,通过降低摘要开销和重排复杂度显著提升了效率。