Large language model (LLM) rerankers have become an important component of modern retrieval and retrieval-augmented generation pipelines, but their high computational cost limits their applicability to long candidate lists. In this paper, we propose \textbf{CompRank}, a token-efficient reranking framework that reduces redundant computation by aligning reranker design with the sparsity of ranking signals. CompRank decouples document representations from candidate order and query context, enabling reusable document-side states; applies segment-wise token compression to reduce query--document interaction cost; and introduces a CopyNet-style objective that directly aligns attention-based document scoring with training supervision. Experiments on seven BEIR datasets show that CompRank achieves strong reranking performance while retaining only 10.2\% of document tokens, reaching an average NDCG@10 of 39.2 compared with 39.7 under full-token attention. Further scaling experiments on TREC-COVID show that CompRank remains stable when evaluated on candidate lists of up to 500 documents after training on 30-document lists, while achieving $4.9\times$--$9.5\times$ end-to-end speedup over generation-based listwise reranking and approximately $1.3\times$ speedup over the full-token CompRank variant. These results suggest that token-level compression and decoding-free attention scoring provide an effective path toward scalable LLM-based reranking.

翻译：论文摘要：大语言模型重排序器已成为现代检索与检索增强生成流水线的重要组成部分，但其高昂的计算成本限制了其在长候选列表中的应用。本文提出**CompRank**，一种令牌高效的重排序框架，通过将重排序器设计与排序信号的稀疏性对齐来减少冗余计算。CompRank将文档表示与候选顺序和查询上下文解耦，支持可复用的文档侧状态；采用分段式令牌压缩以降低查询-文档交互成本；并引入一种CopyNet风格的目标函数，直接将对数注意力文档评分与训练监督对齐。在七个BEIR数据集上的实验表明，CompRank在仅保留10.2%文档令牌的情况下仍能取得强劲的重排序性能，平均NDCG@10达到39.2，而全令牌注意力机制下的结果为39.7。在TREC-COVID上的进一步扩展实验表明，CompRank在基于30文档列表训练后，评估包含多达500个文档的候选列表时仍保持稳定，同时相比基于生成式列表重排序实现4.9倍至9.5倍的端到端加速，相比全令牌CompRank变体实现约1.3倍加速。这些结果表明，令牌级压缩与免解码注意力评分为实现可扩展的大语言模型重排序提供了有效途径。