Text ranking is a critical task in information retrieval. Recent advances in pre-trained language models (PLMs), especially large language models (LLMs), present new opportunities for applying them to text ranking. While supervised fine-tuning (SFT) with ranking data has been widely explored to better align PLMs with text ranking goals, previous studies have focused primarily on encoder-only and encoder-decoder PLMs. Research on leveraging decoder-only LLMs for text ranking remains scarce. An exception to this is RankLLaMA, which uses direct SFT to explore LLaMA's potential for text ranking. In this work, we propose a two-stage progressive paradigm to better adapt LLMs to text ranking. First, we conduct continual pre-training (CPT) of LLMs on a large weakly-supervised corpus. Second, we perform SFT, and propose an improved optimization strategy building upon RankLLaMA. Our experimental results on multiple benchmarks show that our approach outperforms previous methods in both in-domain and out-domain scenarios.

翻译：文本排序是信息检索中的关键任务。预训练语言模型（尤其是大型语言模型）的最新进展为将其应用于文本排序提供了新的机遇。虽然通过排序数据进行监督微调已被广泛探索以更好地使预训练语言模型与文本排序目标对齐，但先前研究主要集中于仅编码器和编码器-解码器架构的预训练语言模型。利用仅解码器大型语言模型进行文本排序的研究仍然匮乏。RankLLaMA是一个例外，它采用直接监督微调来探索LLaMA在文本排序中的潜力。本工作提出一种两阶段渐进式范式以更好地使大型语言模型适应文本排序任务。首先，我们在大规模弱监督语料库上对大型语言模型进行持续预训练；其次，我们实施监督微调，并在RankLLaMA基础上提出改进的优化策略。在多个基准测试上的实验结果表明，我们的方法在领域内和跨域场景中均优于现有方法。