Text ranking is a critical task in various information retrieval applications, and the recent success of Large Language Models (LLMs) in natural language processing has sparked interest in their application to text ranking. These methods primarily involve combining query and candidate documents and leveraging prompt learning to determine query-document relevance using the LLM's output probabilities for specific tokens or by directly generating a ranked list of candidate documents. Although these approaches have demonstrated promise, a noteworthy disparity arises between the training objective of LLMs, which typically centers around next token prediction, and the objective of evaluating query-document relevance. To address this gap and fully leverage LLM potential in text ranking tasks, we propose a progressive multi-stage training strategy. Firstly, we introduce a large-scale weakly supervised dataset of relevance texts to enable the LLMs to acquire the ability to predict relevant tokens without altering their original training objective. Subsequently, we incorporate supervised training to further enhance LLM ranking capability. Our experimental results on multiple benchmarks demonstrate the superior performance of our proposed method compared to previous competitive approaches, both in in-domain and out-of-domain scenarios.

翻译：文本排序是信息检索应用中的关键任务，大语言模型（LLMs）在自然语言处理领域的成功激发了其在文本排序中的应用兴趣。现有方法主要通过组合查询与候选文档，利用提示学习，基于LLM对特定标记的输出概率或直接生成候选文档的排序列表，来确定查询-文档的相关性。尽管这些方法展示了潜力，但LLMs通常以预测下一标记为训练目标，这与评估查询-文档相关性的目标存在显著差距。为弥合这一差距并充分挖掘LLM在文本排序任务中的潜力，本文提出一种渐进式多阶段训练策略。首先，我们引入大规模弱监督相关性文本数据集，使LLM在不改变原有训练目标的前提下，获得预测相关性标记的能力。随后，通过监督式训练进一步提升LLM的排序能力。在多个基准测试上的实验结果表明，与以往的竞争方法相比，本文方法在域内和域外场景下均展现出更优越的性能。