Text ranking has witnessed significant advancements, attributed to the utilization of dual-encoder enhanced by Pre-trained Language Models (PLMs). Given the proliferation of available PLMs, selecting the most effective one for a given dataset has become a non-trivial challenge. As a promising alternative to human intuition and brute-force fine-tuning, Transferability Estimation (TE) has emerged as an effective approach to model selection. However, current TE methods are primarily designed for classification tasks, and their estimated transferability may not align well with the objectives of text ranking. To address this challenge, we propose to compute the expected rank as transferability, explicitly reflecting the model's ranking capability. Furthermore, to mitigate anisotropy and incorporate training dynamics, we adaptively scale isotropic sentence embeddings to yield an accurate expected rank score. Our resulting method, Adaptive Ranking Transferability (AiRTran), can effectively capture subtle differences between models. On challenging model selection scenarios across various text ranking datasets, it demonstrates significant improvements over previous classification-oriented TE methods, human intuition, and ChatGPT with minor time consumption.

翻译：文本排序领域因预训练语言模型（PLM）增强的双编码器架构而取得显著进展。随着可用PLM数量的激增，如何为特定数据集选择最有效的模型已成为一项重要挑战。作为替代人类直觉与暴力微调的有前景方案，可迁移性估计（TE）已成为模型选择的有效途径。然而，当前TE方法主要针对分类任务设计，其估计的可迁移性可能与文本排序的目标存在偏差。为解决这一挑战，我们提出通过计算期望排名作为可迁移性度量，从而显式反映模型的排序能力。此外，为缓解各向异性问题并融入训练动态特征，我们通过自适应缩放各向同性的句子嵌入来获得精确的期望排名分数。我们提出的自适应排序可迁移性方法（AiRTran）能够有效捕捉模型间的细微差异。在跨多个文本排序数据集的复杂模型选择场景中，该方法相较于以往面向分类的TE方法、人类直觉以及ChatGPT均展现出显著优势，且仅需极低的时间开销。