Cross-encoders distilled from large language models (LLMs) are often more effective re-rankers than cross-encoders fine-tuned on manually labeled data. However, distilled models do not match the effectiveness of their teacher LLMs. We hypothesize that this effectiveness gap is due to the fact that previous work has not applied the best-suited methods for fine-tuning cross-encoders on manually labeled data (e.g., hard-negative sampling, deep sampling, and listwise loss functions). To close this gap, we create a new dataset, Rank-DistiLLM. Cross-encoders trained on Rank-DistiLLM achieve the effectiveness of LLMs while being up to 173 times faster and 24 times more memory efficient. Our code and data is available at https://github.com/webis-de/ECIR-25.

翻译：从大型语言模型（LLMs）蒸馏得到的交叉编码器通常比基于人工标注数据微调的交叉编码器具有更优的重排序效能。然而，蒸馏模型仍无法达到其教师LLMs的效能水平。我们假设这一效能差距源于先前工作未能采用最适合基于人工标注数据微调交叉编码器的方法（例如：困难负样本采样、深度采样及列表损失函数）。为弥合此差距，我们构建了一个新数据集Rank-DistiLLM。基于Rank-DistiLLM训练的交叉编码器在达到LLMs效能水平的同时，速度提升最高达173倍，内存效率提高最高达24倍。我们的代码与数据公开于https://github.com/webis-de/ECIR-25。