Large Language Models (LLMs) are increasingly employed in zero-shot documents ranking, yielding commendable results. However, several significant challenges still persist in LLMs for ranking: (1) LLMs are constrained by limited input length, precluding them from processing a large number of documents simultaneously; (2) The output document sequence is influenced by the input order of documents, resulting in inconsistent ranking outcomes; (3) Achieving a balance between cost and ranking performance is quite challenging. To tackle these issues, we introduce a novel documents ranking method called TourRank, which is inspired by the tournament mechanism. This approach alleviates the impact of LLM's limited input length through intelligent grouping, while the tournament-like points system ensures robust ranking, mitigating the influence of the document input sequence. We test TourRank with different LLMs on the TREC DL datasets and the BEIR benchmark. Experimental results show that TourRank achieves state-of-the-art performance at a reasonable cost.

翻译：大语言模型（LLMs）在零样本文档排序任务中的应用日益广泛，并取得了值得称道的效果。然而，LLMs在排序中仍面临若干重大挑战：（1）LLMs受限于有限的输入长度，无法同时处理大量文档；（2）输出文档序列受文档输入顺序的影响，导致排序结果不一致；（3）在成本与排序性能之间取得平衡极具挑战性。为解决这些问题，我们提出了一种受锦标赛机制启发的新型文档排序方法——TourRank。该方法通过智能分组缓解LLM输入长度限制的影响，同时采用类锦标赛积分系统确保排序的鲁棒性，从而减弱文档输入序列的影响。我们在TREC DL数据集和BEIR基准上使用不同LLMs对TourRank进行了测试。实验结果表明，TourRank能够以合理的成本实现最先进的排序性能。