This paper introduces a novel unsupervised technique that utilizes large language models (LLMs) to determine the most suitable dense retriever for a specific test(target) corpus. Selecting the appropriate dense retriever is vital for numerous IR applications that employ these retrievers, trained on public datasets, to encode or conduct searches within a new private target corpus. The effectiveness of a dense retriever can significantly diminish when applied to a target corpus that diverges in domain or task from the original training set. The problem becomes more pronounced in cases where the target corpus is unlabeled, e.g. in zero-shot scenarios, rendering direct evaluation of the model's effectiveness on the target corpus unattainable. Therefore, the unsupervised selection of an optimally pre-trained dense retriever, especially under conditions of domain shift, emerges as a critical challenge. Existing methodologies for ranking dense retrievers fall short in addressing these domain shift scenarios. To tackle this, our method capitalizes on LLMs to create pseudo-relevant queries, labels, and reference lists by analyzing a subset of documents from the target corpus. This allows for the ranking of dense retrievers based on their performance with these pseudo-relevant signals. Significantly, this strategy is the first to depend exclusively on the target corpus data, removing the necessity for training data and test labels. We assessed the effectiveness of our approach by compiling a comprehensive pool of cutting-edge dense retrievers and comparing our method against traditional dense retriever selection benchmarks. The findings reveal that our proposed solution surpasses the existing benchmarks in both the selection and ranking of dense retrievers.

翻译：本文提出了一种新颖的无监督技术，利用大语言模型来确定最适合特定测试（目标）语料库的密集检索器。选择适当的密集检索器对于众多采用这些检索器、在公开数据集上训练、并在新的私有目标语料库中进行编码或搜索的信息检索应用至关重要。当目标语料库在领域或任务上偏离原始训练集时，密集检索器的有效性可能显著下降。这一问题在目标语料库无标签的情况下（例如零样本场景）尤为突出，导致无法直接评估模型在目标语料库上的效果。因此，无监督地选择最优预训练密集检索器，特别是在领域偏移条件下，成为一项关键挑战。现有的密集检索器排序方法在处理这些领域偏移场景时存在不足。为解决这一难题，我们的方法利用大语言模型，通过分析目标语料库中的文档子集，生成伪相关查询、标签和参考列表，从而根据密集检索器在这些伪相关信号上的表现对其进行排序。值得注意的是，这一策略是首个仅依赖目标语料库数据的方法，消除了对训练数据和测试标签的需求。我们通过构建一个涵盖前沿密集检索器的综合池，并将我们的方法与传统的密集检索器选择基准进行对比，评估了该方法的有效性。结果表明，我们提出的方案在密集检索器的选择与排序方面均超越了现有基准。