Utilizing large language models (LLMs) for zero-shot document ranking is done in one of two ways: 1) prompt-based re-ranking methods, which require no further training but are only feasible for re-ranking a handful of candidate documents due to computational costs; and 2) unsupervised contrastive trained dense retrieval methods, which can retrieve relevant documents from the entire corpus but require a large amount of paired text data for contrastive training. In this paper, we propose PromptReps, which combines the advantages of both categories: no need for training and the ability to retrieve from the whole corpus. Our method only requires prompts to guide an LLM to generate query and document representations for effective document retrieval. Specifically, we prompt the LLMs to represent a given text using a single word, and then use the last token's hidden states and the corresponding logits associated with the prediction of the next token to construct a hybrid document retrieval system. The retrieval system harnesses both dense text embedding and sparse bag-of-words representations given by the LLM. We further explore variations of this core idea that consider the generation of multiple words, and representations that rely on multiple embeddings and sparse distributions. Our experimental evaluation on the MSMARCO, TREC deep learning and BEIR zero-shot document retrieval datasets illustrates that this simple prompt-based LLM retrieval method can achieve a similar or higher retrieval effectiveness than state-of-the-art LLM embedding methods that are trained with large amounts of unsupervised data, especially when using a larger LLM.

翻译：利用大型语言模型（LLMs）进行零样本文档排序主要采用两种方式：1）基于提示的重排序方法，该方法无需额外训练，但由于计算成本限制，仅适用于对少量候选文档进行重排序；2）无监督对比训练的稠密检索方法，该方法能够从整个语料库中检索相关文档，但需要大量成对文本数据进行对比训练。本文提出PromptReps方法，融合了这两类方法的优势：既无需训练，又能实现全语料库检索。我们的方法仅需通过提示引导LLM生成查询与文档表征，即可实现有效的文档检索。具体而言，我们提示LLM使用单个词语表征给定文本，继而利用最后一个词元的隐藏状态及其对应下一词元预测的对数概率，构建混合文档检索系统。该系统同时利用了LLM提供的稠密文本嵌入与稀疏词袋表征。我们进一步探索了该核心思想的多种变体，包括生成多个词语的提示方案，以及基于多重嵌入与稀疏分布的表征方法。在MSMARCO、TREC深度学习及BEIR零样本文档检索数据集上的实验评估表明，这种基于提示的简单LLM检索方法能够达到甚至超越当前最先进的、依赖大量无监督数据训练的LLM嵌入方法的检索效能，尤其是在使用更大规模LLM时效果更为显著。