Vector-based retrieval systems have become a common staple for academic and industrial search applications because they provide a simple and scalable way of extending the search to leverage contextual representations for documents and queries. As these vector-based systems rely on contextual language models, their usage commonly requires GPUs, which can be expensive and difficult to manage. Given recent advances in introducing sparsity into language models for improved inference efficiency, in this paper, we study how sparse language models can be used for dense retrieval to improve inference efficiency. Using the popular retrieval library Tevatron and the MSMARCO, NQ, and TriviaQA datasets, we find that sparse language models can be used as direct replacements with little to no drop in accuracy and up to 4.3x improved inference speeds

翻译：基于向量的检索系统已成为学术和工业搜索应用中的常见工具，因为它们提供了一种简单且可扩展的方式，将搜索扩展以利用文档和查询的上下文表示。由于这些基于向量的系统依赖于上下文语言模型，其使用通常需要GPU，而GPU可能昂贵且难以管理。鉴于近期在语言模型中引入稀疏性以提高推理效率的进展，本文研究了如何利用稀疏语言模型进行密集检索以提升推理效率。通过使用流行的检索库Tevatron以及MSMARCO、NQ和TriviaQA数据集，我们发现稀疏语言模型可作为直接替代方案，在精度几乎不下降的情况下，推理速度最高提升4.3倍。