Practitioners working on dense retrieval today face a bewildering number of choices. Beyond selecting the embedding model, another consequential choice is the actual implementation of nearest-neighbor vector search. While best practices recommend HNSW indexes, flat vector indexes with brute-force search represent another viable option, particularly for smaller corpora and for rapid prototyping. In this paper, we provide experimental results on the BEIR dataset using the open-source Lucene search library that explicate the tradeoffs between HNSW and flat indexes (including quantized variants) from the perspectives of indexing time, query evaluation performance, and retrieval quality. With additional comparisons between dense and sparse retrievers, our results provide guidance for today's search practitioner in understanding the design space of dense and sparse retrievers. To our knowledge, we are the first to provide operational advice supported by empirical experiments in this regard.

翻译：当前从事密集检索的研究者面临着令人困惑的众多选择。除了选择嵌入模型外，另一个关键决策是最近邻向量搜索的实际实现方式。尽管最佳实践推荐使用HNSW索引，但采用暴力搜索的扁平向量索引也是可行的替代方案，尤其适用于较小规模的语料库和快速原型开发。本文基于开源Lucene搜索库在BEIR数据集上进行了实验，从索引构建时间、查询评估性能和检索质量等角度，阐明了HNSW索引与扁平索引（包括量化变体）之间的权衡。通过进一步比较密集检索器与稀疏检索器，我们的研究结果为当今搜索实践者理解密集与稀疏检索器的设计空间提供了指导。据我们所知，我们是首个通过实证实验为此类操作建议提供支持的研究。