Dense retrieval has become the dominant paradigm in information retrieval, in which each document is scored against a query by the inner product of their vector embeddings, and the top-$k$ documents by score are retrieved for this query. However, since each document's score depends solely on the embedding of the query and itself, the retrieval process is oblivious to the content of the entire corpus. Therefore, dense retrieval cannot avoid selecting semantically similar documents from the corpus, which may result in a non-diverse, redundant set of retrieved documents. To this end, we approach retrieval as a joint decoding problem, in which documents are selected as a set with regard to the context of the rest of the corpus. To achieve this, we propose Non-Negative elastic Net (NNN) decoding, which selects documents whose embeddings jointly reconstruct the query embedding as a sparse non-negative linear combination. Our main theoretical result establishes a strict separation between dense retrieval and NNN decoding. For any corpus, every query correctly handled by dense retrieval is also handled by NNN decoding, while on corpora containing correlated documents, NNN decoding additionally handles queries that dense retrieval cannot. Experimental results indicate that applying NNN decoding to frozen embeddings trained for inner-product scoring yields consistent improvements across several benchmarks. Moreover, we introduce an end-to-end training procedure which optimizes the embeddings for NNN decoding, producing significant performance gains surpassing in all metrics and benchmarks compared to dense retrieval. Our work establishes a new paradigm for leveraging dense embeddings in information retrieval, beyond the standard practice of inner-product scoring.

翻译：密集检索已成为信息检索中的主导范式，其中每个文档通过其向量嵌入与查询的内积进行评分，并根据分数检索该查询的前k个文档。然而，由于每个文档的分数仅取决于查询及其自身的嵌入，检索过程对整个语料库的内容一无所知。因此，密集检索无法避免从语料库中选择语义相似的文档，这可能导致检索到的文档集合缺乏多样性且冗余。为此，我们将检索视为一个联合解码问题，其中文档作为集合被选择，并考虑语料库其余部分的上下文。为实现这一目标，我们提出非负弹性网络（NNN）解码，该方法选择其嵌入能联合将查询嵌入重构为稀疏非负线性组合的文档。我们的主要理论结果建立了密集检索与NNN解码之间的严格分离。对于任意语料库，密集检索正确处理的每个查询也能被NNN解码处理，而在包含相关文档的语料库中，NNN解码额外处理了密集检索无法处理的查询。实验结果表明，将NNN解码应用于为内积评分训练的冻结嵌入，在多个基准测试中带来了一致的改进。此外，我们引入了一种端到端的训练程序，优化NNN解码的嵌入，在所有指标和基准测试中相比密集检索实现了显著的性能提升。我们的工作建立了一种利用密集嵌入进行信息检索的新范式，超越了标准的内积评分实践。