Why do even the most powerful 7B-parameter embedding models struggle with simple retrieval tasks that the decades old BM25 handles with ease? Recent theory suggests that this happens because of a dimensionality bottleneck. This occurs when we force infinite linguistic nuances into small, fixed-length learned vectors. We developed NUMEN to break this bottleneck by removing the learning process entirely. Instead of training heavy layers to map text to a constrained space, NUMEN uses deterministic character hashing to project language directly onto high-dimensional vectors. This approach requires no training, supports an unlimited vocabulary, and allows the geometric capacity scale as needed. On the LIMIT benchmark, NUMEN achieves 93.90 % Recall@100 at 32,768 dimensions. This makes it the first dense retrieval model to officially surpass the sparse BM25 baseline 93.6 %. Our findings show that the real problem in dense retrieval isn't the architecture, but the embedding layer itself. The solution isn't necessarily smarter training, but simply providing more room to breathe.

翻译：为何即使是最强大的70亿参数嵌入模型，在应对数十年前的BM25算法轻松处理的简单检索任务时仍显乏力？近期理论表明，这源于维度瓶颈效应——当我们将无限的语言细微差异强行压缩至有限长度的学习向量时，就会发生这种现象。我们开发了NUMEN模型来突破这一瓶颈，其核心在于完全移除学习过程。与训练繁重的网络层将文本映射到受限空间不同，NUMEN采用确定性字符哈希技术，直接将语言投影到高维向量空间。该方法无需训练过程，支持无限词汇表，并允许几何容量按需扩展。在LIMIT基准测试中，NUMEN在32,768维度下实现了93.90%的Recall@100指标，成为首个正式超越稀疏检索基线BM25（93.6%）的稠密检索模型。我们的研究揭示：稠密检索的真正问题并非架构设计，而是嵌入层本身；解决方案未必需要更精巧的训练策略，而只需提供更充裕的表示空间。