Learned Sparse IR models, such as SPLADE, offer an excellent efficiency-effectiveness tradeoff. However, they rely on the underlying backbone vocabulary, which might hinder performance (polysemicity and synonymy) and pose a challenge for multi-lingual and multi-modal usages. To solve this limitation, we propose to replace the backbone vocabulary with a latent space of semantic concepts learned using Sparse Auto-Encoders (SAE). Throughout this paper, we study the compatibility of these 2 concepts, explore training approaches, and analyze the differences between our SAE-SPLADE model and traditional SPLADE models. Our experiments demonstrate that SAE-SPLADE achieves retrieval performance comparable to SPLADE on both in-domain and out-of-domain tasks while offering improved efficiency.

翻译：学习的稀疏IR模型（如SPLADE）在效率与效果之间实现了出色的平衡。然而，这类模型依赖于底层骨干词汇表，这可能削弱其性能（如多义性和同义性问题），并给多语言及多模态应用场景带来挑战。为解决这一局限性，我们提出利用稀疏自编码器（SAE）学习到的语义概念潜在空间替代骨干词汇表。本文系统研究了这两个概念的兼容性，探索了训练方法，并分析了SAE-SPLADE模型与传统SPLADE模型之间的差异。实验结果表明，SAE-SPLADE在领域内和跨领域检索任务中均能达到与SPLADE相当的检索性能，同时效率得到显著提升。