Traditional embeddings represent datapoints as vectors, which makes similarity easy to compute but limits how well they capture hierarchies and compositionality. We propose a fundamentally different approach: representing concepts as linear subspaces. By spanning multiple dimensions, subspaces can model broader concepts with higher-dimensional regions and nest more specific concepts within them. This geometry naturally captures generality through dimension, hierarchy through inclusion, and enables an emergent structure for composition via linear algebraic operations. To make this paradigm trainable, we introduce a differentiable subspace parameterization via soft projection matrices, allowing the effective dimension of each subspace to be learned. Our method not only achieves state-of-the-art performance on hierarchical and natural language inference benchmarks but also provides a geometrically-grounded model of entailment. Further, we demonstrate that while standard vector embeddings degrade to near-random performance on negated queries, subspace embeddings natively capture logical composition without explicit supervision, while preserving compatibility with efficient Euclidean vector search.

翻译：传统嵌入将数据点表示为向量，这使得相似度计算简便，但限制了对层次结构和组合性的捕捉能力。我们提出了一种根本不同的方法：将概念表示为线性子空间。通过跨越多个维度，子空间能够以高维区域建模更宽泛的概念，并将更具体的概念嵌套其中。这种几何结构通过维度自然表达一般性，通过包含关系表达层次性，并通过线性代数运算催生组合的涌现结构。为使这一范式可训练，我们通过软投影矩阵引入可微的子空间参数化方法，使每个子空间的有效维度得以学习。我们的方法不仅在层次结构和自然语言推理基准测试中取得了最先进的性能，还提供了基于几何的蕴含关系建模。此外，我们证明：当标准向量嵌入在取反查询中退化至接近随机性能时，子空间嵌入无需显式监督即能原生地捕捉逻辑组合，同时保持与高效欧几里得向量搜索的兼容性。