For a long time, additive quantizers, such as product quantization, have been considered the gold standard in terms of accuracy and efficiency. Recently, scalar quantization has re-emerged from the depths of history with a new wave of data-agnostic techniques. Inscribed in this general framework, we turn our attention to data-driven methods, showing that new highs in recall and speed can be achieved by reducing the number of dimensions while increasing the bitrate per dimension. Critically, this dimensionality reduction needs to be learned from data to be successful. We present ASH (Asymmetric Scalar Hashing), a data-driven encoder-decoder framework that applies dimensionality reduction to database vectors via a learned orthonormal projection, followed by scalar quantization, while keeping queries in their original form. This asymmetric design enables higher accuracy than the best additive and scalar quantizers at iso-compression, while admitting highly efficient similarity computations via SIMD operations. ASH has short learning and encoding times, making it attractive for real-world deployment. Extensive experiments on a variety of datasets demonstrate that ASH achieves state-of-the-art ANN recall and speeds across all compression regimes.

翻译：长期以来，加法量化器（如乘积量化）在精度与效率方面一直被视为黄金标准。近期，标量量化伴随着新一波数据无关技术从历史深处重新浮现。在此总体框架内，我们聚焦于数据驱动方法，展示通过减少维度数量同时提高每个维度的比特率，可以在召回率和速度上达到新高度。至关重要的是，这种降维需要从数据中学习才能成功。我们提出ASH（非对称标量哈希），一种基于数据驱动的编码器-解码器框架，该框架通过可学习的标准正交投影对数据库向量进行降维，随后进行标量量化，同时保持查询向量原始形式不变。这种非对称设计能够在等压缩率下实现优于最优加法量化器与标量量化器的精度，同时通过SIMD操作支持高效的相似度计算。ASH具有较短的学习与编码时间，使其在实际部署中具有吸引力。在多种数据集上的广泛实验表明，ASH在所有压缩率下均实现了最先进的ANN召回率与速度。