Semantic hashing represents documents as compact binary vectors (hash codes) and allows both efficient and effective similarity search in large-scale information retrieval. The state of the art has primarily focused on learning hash codes that improve similarity search effectiveness, while assuming a brute-force linear scan strategy for searching over all the hash codes, even though much faster alternatives exist. One such alternative is multi-index hashing, an approach that constructs a smaller candidate set to search over, which depending on the distribution of the hash codes can lead to sub-linear search time. In this work, we propose Multi-Index Semantic Hashing (MISH), an unsupervised hashing model that learns hash codes that are both effective and highly efficient by being optimized for multi-index hashing. We derive novel training objectives, which enable to learn hash codes that reduce the candidate sets produced by multi-index hashing, while being end-to-end trainable. In fact, our proposed training objectives are model agnostic, i.e., not tied to how the hash codes are generated specifically in MISH, and are straight-forward to include in existing and future semantic hashing models. We experimentally compare MISH to state-of-the-art semantic hashing baselines in the task of document similarity search. We find that even though multi-index hashing also improves the efficiency of the baselines compared to a linear scan, they are still upwards of 33% slower than MISH, while MISH is still able to obtain state-of-the-art effectiveness.

翻译：语义混杂是压缩二进制矢量( hash code) 的文件, 并允许在大规模信息检索中进行高效且有效的相似搜索。 艺术状态主要侧重于学习能提高相似搜索效力的散列代码( MISH ), 而同时假设一种用于搜索所有散列代码的粗力线性扫描战略, 尽管存在更快的替代方法 。 一种这样的替代方法是多指数散列, 一种构建一个小候选人群以搜索为对象的方法, 取决于散列代码的分布, 它可以导致亚线性搜索时间。 在这项工作中, 我们提议多 Index 语义类类集搜索( MISH) (MIS), 这是一种非超超导型的散列的散列代码, 通过优化多指数类集搜索功能搜索, 高效力。 我们推出的新培训目标, 减少多指数类集生成的候选数据集, 同时又可以找到端到端端端端端点的。 事实上, 我们提出的培训目标是模型的模型,, 即甚至连线性直线性类搜索( MISH) (W- sal) lave- sal) rode) ro- sal to sal to sal lave lave lave lave lave laved to to to to the lave laved the sal de deal deal deal deal decude