Unsupervised semantic hashing has emerged as an indispensable technique for fast image search, which aims to convert images into binary hash codes without relying on labels. Recent advancements in the field demonstrate that employing large-scale backbones (e.g., ViT) in unsupervised semantic hashing models can yield substantial improvements. However, the inference delay has become increasingly difficult to overlook. Knowledge distillation provides a means for practical model compression to alleviate this delay. Nevertheless, the prevailing knowledge distillation approaches are not explicitly designed for semantic hashing. They ignore the unique search paradigm of semantic hashing, the inherent necessities of the distillation process, and the property of hash codes. In this paper, we propose an innovative Bit-mask Robust Contrastive knowledge Distillation (BRCD) method, specifically devised for the distillation of semantic hashing models. To ensure the effectiveness of two kinds of search paradigms in the context of semantic hashing, BRCD first aligns the semantic spaces between the teacher and student models through a contrastive knowledge distillation objective. Additionally, to eliminate noisy augmentations and ensure robust optimization, a cluster-based method within the knowledge distillation process is introduced. Furthermore, through a bit-level analysis, we uncover the presence of redundancy bits resulting from the bit independence property. To mitigate these effects, we introduce a bit mask mechanism in our knowledge distillation objective. Finally, extensive experiments not only showcase the noteworthy performance of our BRCD method in comparison to other knowledge distillation methods but also substantiate the generality of our methods across diverse semantic hashing models and backbones. The code for BRCD is available at https://github.com/hly1998/BRCD.

翻译：无监督语义哈希已成为快速图像搜索中不可或缺的技术，其目标是在不依赖标签的情况下将图像转换为二进制哈希码。该领域的最新进展表明，在无监督语义哈希模型中使用大规模骨干网络（如ViT）可以带来显著性能提升。然而，推理延迟问题日益难以忽视。知识蒸馏提供了一种实用的模型压缩手段来缓解这一延迟。但现有知识蒸馏方法并非专为语义哈希设计，忽视了语义哈希独特的搜索范式、蒸馏过程的固有需求以及哈希码的特性。本文提出一种创新的比特掩码鲁棒对比知识蒸馏（BRCD）方法，专门针对语义哈希模型的蒸馏设计。为确保语义哈希中两种搜索范式的有效性，BRCD首先通过对比知识蒸馏目标对齐师生模型之间的语义空间。此外，为消除噪声增强并确保鲁棒优化，在知识蒸馏过程中引入了基于聚类的方法。同时，通过比特级分析，我们发现比特独立性导致的冗余比特问题。为减轻其影响，我们在知识蒸馏目标中引入比特掩码机制。最终，大量实验不仅展示了BRCD方法相较于其他知识蒸馏方法的显著性能，也验证了该方法在不同语义哈希模型和骨干网络中的通用性。BRCD代码开源地址：https://github.com/hly1998/BRCD