Zero-Shot Sketch-Based Image Retrieval (ZSSBIR) is an emerging task. The pioneering work focused on the modal gap but ignored inter-class information. Although recent work has begun to consider the triplet-based or contrast-based loss to mine inter-class information, positive and negative samples need to be carefully selected, or the model is prone to lose modality-specific information. To respond to these issues, an Ontology-Aware Network (OAN) is proposed. Specifically, the smooth inter-class independence learning mechanism is put forward to maintain inter-class peculiarity. Meanwhile, distillation-based consistency preservation is utilized to keep modality-specific information. Extensive experiments have demonstrated the superior performance of our algorithm on two challenging Sketchy and Tu-Berlin datasets.
翻译:零样本草图图像检索(ZSSBIR)是一项新兴任务。早期研究主要关注模态差异,却忽略了类别间信息。尽管近期研究开始采用基于三元组或对比损失的策略挖掘类别间信息,但正负样本需精心选取,否则模型易丢失模态特定信息。针对上述问题,本文提出一种本体感知网络(OAN)。具体而言,引入平滑的类别间独立性学习机制以维持类别间特异性,同时采用基于蒸馏的一致性保持策略保留模态特定信息。大量实验表明,本算法在两个具有挑战性的Sketchy和Tu-Berlin数据集上均展现出优越性能。