The huge domain gap between sketches and photos and the highly abstract sketch representations pose challenges for sketch-based image retrieval (\underline{SBIR}). The zero-shot sketch-based image retrieval (\underline{ZS-SBIR}) is more generic and practical but poses an even greater challenge because of the additional knowledge gap between the seen and unseen categories. To simultaneously mitigate both gaps, we propose an \textbf{A}pproaching-and-\textbf{C}entralizing \textbf{Net}work (termed "\textbf{ACNet}") to jointly optimize sketch-to-photo synthesis and the image retrieval. The retrieval module guides the synthesis module to generate large amounts of diverse photo-like images which gradually approach the photo domain, and thus better serve the retrieval module than ever to learn domain-agnostic representations and category-agnostic common knowledge for generalizing to unseen categories. These diverse images generated with retrieval guidance can effectively alleviate the overfitting problem troubling concrete category-specific training samples with high gradients. We also discover the use of proxy-based NormSoftmax loss is effective in the zero-shot setting because its centralizing effect can stabilize our joint training and promote the generalization ability to unseen categories. Our approach is simple yet effective, which achieves state-of-the-art performance on two widely used ZS-SBIR datasets and surpasses previous methods by a large margin.

翻译：草图和照片之间的巨大域差异以及高度抽象的草图表示为草图图像检索（SBIR）带来了挑战。零样本草图图像检索（ZS-SBIR）更加通用和实用，但由于可见与不可见类别之间的额外知识差距，其难度更大。为同时缓解这两个差距，我们提出了一种趋近与居中网络（简称"ACNet"），以联合优化草图到照片的合成与图像检索。检索模块引导合成模块生成大量多样化的类照片图像，这些图像逐渐趋近照片域，从而比以往更好地服务于检索模块，以学习域无关表示和类别无关的通用知识，从而泛化到不可见类别。这些在检索引导下生成的多样化图像能有效缓解困扰具体类别训练样本的高梯度过拟合问题。我们还发现，基于代理的NormSoftmax损失在零样本设置中效果显著，因为其居中效应能够稳定我们的联合训练，并提升对不可见类别的泛化能力。我们的方法简单而有效，在两个广泛使用的ZS-SBIR数据集上达到了最先进的性能，并大幅超越了先前的方法。