Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn sketches limits data-dependent cross-modal learning algorithms, DFL can prove to be a much more practical paradigm. We thus propose Data-Free (DF)-SBIR, where, unlike existing DFL problems, pre-trained, single-modality classification models have to be leveraged to learn a cross-modal metric-space for retrieval without access to any training data. The widespread availability of pre-trained classification models, along with the difficulty in acquiring paired photo-sketch datasets for SBIR justify the practicality of this setting. We present a methodology for DF-SBIR, which can leverage knowledge from models independently trained to perform classification on photos and sketches. We evaluate our model on the Sketchy, TU-Berlin, and QuickDraw benchmarks, designing a variety of baselines based on state-of-the-art DFL literature, and observe that our method surpasses all of them by significant margins. Our method also achieves mAPs competitive with data-dependent approaches, all the while requiring no training data. Implementation is available at \url{https://github.com/abhrac/data-free-sbir}.
翻译:对深度学习模型隐私性和匿名性保护的日益关注推动了无数据学习领域的研究。我们首次发现,在草图图像检索这类数据稀缺任务中,由于难以获取配对的照片与手绘草图数据集而限制了数据依赖的跨模态学习算法,无数据学习被证明是一种更为实用的范式。为此,我们提出无数据草图图像检索方法,与现有无数据学习问题不同,该方法需要利用预训练的单模态分类模型,在无需任何训练数据的情况下学习用于检索的跨模态度量空间。预训练分类模型的广泛可及性,结合草图图像检索中配对照片-草图数据集获取的困难性,充分验证了该场景的实用性。我们提出一种无数据草图图像检索方法,能够利用分别在照片和草图上独立训练的分类模型知识。在Sketchy、TU-Berlin和QuickDraw基准测试上,我们基于最新无数据学习文献设计了多种基线方法,实验表明我们的方法显著超越所有基线。该方法在完全无需训练数据的情况下,取得了与数据依赖方法相竞争的平均精度均值。实现代码见:\url{https://github.com/abhrac/data-free-sbir}。