Few-shot learning aims to generalize the recognizer from seen categories to an entirely novel scenario. With only a few support samples, several advanced methods initially introduce class names as prior knowledge for identifying novel classes. However, obstacles still impede achieving a comprehensive understanding of how to harness the mutual advantages of visual and textual knowledge. In this paper, we propose a coherent Bidirectional Knowledge Permeation strategy called BiKop, which is grounded in a human intuition: A class name description offers a general representation, whereas an image captures the specificity of individuals. BiKop primarily establishes a hierarchical joint general-specific representation through bidirectional knowledge permeation. On the other hand, considering the bias of joint representation towards the base set, we disentangle base-class-relevant semantics during training, thereby alleviating the suppression of potential novel-class-relevant information. Experiments on four challenging benchmarks demonstrate the remarkable superiority of BiKop. Our code will be publicly available.
翻译:少样本学习旨在将识别器从已见类别泛化至全新场景。现有先进方法通常引入类别名称作为先验知识来识别新类别,但如何有效利用视觉与文本知识的相互优势仍存在障碍。本文提出一种基于人类直觉的连贯双向知识渗透策略BiKop:类别名称描述提供通用表征,而图像捕捉个体特异性。BiKop通过双向知识渗透构建层次化的联合通用-特异性表征。同时,针对联合表征偏向基类的偏差,我们在训练过程中解耦基类相关语义,从而缓解对潜在新类相关信息的抑制。在四个挑战性基准上的实验证明了BiKop的显著优越性。代码将公开提供。