In recent years, deep models have achieved remarkable success in various vision tasks. However, their performance heavily relies on large training datasets. In contrast, humans exhibit hybrid learning, seamlessly integrating structured knowledge for cross-domain recognition or relying on a smaller amount of data samples for few-shot learning. Motivated by this human-like epistemic process, we aim to extend hybrid learning to computer vision tasks by integrating structured knowledge with data samples for more effective representation learning. Nevertheless, this extension faces significant challenges due to the substantial gap between structured knowledge and deep features learned from data samples, encompassing both dimensions and knowledge granularity. In this paper, a novel Epistemic Graph Layer (EGLayer) is introduced to enable hybrid learning, enhancing the exchange of information between deep features and a structured knowledge graph. Our EGLayer is composed of three major parts, including a local graph module, a query aggregation model, and a novel correlation alignment loss function to emulate human epistemic ability. Serving as a plug-and-play module that can replace the standard linear classifier, EGLayer significantly improves the performance of deep models. Extensive experiments demonstrates that EGLayer can greatly enhance representation learning for the tasks of cross-domain recognition and few-shot learning, and the visualization of knowledge graphs can aid in model interpretation.

翻译：近年来，深度模型在各种视觉任务中取得了显著成功。然而，其性能高度依赖于大规模训练数据集。相比之下，人类展现出混合学习能力，能够无缝整合结构化知识进行跨领域识别，或仅依赖少量数据样本进行少样本学习。受这种类人认知过程的启发，我们旨在通过将结构化知识与数据样本相结合，将混合学习扩展到计算机视觉任务中，以实现更有效的表征学习。然而，由于结构化知识与从数据样本中学习的深度特征之间存在维度与知识粒度的显著差异，这一扩展面临重大挑战。本文提出了一种新型认知图谱层（EGLayer），通过增强深度特征与结构化知识图谱之间的信息交换来实现混合学习。我们的EGLayer由三个主要部分组成：局部图模块、查询聚合模型以及一种用于模拟人类认知能力的新型相关性对齐损失函数。作为可替代标准线性分类器的即插即用模块，EGLayer显著提升了深度模型的性能。大量实验证明，EGLayer能极大增强跨领域识别与少样本学习任务中的表征学习能力，且知识图谱的可视化有助于模型解释。