The data-hungry problem, characterized by insufficiency and low-quality of data, poses obstacles for deep learning models. Transfer learning has been a feasible way to transfer knowledge from high-quality external data of source domains to limited data of target domains, which follows a domain-level knowledge transfer to learn a shared posterior distribution. However, they are usually built on strong assumptions, e.g., the domain invariant posterior distribution, which is usually unsatisfied and may introduce noises, resulting in poor generalization ability on target domains. Inspired by Graph Neural Networks (GNNs) that aggregate information from neighboring nodes, we redefine the paradigm as learning a knowledge-enhanced posterior distribution for target domains, namely Knowledge Bridge Learning (KBL). KBL first learns the scope of knowledge transfer by constructing a Bridged-Graph that connects knowledgeable samples to each target sample and then performs sample-wise knowledge transfer via GNNs.KBL is free from strong assumptions and is robust to noises in the source data. Guided by KBL, we propose the Bridged-GNN} including an Adaptive Knowledge Retrieval module to build Bridged-Graph and a Graph Knowledge Transfer module. Comprehensive experiments on both un-relational and relational data-hungry scenarios demonstrate the significant improvements of Bridged-GNN compared with SOTA methods
翻译:数据稀缺问题,其表现为数据不足和质量低下,对深度学习模型构成了障碍。迁移学习是一种可行的方法,可将源域高质量外部数据的知识迁移至目标域的有限数据中,其通过域级知识迁移来学习共享的后验分布。然而,这些方法通常建立在强假设之上,例如域不变后验分布,这一假设通常无法满足且可能引入噪声,导致在目标域上的泛化能力较差。受图神经网络(GNN)从邻接节点聚合信息的启发,我们重新定义了知识迁移范式,即学习面向目标域的知识增强后验分布,称为知识桥接学习(KBL)。KBL首先通过构建一个连接知识样本与每个目标样本的桥接图来界定知识迁移的范围,然后通过图神经网络执行样本级知识迁移。KBL不受强假设的约束,且对源数据中的噪声具有鲁棒性。基于KBL的指导,我们提出了Bridged-GNN,包括一个自适应知识检索模块(用于构建桥接图)和一个图知识迁移模块。在非关系型和关系型数据稀缺场景下的综合实验表明,Bridged-GNN相较于现有最优方法具有显著性能提升。