Non-IID transfer learning on graphs is crucial in many high-stakes domains. The majority of existing works assume stationary distribution for both source and target domains. However, real-world graphs are intrinsically dynamic, presenting challenges in terms of domain evolution and dynamic discrepancy between source and target domains. To bridge the gap, we shift the problem to the dynamic setting and pose the question: given the label-rich source graphs and the label-scarce target graphs both observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer it, we propose a generalization bound for dynamic non-IID transfer learning on graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target graphs. Inspired by the theoretical results, we introduce a novel generic framework named EvoluNet. It leverages a transformer-based temporal encoding module to model temporal information of the evolving domains and then uses a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, EvoluNet outperforms the state-of-the-art models by up to 12.1%, demonstrating its effectiveness in transferring knowledge from dynamic source graphs to dynamic target graphs.
翻译:图上的非独立同分布迁移学习在许多高风险领域至关重要。现有研究大多假设源域和目标域的分布是平稳的。然而,现实世界中的图本质上是动态的,这带来了域演化以及源域与目标域之间动态差异的挑战。为弥补这一差距,我们将问题转向动态设定,并提出以下问题:给定在先前T个时间戳中观测到的标签丰富的源图和标签稀缺的目标图,我们如何能有效刻画不断演化的域差异,并优化目标域在即将到来的T+1时间戳上的泛化性能?为回答此问题,我们提出了一个图上动态非独立同分布迁移学习的泛化界,该界表明泛化性能主要受域演化以及源图与目标图之间的域差异所主导。受理论结果启发,我们引入了一个名为EvoluNet的新型通用框架。它利用基于Transformer的时间编码模块来建模演化域的时序信息,然后使用动态域统一模块来高效地学习源域和目标域之间的域不变表示。最终,EvoluNet以高达12.1%的优势超越了现有最先进的模型,证明了其在将知识从动态源图迁移到动态目标图方面的有效性。