Continual learning seeks to empower models to progressively acquire information from a sequence of tasks. This approach is crucial for many real-world systems, which are dynamic and evolve over time. Recent research has witnessed a surge in the exploration of Graph Neural Networks (GNN) in Node-wise Graph Continual Learning (NGCL), a practical yet challenging paradigm involving the continual training of a GNN on node-related tasks. Despite recent advancements in continual learning strategies for GNNs in NGCL, a thorough theoretical understanding, especially regarding its learnability, is lacking. Learnability concerns the existence of a learning algorithm that can produce a good candidate model from the hypothesis/weight space, which is crucial for model selection in NGCL development. This paper introduces the first theoretical exploration of the learnability of GNN in NGCL, revealing that learnability is heavily influenced by structural shifts due to the interconnected nature of graph data. Specifically, GNNs may not be viable for NGCL under significant structural changes, emphasizing the need to manage structural shifts. To mitigate the impact of structural shifts, we propose a novel experience replay method termed Structure-Evolution-Aware Experience Replay (SEA-ER). SEA-ER features an innovative experience selection strategy that capitalizes on the topological awareness of GNNs, alongside a unique replay strategy that employs structural alignment to effectively counter catastrophic forgetting and diminish the impact of structural shifts on GNNs in NGCL. Our extensive experiments validate our theoretical insights and the effectiveness of SEA-ER.

翻译：持续学习旨在使模型能够从一系列任务中逐步获取信息。这种方法对于许多动态且随时间演变的现实世界系统至关重要。近期研究见证了图神经网络在节点级图持续学习中的探索热潮，这是一种实用但具有挑战性的范式，涉及在图节点相关任务上对GNN进行持续训练。尽管GNN在NGCL中的持续学习策略已取得进展，但对其可学习性等关键理论问题仍缺乏深入理解。可学习性关注的是是否存在一种学习算法能够从假设/权重空间中产生良好的候选模型，这对于NGCL开发中的模型选择至关重要。本文首次对GNN在NGCL中的可学习性进行理论探索，揭示了由于图数据的互联特性，结构偏移会严重影响可学习性。具体而言，在显著的结构变化下，GNN可能无法适用于NGCL，这凸显了管理结构偏移的必要性。为减轻结构偏移的影响，我们提出了一种新颖的经验回放方法——结构演化感知经验回放。SEA-ER采用创新的经验选择策略，充分利用GNN的拓扑感知能力，并结合独特的回放策略，通过结构对齐有效缓解灾难性遗忘，降低结构偏移对NGCL中GNN的影响。大量实验结果验证了我们的理论见解与SEA-ER的有效性。