Memory replay based techniques have shown great success for continual learning with incrementally accumulated Euclidean data. Directly applying them to continually expanding graphs, however, leads to the potential memory explosion problem due to the need to buffer representative nodes and their associated topological neighborhood structures. To this end, we systematically analyze the key challenges in the memory explosion problem, and present a general framework, i.e., Parameter Decoupled Graph Neural Networks (PDGNNs) with Topology-aware Embedding Memory (TEM), to tackle this issue. The proposed framework not only reduces the memory space complexity from $\mathcal{O}(nd^L)$ to $\mathcal{O}(n)$~\footnote{$n$: memory budget, $d$: average node degree, $L$: the radius of the GNN receptive field}, but also fully utilizes the topological information for memory replay. Specifically, PDGNNs decouple trainable parameters from the computation ego-subgraph via \textit{Topology-aware Embeddings} (TEs), which compress ego-subgraphs into compact vectors (i.e., TEs) to reduce the memory consumption. Based on this framework, we discover a unique \textit{pseudo-training effect} in continual learning on expanding graphs and this effect motivates us to develop a novel \textit{coverage maximization sampling} strategy that can enhance the performance with a tight memory budget. Thorough empirical studies demonstrate that, by tackling the memory explosion problem and incorporating topological information into memory replay, PDGNNs with TEM significantly outperform state-of-the-art techniques, especially in the challenging class-incremental setting.

翻译：基于记忆回放的技术在处理逐步积累的欧几里得数据的连续学习中取得了显著成功。然而，直接将其应用于不断扩展的图数据时，由于需要缓冲代表性节点及其相关的拓扑邻域结构，可能导致潜在的记忆爆炸问题。为此，我们系统分析了记忆爆炸问题的关键挑战，并提出了一种通用框架——带有拓扑感知嵌入记忆（TEM）的参数解耦图神经网络（PDGNNs）——来解决该问题。该框架不仅将记忆空间复杂度从$\mathcal{O}(nd^L)$降低至$\mathcal{O}(n)$~\footnote{$n$：记忆预算，$d$：节点平均度数，$L$：GNN感受野半径}，还充分利用拓扑信息进行记忆回放。具体而言，PDGNNs通过\textit{拓扑感知嵌入（TEs）}将可训练参数与计算自子图解耦，将自子图压缩为紧凑向量（即TEs）以减少记忆消耗。基于此框架，我们发现在扩展图的连续学习中存在独特的\textit{伪训练效应}，这启发我们开发了一种新颖的\textit{覆盖最大化采样}策略，可在严格记忆预算下提升性能。充分的实证研究表明，通过解决记忆爆炸问题并将拓扑信息融入记忆回放，带有TEM的PDGNNs显著优于最先进技术，尤其在具有挑战性的类增量学习场景中表现突出。