Graph Lottery Ticket (GLT), a combination of core subgraph and sparse subnetwork, has been proposed to mitigate the computational cost of deep Graph Neural Networks (GNNs) on large input graphs while preserving original performance. However, the winning GLTs in exisiting studies are obtained by applying iterative magnitude-based pruning (IMP) without re-evaluating and re-considering the pruned information, which disregards the dynamic changes in the significance of edges/weights during graph/model structure pruning, and thus limits the appeal of the winning tickets. In this paper, we formulate a conjecture, i.e., existing overlooked valuable information in the pruned graph connections and model parameters which can be re-grouped into GLT to enhance the final performance. Specifically, we propose an adversarial complementary erasing (ACE) framework to explore the valuable information from the pruned components, thereby developing a more powerful GLT, referred to as the ACE-GLT. The main idea is to mine valuable information from pruned edges/weights after each round of IMP, and employ the ACE technique to refine the GLT processing. Finally, experimental results demonstrate that our ACE-GLT outperforms existing methods for searching GLT in diverse tasks. Our code will be made publicly available.
翻译:图彩票(GLT)作为一种核心子图与稀疏子网络的组合,已被提出用于在保留原始性能的同时,降低深度图神经网络(GNN)处理大规模输入图的计算成本。然而,现有研究中的获胜GLT是通过迭代幅度剪枝(IMP)获得的,该方法未重新评估和重新考虑被剪枝的信息,忽视了图/模型结构剪枝过程中边/权重的动态重要性变化,从而限制了获胜彩票的吸引力。本文提出一个猜想:被剪枝的图连接和模型参数中存在被忽视的有价值信息,这些信息可以重新整合到GLT中,以提升最终性能。具体而言,我们提出了一种对抗互补擦除(ACE)框架,用于从被剪枝组件中探索有价值信息,从而开发出更强大的GLT,称为ACE-GLT。其核心思想是在每轮IMP后从被剪枝的边/权中挖掘有价值信息,并利用ACE技术优化GLT处理过程。最终,实验结果表明,我们的ACE-GLT在多种任务中优于现有搜索GLT的方法。我们的代码将公开发布。