Graph neural networks (GNNs) have become increasingly popular for classification tasks on graph-structured data. Yet, the interplay between graph topology and feature evolution in GNNs is not well understood. In this paper, we focus on node-wise classification, illustrated with community detection on stochastic block model graphs, and explore the feature evolution through the lens of the "Neural Collapse" (NC) phenomenon. When training instance-wise deep classifiers (e.g. for image classification) beyond the zero training error point, NC demonstrates a reduction in the deepest features' within-class variability and an increased alignment of their class means to certain symmetric structures. We start with an empirical study that shows that a decrease in within-class variability is also prevalent in the node-wise classification setting, however, not to the extent observed in the instance-wise case. Then, we theoretically study this distinction. Specifically, we show that even an "optimistic" mathematical model requires that the graphs obey a strict structural condition in order to possess a minimizer with exact collapse. Interestingly, this condition is viable also for heterophilic graphs and relates to recent empirical studies on settings with improved GNNs' generalization. Furthermore, by studying the gradient dynamics of the theoretical model, we provide reasoning for the partial collapse observed empirically. Finally, we present a study on the evolution of within- and between-class feature variability across layers of a well-trained GNN and contrast the behavior with spectral methods.

翻译：图神经网络（GNN）在面向图结构数据的分类任务中日益普及。然而，图拓扑结构与GNN中特征演化之间的相互作用尚不明确。本文聚焦于节点级分类，以随机块模型图上的社区检测为例，通过“神经坍缩”（NC）现象审视特征演化。当训练实例级深度分类器（如图像分类）超过零训练误差点时，NC表现为最深层特征的类内方差降低及其类均值向特定对称结构的对齐程度提升。我们首先通过实证研究发现：在节点级分类场景中同样普遍存在类内方差降低现象，但其幅度不及实例级分类情形。随后，我们从理论上探究这一差异：具体而言，即使采用“乐观”数学模型，也需要图满足严格的结构条件才能拥有具备完全坍缩特性的最小化解。有趣的是，该条件在异配图中亦适用，且与近期关于提升GNN泛化能力的实证研究相关联。此外，通过分析理论模型的梯度动力学，我们为实证观测到的部分坍缩现象提供了推理解释。最后，我们研究了训练良好的GNN各层中类内与类间特征方差的演化规律，并将其与谱方法的行为进行对比。