Graph neural networks (GNNs) have become increasingly popular for classification tasks on graph-structured data. Yet, the interplay between graph topology and feature evolution in GNNs is not well understood. In this paper, we focus on node-wise classification, illustrated with community detection on stochastic block model graphs, and explore the feature evolution through the lens of the "Neural Collapse" (NC) phenomenon. When training instance-wise deep classifiers (e.g. for image classification) beyond the zero training error point, NC demonstrates a reduction in the deepest features' within-class variability and an increased alignment of their class means to certain symmetric structures. We start with an empirical study that shows that a decrease in within-class variability is also prevalent in the node-wise classification setting, however, not to the extent observed in the instance-wise case. Then, we theoretically study this distinction. Specifically, we show that even an "optimistic" mathematical model requires that the graphs obey a strict structural condition in order to possess a minimizer with exact collapse. Interestingly, this condition is viable also for heterophilic graphs and relates to recent empirical studies on settings with improved GNNs' generalization. Furthermore, by studying the gradient dynamics of the theoretical model, we provide reasoning for the partial collapse observed empirically. Finally, we present a study on the evolution of within- and between-class feature variability across layers of a well-trained GNN and contrast the behavior with spectral methods.

翻译：图神经网络（GNNs）已广泛应用于图结构数据的分类任务。然而，图拓扑结构与GNN中特征演化之间的相互作用尚未得到充分理解。本文聚焦节点分类任务，以随机块模型图上的社区检测为例，通过"神经坍缩"（NC）现象视角探索特征演化规律。当训练实例级深度分类器（如图像分类）超过零训练误差点时，NC现象表明：最深层特征的类内变异性降低，且其类均值与特定对称结构的对齐性增强。我们首先通过实证研究发现：在节点分类场景中，类内变异性的降低同样普遍存在，但其程度不及实例分类场景。随后，我们从理论上分析了这一差异——具体而言，即使采用"乐观"数学建模，图结构仍需满足严苛的约束条件才能存在精确坍缩的极小值点。有趣的是，该条件对异配图同样成立，且与近期关于提升GNN泛化能力的实证研究存在关联。此外，通过分析理论模型的梯度动力学，我们为实证观察到的部分坍缩现象提供了理论解释。最后，我们研究了训练完备的GNN各层中类内与类间特征变异性的演化规律，并将其与谱方法进行对比分析。