Neural Persistence is a prominent measure for quantifying neural network complexity, proposed in the emerging field of topological data analysis in deep learning. In this work, however, we find both theoretically and empirically that the variance of network weights and spatial concentration of large weights are the main factors that impact neural persistence. Whilst this captures useful information for linear classifiers, we find that no relevant spatial structure is present in later layers of deep neural networks, making neural persistence roughly equivalent to the variance of weights. Additionally, the proposed averaging procedure across layers for deep neural networks does not consider interaction between layers. Based on our analysis, we propose an extension of the filtration underlying neural persistence to the whole neural network instead of single layers, which is equivalent to calculating neural persistence on one particular matrix. This yields our deep graph persistence measure, which implicitly incorporates persistent paths through the network and alleviates variance-related issues through standardisation. Code is available at https://github.com/ExplainableML/Deep-Graph-Persistence .

翻译：神经持久性是衡量神经网络复杂性的重要指标，源于深度学习领域新兴的拓扑数据分析方法。然而，本研究发现，网络权重的方差和大权重的空间集中性是影响神经持久性的主要因素，这在理论和实验上均已得到验证。尽管该指标对线性分类器具有参考价值，但深度神经网络后期层中不存在显著的空间结构，导致神经持久性近似等价于权重的方差。此外，现有针对深度神经网络提出的跨层平均化方法未考虑层间相互作用。基于上述分析，我们提出将神经持久性所依赖的过滤机制从单层扩展至整个神经网络，这等价于在特定矩阵上计算神经持久性。由此得到的深度图持久性度量方法，能够隐式整合网络中的持久路径，并通过标准化消除方差相关问题。代码开源地址：https://github.com/ExplainableML/Deep-Graph-Persistence