We introduce Fiedler regularization, a novel approach for regularizing neural networks that utilizes spectral/graphical information. Existing regularization methods often focus on penalizing weights in a global/uniform manner that ignores the connectivity structure of the neural network. We propose to use the Fiedler value of the neural network's underlying graph as a tool for regularization. We provide theoretical motivation for this approach via spectral graph theory. We demonstrate several useful properties of the Fiedler value that make it useful as a regularization tool. We provide an approximate, variational approach for faster computation during training. We provide an alternative formulation of this framework in the form of a structurally weighted $\text{L}_1$ penalty, thus linking our approach to sparsity induction. We provide uniform generalization error bounds for Fiedler regularization via a Rademacher complexity analysis. We performed experiments on datasets that compare Fiedler regularization with classical regularization methods such as dropout and weight decay. Results demonstrate the efficacy of Fiedler regularization. This is a journal extension of the conference paper by Tam and Dunson (2020).

翻译：我们提出Fiedler正则化，一种利用谱/图信息对神经网络进行正则化的新方法。现有正则化方法通常以忽略神经网络连接结构的全局/统一方式惩罚权重。我们建议将神经网络底层图的Fiedler值用作正则化工具。通过谱图理论，我们为该方法的理论动机提供了依据。我们展示了Fiedler值的若干有用性质，使其成为有效的正则化工具。我们提出了一种近似的变分方法，以在训练过程中实现更快计算。我们给出了该框架的另一种形式，即结构加权的$\text{L}_1$惩罚项，从而将我们的方法与稀疏性诱导联系起来。通过Rademacher复杂度分析，我们为Fiedler正则化提供了统一的泛化误差界。我们在数据集上进行了实验，将Fiedler正则化与经典正则化方法（如dropout和权重衰减）进行比较。结果证明了Fiedler正则化的有效性。本文是Tam和Dunson（2020）会议论文的期刊扩展版。