We propose a new bound for generalization of neural networks using Koopman operators. Whereas most of existing works focus on low-rank weight matrices, we focus on full-rank weight matrices. Our bound is tighter than existing norm-based bounds when the condition numbers of weight matrices are small. Especially, it is completely independent of the width of the network if the weight matrices are orthogonal. Our bound does not contradict to the existing bounds but is a complement to the existing bounds. As supported by several existing empirical results, low-rankness is not the only reason for generalization. Furthermore, our bound can be combined with the existing bounds to obtain a tighter bound. Our result sheds new light on understanding generalization of neural networks with full-rank weight matrices, and it provides a connection between operator-theoretic analysis and generalization of neural networks.

翻译：我们利用Koopman算子提出了神经网络泛化能力的新界限。现有研究大多聚焦于低秩权重矩阵，而本文则关注全秩权重矩阵。当权重矩阵的条件数较小时，我们的界限优于现有的基于范数的界限。特别地，当权重矩阵为正交矩阵时，该界限完全独立于网络宽度。该界限与现有界限并不矛盾，而是对现有界限的补充。多个现有实证结果支持，低秩性并非泛化的唯一原因。此外，我们的界限可与现有界限结合以获得更紧的界。该成果为理解含全秩权重矩阵的神经网络泛化提供了新视角，并建立了算子理论分析与神经网络泛化之间的联系。