We propose a new bound for generalization of neural networks using Koopman operators. Unlike most of the existing works, we focus on the role of the final nonlinear transformation of the networks. Our bound is described by the reciprocal of the determinant of the weight matrices and is tighter than existing norm-based bounds when the weight matrices do not have small singular values. According to existing theories about the low-rankness of the weight matrices, it may be counter-intuitive that we focus on the case where singular values of weight matrices are not small. However, motivated by the final nonlinear transformation, we can see that our result sheds light on a new perspective regarding a noise filtering property of neural networks. Since our bound comes from Koopman operators, this work also provides a connection between operator-theoretic analysis and generalization of neural networks. Numerical results support the validity of our theoretical results.

翻译：我们提出了一种利用Koopman算子刻画神经网络泛化性能的新上界。与现有研究不同，本文重点关注网络最终非线性变换的作用。该上界由权重矩阵行列式的倒数表示，当权重矩阵不存在小奇异值时，其紧性优于现有基于范数的上界。根据现有的权重矩阵低秩性理论，关注权重矩阵奇异值非小情况可能违反直觉。但受最终非线性变换的启发，我们的结果揭示了神经网络噪声滤波特性的新视角。由于该上界源于Koopman算子，本工作还建立了算子理论分析与神经网络泛化之间的关联。数值实验验证了理论结果的有效性。