We consider the existence of fixed points of nonnegative neural networks, i.e., neural networks that take as an input and produce as an output nonnegative vectors. We first show that nonnegative neural networks with nonnegative weights and biases can be recognized as monotonic and (weakly) scalable functions within the framework of nonlinear Perron-Frobenius theory. This fact enables us to provide conditions for the existence of fixed points of nonnegative neural networks, and these conditions are weaker than those obtained recently using arguments in convex analysis. Furthermore, we prove that the shape of the fixed point set of nonnegative neural networks with nonnegative weights and biases is an interval, which under mild conditions degenerates to a point. These results are then used to obtain the existence of fixed points of more general types of nonnegative neural networks. The results of this paper contribute to the understanding of the behavior of autoencoders, and they provide insight into neural networks designed using the loop-unrolling technique, which can be seen as a fixed point searching algorithm. The chief theoretical results of this paper are verified in numerical simulations.
翻译:我们考虑非负神经网络(即输入和输出均为非负向量的神经网络)的固定点的存在性问题。首先,我们证明具有非负权重和偏置的非负神经网络在非线性Perron-Frobenius理论框架下可被视为单调且(弱)可缩放的函数。这一事实使我们能够为非负神经网络固定点的存在性提供条件,且这些条件比近期利用凸分析论证所得的条件更弱。此外,我们证明具有非负权重和偏置的非负神经网络的固定点集形状为区间,该区间在温和条件下退化为单点。随后,这些结果被用于获得更一般类型的非负神经网络固定点的存在性。本文的结论有助于理解自编码器的行为,并为采用循环展开技术设计的神经网络(可视为固定点搜索算法)提供理论洞见。本文的主要理论结果通过数值模拟得到验证。