In this paper, we establish a sharp upper bound on the the number of fixed points a certain class of neural networks can have. The networks under study (autoencoders) can be viewed as discrete dynamical systems whose nonlinearities are given by the choice of activation functions. To this end, we introduce a new class $\mathcal{F}$ of $C^1$ activation functions that is closed under composition, and contains e.g. the logistic sigmoid function. We use this class to show that any 1-dimensional neural network of arbitrary depth with activation functions in $\mathcal{F}$ has at most three fixed points. Due to the simple nature of such networks, we are able to completely understand their fixed points, providing a foundation to the much needed connection between application and theory of deep neural networks.

翻译：本文中，我们针对某一类神经网络能够拥有的固定点数量建立了严格的上界。所研究的网络（自编码器）可视为离散动力系统，其非线性特性由激活函数的选择决定。为此，我们引入了一类新的$C^1$激活函数类$\mathcal{F}$，该类函数在复合运算下封闭，且包含如逻辑斯蒂克Sigmoid函数等。利用这一函数类，我们证明任意深度的一维神经网络，若其激活函数属于$\mathcal{F}$，则至多拥有三个固定点。由于此类网络结构简单，我们得以完全理解其固定点性质，从而为深度神经网络理论与应用之间亟待建立的关联提供了基础。