Theoretical understanding of the behavior of infinitely-wide neural networks has been rapidly developed for various architectures due to the celebrated mean-field theory. However, there is a lack of a clear, intuitive framework for extending our understanding to finite networks that are of more practical and realistic importance. In the present contribution, we demonstrate that the behavior of properly initialized neural networks can be understood in terms of universal critical phenomena in absorbing phase transitions. More specifically, we study the order-to-chaos transition in the fully-connected feedforward neural networks and the convolutional ones to show that (i) there is a well-defined transition from the ordered state to the chaotics state even for the finite networks, and (ii) difference in architecture is reflected in that of the universality class of the transition. Remarkably, the finite-size scaling can also be successfully applied, indicating that intuitive phenomenological argument could lead us to semi-quantitative description of the signal propagation dynamics.

翻译：对无限宽神经网络行为的理论理解，由于著名的平均场理论，已在各种架构中迅速发展。然而，缺乏一个清晰、直观的框架，将我们的理解扩展到更具实际和现实重要性的有限网络。在本文中，我们证明了适当初始化的神经网络的行为可以通过吸收相变中的普遍临界现象来理解。具体而言，我们研究了全连接前馈神经网络和卷积神经网络中的有序-混沌转变，表明：(i) 即使是有限网络，也存在从有序态到混沌态的明确转变，以及(ii) 架构的差异反映在转变的普适类差异上。值得注意的是，有限尺寸标度也可以成功应用，这表明直观的现象学论证可以引导我们对信号传播动力学进行半定量描述。