Hardware implementation of neural network are an essential step to implement next generation efficient and powerful artificial intelligence solutions. Besides the realization of a parallel, efficient and scalable hardware architecture, the optimization of the system's extremely large parameter space with sampling-efficient approaches is essential. Here, we analytically derive the scaling laws for highly efficient Coordinate Descent applied to optimizing the readout layer of a random recurrently connection neural network, a reservoir. We demonstrate that the convergence is exponential and scales linear with the network's number of neurons. Our results perfectly reproduce the convergence and scaling of a large-scale photonic reservoir implemented in a proof-of-concept experiment. Our work therefore provides a solid foundation for such optimization in hardware networks, and identifies future directions that are promising for optimizing convergence speed during learning leveraging measures of a neural network's amplitude statistics and the weight update rule.

翻译：神经网络的硬件实现是实现下一代高效且强大的人工智能解决方案的关键步骤。除了实现并行、高效且可扩展的硬件架构外，采用采样高效的方法优化系统极其庞大的参数空间至关重要。本文通过解析推导，获得了针对随机递归连接神经网络（储层）读出层优化的高效坐标下降法的标度律。我们证明其收敛速度呈指数级增长，且与网络神经元数量呈线性扩展关系。我们的结果完美再现了概念验证实验中大规模光子储层的收敛性与扩展性特性。因此，本研究为硬件网络中的此类优化提供了坚实的理论基础，并指明了未来有前景的研究方向——通过利用神经网络的振幅统计特性和权重更新规则来优化学习过程中的收敛速度。