Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis. By framing neural networks as operators with fixed points representing desired solutions, we develop a theoretical framework grounded in iterative methods for operator equations. Under defined conditions, we present convergence proofs based on fixed point theory. We demonstrate that popular architectures, such as diffusion models and AlphaFold, inherently employ iterative operator learning. Empirical assessments highlight that performing iterations through network operators improves performance. We also introduce an iterative graph neural network, PIGN, that further demonstrates benefits of iterations. Our work aims to enhance the understanding of deep learning by merging insights from numerical analysis, potentially guiding the design of future networks with clearer theoretical underpinnings and improved performance.

翻译：深度神经网络虽在众多应用中取得成功，但其运行往往缺乏坚实的理论基础。本文通过建立深度学习与经典数值分析之间的平行关系来弥合这一差距。通过将神经网络视为以固定点为期望解的算子，我们构建了一个基于算子方程迭代方法的理论框架。在既定条件下，我们基于不动点理论给出了收敛性证明。我们证明了扩散模型和AlphaFold等主流架构本质上采用了迭代算子学习。实证评估表明，通过网络算子进行迭代能够提升性能。我们还提出了一种迭代图神经网络PIGN，进一步展示了迭代的优势。本研究旨在通过融合数值分析的观点来深化对深度学习的理解，有望为未来网络设计提供更清晰的理论依据并提升其性能。