Many engineering and scientific fields have recently become interested in modeling terms in partial differential equations (PDEs) with neural networks, which requires solving the inverse problem of learning neural network terms from observed data in order to approximate missing or unresolved physics in the PDE model. The resulting neural-network PDE model, being a function of the neural network parameters, can be calibrated to the available ground truth data by optimizing over the PDE using gradient descent, where the gradient is evaluated in a computationally efficient manner by solving an adjoint PDE. These neural PDE models have emerged as an important research area in scientific machine learning. In this paper, we study the convergence of the adjoint gradient descent optimization method for training neural PDE models in the limit where both the number of hidden units and the training time tend to infinity. Specifically, for a general class of nonlinear parabolic PDEs with a neural network embedded in the source term, we prove convergence of the trained neural-network PDE solution to the target data (i.e., a global minimizer). The global convergence proof poses a unique mathematical challenge that is not encountered in finite-dimensional neural network convergence analyses due to (i) the neural network training dynamics involving a non-local neural network kernel operator in the infinite-width hidden layer limit where the kernel lacks a spectral gap for its eigenvalues and (ii) the nonlinearity of the limit PDE system, which leads to a non-convex optimization problem in the neural network function even in the infinite-width hidden layer limit (unlike in typical neural network training cases where the optimization problem becomes convex in the large neuron limit). The theoretical results are illustrated and empirically validated by numerical studies.

翻译：近年来，许多工程与科学领域对使用神经网络建模偏微分方程（PDE）中的项产生了浓厚兴趣，这需要解决从观测数据中学习神经网络项的反问题，以近似PDE模型中缺失或未解析的物理过程。由此得到的神经网络PDE模型作为神经网络参数的函数，可通过基于梯度下降的PDE优化来校准可用的真实数据，其中梯度通过求解伴随PDE以计算高效的方式评估。这类神经PDE模型已成为科学机器学习中的重要研究方向。本文研究了在隐藏单元数量和训练时间均趋于无穷的极限下，用于训练神经PDE模型的伴随梯度下降优化方法的收敛性。具体而言，针对源项中嵌入神经网络的一类非线性抛物型PDE，我们证明了训练后的神经网络PDE解收敛于目标数据（即全局极小值）。全局收敛性证明面临独特的数学挑战，这些挑战在有限维神经网络收敛分析中未曾出现，原因在于：（i）神经网络训练动力学在无限宽度隐藏层极限下涉及非局部神经网络核算子，该核算子的特征值缺乏谱间隙；（ii）极限PDE系统的非线性特性导致神经网络函数上的优化问题即使在无限宽度隐藏层极限下仍为非凸问题（这与典型神经网络训练中优化问题在大神经元极限下变为凸的情形不同）。数值研究对理论结果进行了阐释与实证验证。