Unlike conventional grid and mesh based methods for solving partial differential equations (PDEs), neural networks have the potential to break the curse of dimensionality, providing approximate solutions to problems where using classical solvers is difficult or impossible. While global minimization of the PDE residual over the network parameters works well for boundary value problems, catastrophic forgetting impairs the applicability of this approach to initial value problems (IVPs). In an alternative local-in-time approach, the optimization problem can be converted into an ordinary differential equation (ODE) on the network parameters and the solution propagated forward in time; however, we demonstrate that current methods based on this approach suffer from two key issues. First, following the ODE produces an uncontrolled growth in the conditioning of the problem, ultimately leading to unacceptably large numerical errors. Second, as the ODE methods scale cubically with the number of model parameters, they are restricted to small neural networks, significantly limiting their ability to represent intricate PDE initial conditions and solutions. Building on these insights, we develop Neural IVP, an ODE based IVP solver which prevents the network from getting ill-conditioned and runs in time linear in the number of parameters, enabling us to evolve the dynamics of challenging PDEs with neural networks.

翻译：不同于传统的基于网格和格点求解偏微分方程的方法，神经网络具有突破维度灾难的潜力，能够为使用经典求解器难以或无法解决的问题提供近似解。虽然对网络参数进行全局最小化偏微分方程残差的方法适用于边值问题，但灾难性遗忘削弱了该方法在初值问题上的适用性。在另一种局部时间方法中，优化问题可转化为网络参数上的常微分方程，并随时间向前传播解；然而，我们证明当前基于此类方法的技术存在两个关键问题。第一，沿常微分方程跟踪会导致问题条件数不可控增长，最终产生不可接受的数值误差。第二，由于常微分方程方法随模型参数数量呈三次方扩展，它们仅限于小型神经网络，显著限制了其表征复杂偏微分方程初值条件和解的能力。基于这些发现，我们开发了Neural IVP——一种基于常微分方程的初值问题求解器，其既能防止网络陷入病态，又能实现与参数数量线性相关的时间复杂度，从而能够用神经网络演化具有挑战性的偏微分方程动力学过程。