Unlike conventional grid and mesh based methods for solving partial differential equations (PDEs), neural networks have the potential to break the curse of dimensionality, providing approximate solutions to problems where using classical solvers is difficult or impossible. While global minimization of the PDE residual over the network parameters works well for boundary value problems, catastrophic forgetting impairs the applicability of this approach to initial value problems (IVPs). In an alternative local-in-time approach, the optimization problem can be converted into an ordinary differential equation (ODE) on the network parameters and the solution propagated forward in time; however, we demonstrate that current methods based on this approach suffer from two key issues. First, following the ODE produces an uncontrolled growth in the conditioning of the problem, ultimately leading to unacceptably large numerical errors. Second, as the ODE methods scale cubically with the number of model parameters, they are restricted to small neural networks, significantly limiting their ability to represent intricate PDE initial conditions and solutions. Building on these insights, we develop Neural IVP, an ODE based IVP solver which prevents the network from getting ill-conditioned and runs in time linear in the number of parameters, enabling us to evolve the dynamics of challenging PDEs with neural networks.

翻译：不同于基于网格的传统偏微分方程求解方法，神经网络具有突破维度诅咒的潜力，可为使用经典求解器难以或无法求解的问题提供近似解。虽然对网络参数进行全局最小化PDE残差的方法适用于边值问题，但灾难性遗忘限制了该方法在初值问题(IVP)中的应用。在另一种局部时间推进方法中，优化问题可转化为网络参数上的常微分方程(ODE)，并随时间向前传播解；然而，我们证明了当前基于该方法的方案存在两个关键问题。其一，沿ODE推进会导致问题条件数不受控增长，最终产生不可接受的大数值误差。其二，由于ODE方法随模型参数数量呈三次方规模增长，其应用仅限于小型神经网络，显著限制了网络对复杂PDE初始条件及解的表示能力。基于上述发现，我们开发了Neural IVP——一种基于ODE的IVP求解器，该求解器可防止网络病态化且运行时间与参数数量呈线性关系，使我们能够用神经网络演化具有挑战性的PDE动力学过程。