Deep neural networks have been shown to provide accurate function approximations in high dimensions. However, fitting network parameters requires informative training data that are often challenging to collect in science and engineering applications. This work proposes Neural Galerkin schemes based on deep learning that generate training data with active learning for numerically solving high-dimensional partial differential equations. Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train networks by minimizing the residual sequentially over time, which enables adaptively collecting new training data in a self-informed manner that is guided by the dynamics described by the partial differential equations. This is in contrast to other machine learning methods that aim to fit network parameters globally in time without taking into account training data acquisition. Our finding is that the active form of gathering training data of the proposed Neural Galerkin schemes is key for numerically realizing the expressive power of networks in high dimensions. Numerical experiments demonstrate that Neural Galerkin schemes have the potential to enable simulating phenomena and processes with many variables for which traditional and other deep-learning-based solvers fail, especially when features of the solutions evolve locally such as in high-dimensional wave propagation problems and interacting particle systems described by Fokker-Planck and kinetic equations.

翻译：深度神经网络在高维函数逼近中展现了高精度，但拟合网络参数需依赖信息丰富的训练数据，这在科学与工程应用中往往难以获取。本文提出基于深度学习的神经加勒金方案，通过主动学习生成训练数据以数值求解高维偏微分方程。该方案基于Dirac-Frenkel变分原理，通过随时间序贯最小化残差来训练网络，从而能够根据偏微分方程描述的动力学过程，以自感知方式自适应地收集新训练数据。这与旨在全局时间拟合网络参数而不考虑训练数据获取的其他机器学习方法形成鲜明对比。研究发现，本文提出的神经加勒金方案主动收集训练数据的机制，是实现网络在高维空间中表达能力数值化的关键。数值实验表明，该方案具有模拟多变量现象与过程的潜力，对于传统及其他基于深度学习的求解器难以处理的问题——尤其当解的特征局部演化时（如高维波动传播问题、由Fokker-Planck方程和动力学方程描述的相互作用粒子系统）——具有显著优势。