The spatiotemporal resolution of Partial Differential Equations (PDEs) plays important roles in the mathematical description of the world's physical phenomena. In general, scientists and engineers solve PDEs numerically by the use of computationally demanding solvers. Recently, deep learning algorithms have emerged as a viable alternative for obtaining fast solutions for PDEs. Models are usually trained on synthetic data generated by solvers, stored on disk and read back for training. This paper advocates that relying on a traditional static dataset to train these models does not allow the full benefit of the solver to be used as a data generator. It proposes an open source online training framework for deep surrogate models. The framework implements several levels of parallelism focused on simultaneously generating numerical simulations and training deep neural networks. This approach suppresses the I/O and storage bottleneck associated with disk-loaded datasets, and opens the way to training on significantly larger datasets. Experiments compare the offline and online training of four surrogate models, including state-of-the-art architectures. Results indicate that exposing deep surrogate models to more dataset diversity, up to hundreds of GB, can increase model generalization capabilities. Fully connected neural networks, Fourier Neural Operator (FNO), and Message Passing PDE Solver prediction accuracy is improved by 68%, 16% and 7%, respectively.

翻译：偏微分方程（PDEs）的时空分辨率对描述世界物理现象的数学建模具有重要作用。通常，科学家和工程师通过计算密集型求解器对偏微分方程进行数值求解。近年来，深度学习算法已成为获取偏微分方程快速求解方案的有效替代方案。模型通常基于求解器生成的合成数据进行训练，这些数据存储于磁盘中并回读用于训练。本文主张，依赖传统静态数据集训练此类模型，无法充分利用求解器作为数据生成器的潜力。为此，我们提出一个面向深度代理模型的开源在线训练框架。该框架实现了多级并行机制，能够同时生成数值模拟结果并训练深度神经网络。该方法消除了磁盘加载数据集带来的输入/输出与存储瓶颈，为在更大规模数据集上训练开辟了新途径。实验对比了四种代理模型（包括最先进架构）的离线与在线训练效果。结果表明，让深度代理模型接触更大数据多样性（高达数百GB）可提升模型泛化能力。全连接神经网络、傅里叶神经算子（FNO）及消息传递PDE求解器的预测精度分别提升68%、16%和7%。