We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.

翻译：我们提出增强物理信息神经网络（PINNs）训练的方法。为此，我们针对广泛使用的L-BFGS优化器引入了非线性加性和乘性预处理策略。这些非线性预处理器基于Schwarz区域分解框架构建，其中网络参数以分层方式分解。通过一系列数值实验，我们证明加性和乘性预处理器均能显著改进标准L-BFGS优化器的收敛性能，同时提供更准确的底层偏微分方程解。此外，加性预处理器具有天然并行性，因此催生了一种新型模型并行方法。