We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.

翻译：我们提出了一种提升物理信息神经网络（PINNs）训练效果的方法。为此，针对广泛使用的L-BFGS优化器，我们引入了非线性的加性和乘性预条件策略。该非线性预条件器基于Schwarz区域分解框架构建，其中网络参数以逐层方式进行分解。通过一系列数值实验，我们证明加性和乘性预条件器均能显著改善标准L-BFGS优化器的收敛性能，同时为底层偏微分方程提供更精确的解。此外，加性预条件器具有内在并行性，从而开创了一种新颖的模型并行方法。