Physics-informed Neural Networks (PINNs) have recently achieved remarkable progress in solving Partial Differential Equations (PDEs) in various fields by minimizing a weighted sum of PDE loss and boundary loss. However, there are several critical challenges in the training of PINNs, including the lack of theoretical frameworks and the imbalance between PDE loss and boundary loss. In this paper, we present an analysis of second-order non-homogeneous PDEs, which are classified into three categories and applicable to various common problems. We also characterize the connections between the training loss and actual error, guaranteeing convergence under mild conditions. The theoretical analysis inspires us to further propose MultiAdam, a scale-invariant optimizer that leverages gradient momentum to parameter-wisely balance the loss terms. Extensive experiment results on multiple problems from different physical domains demonstrate that our MultiAdam solver can improve the predictive accuracy by 1-2 orders of magnitude compared with strong baselines.

翻译：物理信息神经网络（PINNs）通过最小化偏微分方程损失与边界损失的加权和，近年来在求解各领域偏微分方程方面取得了显著进展。然而，PINNs的训练面临若干关键挑战，包括缺乏理论框架以及偏微分方程损失与边界损失之间的不平衡。本文对二阶非齐次偏微分方程进行了分析，将其分为三类并适用于多种常见问题。我们还刻画了训练损失与实际误差之间的联系，在温和条件下确保了收敛性。该理论分析启发我们进一步提出MultiAdam——一种利用梯度动量在参数级别平衡损失项的尺度不变优化器。针对不同物理领域的多个问题开展的大量实验结果表明，与强基线方法相比，我们的MultiAdam求解器可将预测精度提升1-2个数量级。