Physics-informed neural networks (PINNs) have emerged as a prominent approach for solving partial differential equations (PDEs) by minimizing a combined loss function that incorporates both boundary loss and PDE residual loss. Despite their remarkable empirical performance in various scientific computing tasks, PINNs often fail to generate reasonable solutions, and such pathological behaviors remain difficult to explain and resolve. In this paper, we identify that PINNs can be adversely trained when gradients of each loss function exhibit a significant imbalance in their magnitudes and present a negative inner product value. To address these issues, we propose a novel optimization framework, Dual Cone Gradient Descent (DCGD), which adjusts the direction of the updated gradient to ensure it falls within a dual cone region. This region is defined as a set of vectors where the inner products with both the gradients of the PDE residual loss and the boundary loss are non-negative. Theoretically, we analyze the convergence properties of DCGD algorithms in a non-convex setting. On a variety of benchmark equations, we demonstrate that DCGD outperforms other optimization algorithms in terms of various evaluation metrics. In particular, DCGD achieves superior predictive accuracy and enhances the stability of training for failure modes of PINNs and complex PDEs, compared to existing optimally tuned models. Moreover, DCGD can be further improved by combining it with popular strategies for PINNs, including learning rate annealing and the Neural Tangent Kernel (NTK).

翻译：物理信息神经网络（PINNs）已成为求解偏微分方程（PDEs）的一种主流方法，其通过最小化一个结合边界损失与PDE残差损失的复合损失函数来实现。尽管在各种科学计算任务中展现出卓越的实证性能，PINNs 常常无法生成合理的解，此类病态行为仍难以解释和解决。本文发现，当各损失函数的梯度在量级上呈现显著不平衡且具有负内积值时，PINNs 可能受到不利的训练影响。为解决这些问题，我们提出了一种新颖的优化框架——双锥梯度下降法（DCGD），该方法通过调整更新梯度的方向，确保其落入一个双锥区域。该区域被定义为一组向量，其与PDE残差损失梯度和边界损失梯度的内积均为非负。理论上，我们在非凸设定下分析了DCGD算法的收敛性质。在一系列基准方程上的实验表明，DCGD在多项评估指标上均优于其他优化算法。特别是，与现有经优化调参的模型相比，DCGD在PINNs的失效模式及复杂PDEs的训练中实现了更优的预测精度并增强了训练稳定性。此外，DCGD可通过结合学习率退火与神经正切核（NTK）等PINNs常用策略得到进一步改进。