This paper explores the difficulties in solving partial differential equations (PDEs) using physics-informed neural networks (PINNs). PINNs use physics as a regularization term in the objective function. However, a drawback of this approach is the requirement for manual hyperparameter tuning, making it impractical in the absence of validation data or prior knowledge of the solution. Our investigations of the loss landscapes and backpropagated gradients in the presence of physics reveal that existing methods produce non-convex loss landscapes that are hard to navigate. Our findings demonstrate that high-order PDEs contaminate backpropagated gradients and hinder convergence. To address these challenges, we introduce a novel method that bypasses the calculation of high-order derivative operators and mitigates the contamination of backpropagated gradients. Consequently, we reduce the dimension of the search space and make learning PDEs with non-smooth solutions feasible. Our method also provides a mechanism to focus on complex regions of the domain. Besides, we present a dual unconstrained formulation based on Lagrange multiplier method to enforce equality constraints on the model's prediction, with adaptive and independent learning rates inspired by adaptive subgradient methods. We apply our approach to solve various linear and non-linear PDEs.

翻译：本文探讨了使用物理信息神经网络（PINNs）求解偏微分方程（PDEs）时遇到的困难。PINNs将物理规律作为目标函数中的正则化项。然而，该方法的一个缺点是需要手动调整超参数，这使得它在缺乏验证数据或先验解知识的情况下难以实际应用。我们对损失景观和反向传播梯度在有物理约束情况下的研究显示，现有方法产生的非凸损失景观难以导航。我们的发现表明，高阶PDE会污染反向传播梯度并阻碍收敛。为应对这些挑战，我们提出了一种新方法，该方法避免计算高阶导数算子，从而减轻反向传播梯度的污染。因此，我们降低了搜索空间的维数，并使学习具有非光滑解的PDE成为可能。我们的方法还提供了一种机制，以聚焦于域中的复杂区域。此外，我们基于拉格朗日乘子法提出了一种对偶无约束公式，用于对模型预测施加等式约束，并采用受自适应次梯度方法启发的自适应独立学习率。我们将该方法应用于求解各种线性和非线性PDE。