This paper explores the difficulties in solving partial differential equations (PDEs) using physics-informed neural networks (PINNs). PINNs use physics as a regularization term in the objective function. However, a drawback of this approach is the requirement for manual hyperparameter tuning, making it impractical in the absence of validation data or prior knowledge of the solution. Our investigations of the loss landscapes and backpropagated gradients in the presence of physics reveal that existing methods produce non-convex loss landscapes that are hard to navigate. Our findings demonstrate that high-order PDEs contaminate backpropagated gradients and hinder convergence. To address these challenges, we introduce a novel method that bypasses the calculation of high-order derivative operators and mitigates the contamination of backpropagated gradients. Consequently, we reduce the dimension of the search space and make learning PDEs with non-smooth solutions feasible. Our method also provides a mechanism to focus on complex regions of the domain. Besides, we present a dual unconstrained formulation based on Lagrange multiplier method to enforce equality constraints on the model's prediction, with adaptive and independent learning rates inspired by adaptive subgradient methods. We apply our approach to solve various linear and non-linear PDEs.

翻译：本文探讨了使用物理信息神经网络（PINNs）求解偏微分方程（PDEs）时遇到的困难。PINNs将物理定律作为目标函数中的正则化项。然而，这种方法的一个缺陷是需要手动调整超参数，在缺乏验证数据或解的先验知识时难以实际应用。我们对存在物理约束情况下的损失景观和反向传播梯度进行了研究，发现现有方法会产生难以导航的非凸损失景观。我们的结果表明，高阶PDE会污染反向传播梯度并阻碍收敛。为应对这些挑战，我们提出了一种新方法，该方法避免了高阶导数算子的计算，并减轻了反向传播梯度的污染。因此，我们降低了搜索空间的维度，并使学习具有非光滑解的PDE成为可能。我们的方法还提供了一种聚焦于域中复杂区域的机制。此外，我们提出了一种基于拉格朗日乘子法的对偶无约束形式，用于对模型预测施加等式约束，并采用受自适应次梯度方法启发的自适应独立学习率。我们将该方法应用于求解各种线性和非线性PDE。