In this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs). Our key result is that the difficulty in training these models is closely related to the conditioning of a specific differential operator. This operator, in turn, is associated to the Hermitian square of the differential operator of the underlying PDE. If this operator is ill-conditioned, it results in slow or infeasible training. Therefore, preconditioning this operator is crucial. We employ both rigorous mathematical analysis and empirical evaluations to investigate various strategies, explaining how they better condition this critical operator, and consequently improve training.

翻译：本文研究了物理信息机器学习方法（如PINNs）中梯度下降算法的行为，这类方法通过最小化与偏微分方程相关的残差进行求解。我们的核心发现是：此类模型训练困难的程度与特定微分算子的条件数密切相关。该算子本质上是底层偏微分方程微分算子的埃尔米特平方算子。若该算子呈现病态，则会导致训练缓慢甚至无法收敛。因此，对此算子进行预条件处理至关重要。我们通过严谨的数学分析与实证评估相结合的方式，系统研究了多种预条件策略，揭示了它们如何改善这一关键算子的条件数，从而提升训练效果。