Natural Gradient Descent (NGD) has emerged as a promising optimization algorithm for training neural network-based solvers for partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs). However, its practical use is often limited by the high computational cost of solving linear systems involving the Gramian matrix. While matrix-free NGD methods based on the conjugate gradient (CG) method avoid explicit matrix inversion, the ill-conditioning of the Gramian significantly slows the convergence of CG. In this work, we extend matrix-free NGD to broader classes of problems than previously considered and propose the use of Randomized Nystr\"om preconditioning to accelerate convergence of the inner CG solver. The resulting algorithm demonstrates substantial performance improvements over existing NGD-based methods and other state-of-the-art optimizers on a range of PDE problems discretized using neural networks.

翻译：自然梯度下降（NGD）已成为训练基于神经网络的偏微分方程（PDE）求解器（如物理信息神经网络（PINNs））的一种前景广阔的优化算法。然而，其实际应用常受限于求解涉及格拉姆矩阵线性系统的高计算成本。虽然基于共轭梯度（CG）法的无矩阵NGD方法避免了显式矩阵求逆，但格拉姆矩阵的病态性显著降低了CG法的收敛速度。在本工作中，我们将无矩阵NGD扩展到比先前研究更广泛的问题类别，并提出采用随机化Nyström预处理来加速内部CG求解器的收敛。所得算法在一系列使用神经网络离散化的PDE问题上，相较于现有基于NGD的方法及其他最先进的优化器，均展现出显著的性能提升。