Physics-Informed Neural Network (PINN) has become a commonly used machine learning approach to solve partial differential equations (PDE). But, facing high-dimensional secondorder PDE problems, PINN will suffer from severe scalability issues since its loss includes second-order derivatives, the computational cost of which will grow along with the dimension during stacked back-propagation. In this work, we develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. We further discuss the model capacity and provide variance reduction methods to address key limitations in the derivative estimation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is significantly faster. Our code is released at https://github.com/LithiumDA/PINN-without-Stacked-BP.

翻译：物理信息神经网络（PINN）已成为求解偏微分方程（PDE）的常用机器学习方法。然而，面对高维二阶PDE问题，PINN会遭遇严重的可扩展性问题，因为其损失函数包含二阶导数，且堆叠反向传播过程中的计算成本会随维度增加而增长。本研究开发了一种新方法，可显著加速物理信息神经网络的训练。具体而言，我们利用高斯平滑模型参数化PDE解，并证明基于斯坦因恒等式，二阶导数无需反向传播即可高效计算。我们进一步讨论了模型容量，并提出了方差缩减方法以解决导数估计中的关键限制。实验结果表明，与标准PINN训练相比，所提方法在保持精度相当的同时速度显著提升。我们的代码发布在https://github.com/LithiumDA/PINN-without-Stacked-BP。