Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by blending seamlessly data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing Hutchinson Trace Estimation (HTE). Starting with the second-order high-dimensional PDEs ubiquitous in scientific computing, HTE transforms the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach alleviates the computational bottleneck via Taylor-mode automatic differentiation and significantly reduces memory consumption from the Hessian matrix to HVP. We further showcase HTE's convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variance among dimensions. We further extend HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

翻译：物理信息神经网络在处理偏微分方程方面已被证明是有效的，尤其是在能够无缝融合数据与物理信息的情况下。然而，将物理信息神经网络扩展到高维甚至高阶偏微分方程时，由于残差损失中自动微分带来的计算成本，面临着重大挑战。本文通过引入Hutchinson迹估计来解决物理信息神经网络在处理高维和高阶偏微分方程时的局限性。从科学计算中普遍存在的二阶高维偏微分方程入手，Hutchinson迹估计将整个海森矩阵的计算转化为海森向量积。该方法通过泰勒模式自动微分缓解了计算瓶颈，并将内存消耗从海森矩阵显著降低至海森向量积。我们进一步展示了Hutchinson迹估计向原始物理信息神经网络损失的收敛性及其在特定条件下的无偏特性。与随机维度梯度下降法的比较突显了Hutchinson迹估计的独特优势，尤其是在维度间方差显著的情境下。我们还将Hutchinson迹估计扩展到更高阶和更高维的偏微分方程，特别是针对双调和方程。通过使用张量-向量积，Hutchinson迹估计高效地计算了与四阶高维双调和方程相关的巨大张量，从而节省了内存并实现了快速计算。实验设置验证了Hutchinson迹估计的有效性，表明其在内存和速度限制下与随机维度梯度下降法具有可比的收敛速度。此外，Hutchinson迹估计在加速梯度增强物理信息神经网络版本以及双调和方程方面也显示出价值。总体而言，Hutchinson迹估计为科学机器学习领域处理高阶和高维偏微分方程开辟了新的能力。