While physics-informed neural networks (PINNs) have become a popular deep learning framework for tackling forward and inverse problems governed by partial differential equations (PDEs), their performance is known to degrade when larger and deeper neural network architectures are employed. Our study identifies that the root of this counter-intuitive behavior lies in the use of multi-layer perceptron (MLP) architectures with non-suitable initialization schemes, which result in poor trainablity for the network derivatives, and ultimately lead to an unstable minimization of the PDE residual loss. To address this, we introduce Physics-informed Residual Adaptive Networks (PirateNets), a novel architecture that is designed to facilitate stable and efficient training of deep PINN models. PirateNets leverage a novel adaptive residual connection, which allows the networks to be initialized as shallow networks that progressively deepen during training. We also show that the proposed initialization scheme allows us to encode appropriate inductive biases corresponding to a given PDE system into the network architecture. We provide comprehensive empirical evidence showing that PirateNets are easier to optimize and can gain accuracy from considerably increased depth, ultimately achieving state-of-the-art results across various benchmarks. All code and data accompanying this manuscript will be made publicly available at \url{https://github.com/PredictiveIntelligenceLab/jaxpi}.

翻译：尽管物理信息神经网络（PINNs）已成为求解偏微分方程（PDE）正反问题的主流深度学习框架，但研究表明其在采用更大更深网络架构时性能会下降。我们的研究发现，这种反直觉行为的根源在于使用非合适初始化策略的多层感知机（MLP）架构，导致网络导数的可训练性较差，最终引发PDE残差损失的不稳定最小化。为解决此问题，我们提出物理信息残差自适应网络（PirateNets）——一种旨在促进深度PINN模型稳定高效训练的新型架构。PirateNets利用创新的自适应残差连接，使网络初始化为浅层网络，并在训练过程中逐步加深。我们还证明，所提出的初始化方案允许将对应给定PDE系统的适当归纳偏置编码到网络架构中。我们提供的全面实验证据表明，PirateNets更易优化，并能通过显著增加深度提升精度，最终在多个基准测试中取得最先进成果。本文所有代码和数据将在\url{https://github.com/PredictiveIntelligenceLab/jaxpi}公开。