While physics-informed neural networks (PINNs) have become a popular deep learning framework for tackling forward and inverse problems governed by partial differential equations (PDEs), their performance is known to degrade when larger and deeper neural network architectures are employed. Our study identifies that the root of this counter-intuitive behavior lies in the use of multi-layer perceptron (MLP) architectures with non-suitable initialization schemes, which result in poor trainablity for the network derivatives, and ultimately lead to an unstable minimization of the PDE residual loss. To address this, we introduce Physics-informed Residual Adaptive Networks (PirateNets), a novel architecture that is designed to facilitate stable and efficient training of deep PINN models. PirateNets leverage a novel adaptive residual connection, which allows the networks to be initialized as shallow networks that progressively deepen during training. We also show that the proposed initialization scheme allows us to encode appropriate inductive biases corresponding to a given PDE system into the network architecture. We provide comprehensive empirical evidence showing that PirateNets are easier to optimize and can gain accuracy from considerably increased depth, ultimately achieving state-of-the-art results across various benchmarks. All code and data accompanying this manuscript will be made publicly available at \url{https://github.com/PredictiveIntelligenceLab/jaxpi}.

翻译：尽管物理信息神经网络（PINNs）已成为求解偏微分方程（PDEs）正问题和反问题的热门深度学习框架，但已知其性能会随着更大更深层神经网络架构的使用而下降。本研究发现，这种反直觉行为的根源在于采用了多层感知器（MLP）架构与不合适的初始化方案，这导致网络导数的可训练性较差，最终造成PDE残差损失优化不稳定。为解决此问题，我们提出了物理信息残差自适应网络（PirateNets），这是一种旨在促进深度PINN模型稳定高效训练的新型架构。PirateNets利用一种新颖的自适应残差连接，使得网络可初始化为浅层网络，并在训练过程中逐渐加深。我们还证明，所提出的初始化方案允许我们将对应于给定PDE系统的适当归纳偏置编码到网络架构中。我们提供的全面实证结果表明，PirateNets更易于优化，并能通过显著增加深度来提高精度，最终在各种基准测试中取得了最先进的成果。本文配套的所有代码和数据将公开发布于\url{https://github.com/PredictiveIntelligenceLab/jaxpi}。