While physics-informed neural networks (PINNs) have become a popular deep learning framework for tackling forward and inverse problems governed by partial differential equations (PDEs), their performance is known to degrade when larger and deeper neural network architectures are employed. Our study identifies that the root of this counter-intuitive behavior lies in the use of multi-layer perceptron (MLP) architectures with non-suitable initialization schemes, which result in poor trainablity for the network derivatives, and ultimately lead to an unstable minimization of the PDE residual loss. To address this, we introduce Physics-informed Residual Adaptive Networks (PirateNets), a novel architecture that is designed to facilitate stable and efficient training of deep PINN models. PirateNets leverage a novel adaptive residual connection, which allows the networks to be initialized as shallow networks that progressively deepen during training. We also show that the proposed initialization scheme allows us to encode appropriate inductive biases corresponding to a given PDE system into the network architecture. We provide comprehensive empirical evidence showing that PirateNets are easier to optimize and can gain accuracy from considerably increased depth, ultimately achieving state-of-the-art results across various benchmarks. All code and data accompanying this manuscript will be made publicly available at \url{https://github.com/PredictiveIntelligenceLab/jaxpi}.

翻译：虽然物理信息神经网络（PINNs）已成为求解偏微分方程（PDEs）正问题和反问题的常用深度学习框架，但研究表明，当采用更大、更深的神经网络架构时，其性能会出现退化。本研究揭示了这一反直觉行为的根源在于使用多层感知机（MLP）架构时搭配了不合适的初始化方案，这导致网络导数训练困难，并最终引发偏微分方程残差损失的最小化过程不稳定。为解决此问题，我们提出了物理信息残差自适应网络（PirateNets），这是一种旨在促进深度PINN模型稳定高效训练的新型架构。PirateNets利用一种创新的自适应残差连接，使得网络可初始化为浅层网络，并在训练过程中逐步加深。我们还证明了所提出的初始化方案能够将对应于给定偏微分方程系统的适当归纳偏置编码到网络架构中。我们提供了全面的实证证据，表明PirateNets更易于优化，并能通过显著增加的深度提升精度，最终在各项基准测试中取得了最先进的结果。本文附带的全部代码和数据将公开发布于\url{https://github.com/PredictiveIntelligenceLab/jaxpi}。