The physics solvers employed for neural network training are primarily iterative, and hence, differentiating through them introduces a severe computational burden as iterations grow large. Inspired by works in bilevel optimization, we show that full accuracy of the network is achievable through physics significantly coarser than fully converged solvers. We propose Progressively Refined Differentiable Physics (PRDP), an approach that identifies the level of physics refinement sufficient for full training accuracy. By beginning with coarse physics, adaptively refining it during training, and stopping refinement at the level adequate for training, it enables significant compute savings without sacrificing network accuracy. Our focus is on differentiating iterative linear solvers for sparsely discretized differential operators, which are fundamental to scientific computing. PRDP is applicable to both unrolled and implicit differentiation. We validate its performance on a variety of learning scenarios involving differentiable physics solvers such as inverse problems, autoregressive neural emulators, and correction-based neural-hybrid solvers. In the challenging example of emulating the Navier-Stokes equations, we reduce training time by 62%.

翻译：用于神经网络训练的物理求解器主要为迭代式，因此对其进行微分会随着迭代次数增加而引入严重的计算负担。受双层优化研究的启发，我们证明通过比完全收敛求解器粗糙得多的物理模拟即可实现网络的完整精度。我们提出渐进精细化可微分物理（PRDP）方法，该方法能识别出足以达到完整训练精度的物理细化层级。通过从粗糙物理模拟开始，在训练过程中自适应地细化，并在满足训练需求的层级停止细化，该方法可在不牺牲网络精度的前提下显著节省计算资源。我们的研究重点在于对稀疏离散微分算子的迭代线性求解器进行微分，这是科学计算的基础。PRDP方法同时适用于展开式微分和隐式微分。我们在涉及可微分物理求解器的多种学习场景中验证了其性能，包括反问题、自回归神经模拟器以及基于修正的神经混合求解器。在模拟纳维-斯托克斯方程这一挑战性示例中，我们将训练时间减少了62%。