Large-scale finite element simulations of complex physical systems governed by partial differential equations (PDE) crucially depend on adaptive mesh refinement (AMR) to allocate computational budget to regions where higher resolution is required. Existing scalable AMR methods make heuristic refinement decisions based on instantaneous error estimation and thus do not aim for long-term optimality over an entire simulation. We propose a novel formulation of AMR as a Markov decision process and apply deep reinforcement learning (RL) to train refinement policies directly from simulation. AMR poses a new problem for RL as both the state dimension and available action set changes at every step, which we solve by proposing new policy architectures with differing generality and inductive bias. The model sizes of these policy architectures are independent of the mesh size and hence can be deployed on larger simulations than those used at train time. We demonstrate in comprehensive experiments on static function estimation and time-dependent equations that RL policies can be trained on problems without using ground truth solutions, are competitive with a widely-used error estimator, and generalize to larger, more complex, and unseen test problems.

翻译：由偏微分方程（PDE）控制的复杂物理系统的大规模有限元模拟，关键依赖于自适应网格细化（AMR）将计算预算分配给需要更高分辨率的区域。现有的可扩展AMR方法基于瞬时误差估计做出启发式细化决策，因此不追求整个模拟过程的长期最优性。我们提出了一种将AMR建模为马尔可夫决策过程的新框架，并应用深度强化学习（RL）直接从模拟中训练细化策略。由于状态维度和可用动作集在每一步都会变化，这为RL带来了新问题。我们通过提出具有不同通用性和归纳偏置的新策略架构解决了这一问题。这些策略架构的模型大小独立于网格尺寸，因此可以部署在比训练时更大的模拟中。我们在静态函数估计和时间相关方程上的综合实验表明：无需使用真实解即可训练RL策略，这些策略与广泛使用的误差估计器具有竞争力，并能泛化到更大、更复杂且未见过的测试问题。