Physics-informed neural networks (PINNs) and their variants have recently emerged as alternatives to traditional partial differential equation (PDE) solvers, but little literature has focused on devising accurate numerical integration methods for neural networks (NNs), which is essential for getting accurate solutions. In this work, we propose adaptive quadratures for the accurate integration of neural networks and apply them to loss functions appearing in low-dimensional PDE discretisations. We show that at opposite ends of the spectrum, continuous piecewise linear (CPWL) activation functions enable one to bound the integration error, while smooth activations ease the convergence of the optimisation problem. We strike a balance by considering a CPWL approximation of a smooth activation function. The CPWL activation is used to obtain an adaptive decomposition of the domain into regions where the network is almost linear, and we derive an adaptive global quadrature from this mesh. The loss function is then obtained by evaluating the smooth network (together with other quantities, e.g., the forcing term) at the quadrature points. We propose a method to approximate a class of smooth activations by CPWL functions and show that it has a quadratic convergence rate. We then derive an upper bound for the overall integration error of our proposed adaptive quadrature. The benefits of our quadrature are evaluated on a strong and weak formulation of the Poisson equation in dimensions one and two. Our numerical experiments suggest that compared to Monte-Carlo integration, our adaptive quadrature makes the convergence of NNs quicker and more robust to parameter initialisation while needing significantly fewer integration points and keeping similar training times.

翻译：物理信息神经网络（PINNs）及其变体近年来已成为传统偏微分方程（PDE）求解器的替代方案，但鲜有文献关注如何为神经网络（NNs）设计精确的数值积分方法——而这一方法对获得精确解至关重要。本文提出用于神经网络精确积分的自适应求积法，并将其应用于低维偏微分方程离散化中的损失函数。我们证明，在两个极端情况下，连续分段线性（CPWL）激活函数能够界定积分误差，而光滑激活函数则有助于优化问题的收敛。通过考虑光滑激活函数的CPWL近似，我们找到了平衡点。利用CPWL激活函数对区域进行自适应分解，得到网络近似线性的子区域，并基于该网格推导出自适应全局求积法。随后，通过在求积点处评估光滑网络（包括强迫项等其他量）获得损失函数。我们提出用CPWL函数逼近一类光滑激活函数的方法，并证明其具有二次收敛速率，进而推导出自适应求积法整体积分误差的上界。在二维和三维泊松方程的强形式与弱形式中评估了该求积法的优势。数值实验表明，与蒙特卡洛积分相比，自适应求积法在保持相近训练时间的同时，能以更少的积分点使神经网络收敛更快、对参数初始化更鲁棒。