Physics-informed neural networks (PINNs) and their variants have recently emerged as alternatives to traditional partial differential equation (PDE) solvers, but little literature has focused on devising accurate numerical integration methods for neural networks (NNs), which is essential for getting accurate solutions. In this work, we propose adaptive quadratures for the accurate integration of neural networks and apply them to loss functions appearing in low-dimensional PDE discretisations. We show that at opposite ends of the spectrum, continuous piecewise linear (CPWL) activation functions enable one to bound the integration error, while smooth activations ease the convergence of the optimisation problem. We strike a balance by considering a CPWL approximation of a smooth activation function. The CPWL activation is used to obtain an adaptive decomposition of the domain into regions where the network is almost linear, and we derive an adaptive global quadrature from this mesh. The loss function is then obtained by evaluating the smooth network (together with other quantities, e.g., the forcing term) at the quadrature points. We propose a method to approximate a class of smooth activations by CPWL functions and show that it has a quadratic convergence rate. We then derive an upper bound for the overall integration error of our proposed adaptive quadrature. The benefits of our quadrature are evaluated on a strong and weak formulation of the Poisson equation in dimensions one and two. Our numerical experiments suggest that compared to Monte-Carlo integration, our adaptive quadrature makes the convergence of NNs quicker and more robust to parameter initialisation while needing significantly fewer integration points and keeping similar training times.

翻译：物理信息神经网络（PINNs）及其变体近年来已成为传统偏微分方程（PDE）求解器的替代方案，但鲜有文献专注于设计适用于神经网络（NNs）的精确数值积分方法，而这正是获取精确解的关键。本文提出针对神经网络精确积分的自适应求积法，并将其应用于低维PDE离散化中出现的损失函数。研究表明，在频谱的两端：连续分段线性（CPWL）激活函数可约束积分误差，而光滑激活函数则有助于优化问题的收敛。我们通过考虑光滑激活函数的CPWL近似来达成平衡。利用CPWL激活函数对区域进行自适应分解，生成网络近似线性的子区域，并基于该网格导出自适应全局求积法则。随后通过在求积节点处计算光滑网络（及其他量，如强迫项）来获得损失函数。我们提出一种用CPWL函数逼近光滑激活函数族的方法，并证明其具有二次收敛率。进一步推导出所提自适应求积法整体积分误差的上界。通过一维与二维泊松方程的强形式和弱形式评估该求积法的优势。数值实验表明，与蒙特卡洛积分相比，所提自适应求积法能在显著减少积分节点数量并保持相近训练时间的同时，使神经网络收敛更快且对参数初始化更具鲁棒性。