Physics-informed neural networks (PINNs) and their variants have recently emerged as alternatives to traditional partial differential equation (PDE) solvers, but little literature has focused on devising accurate numerical integration methods for neural networks (NNs), which is essential for getting accurate solutions. In this work, we propose adaptive quadratures for the accurate integration of neural networks and apply them to loss functions appearing in low-dimensional PDE discretisations. We show that at opposite ends of the spectrum, continuous piecewise linear (CPWL) activation functions enable one to bound the integration error, while smooth activations ease the convergence of the optimisation problem. We strike a balance by considering a CPWL approximation of a smooth activation function. The CPWL activation is used to obtain an adaptive decomposition of the domain into regions where the network is almost linear, and we derive an adaptive global quadrature from this mesh. The loss function is then obtained by evaluating the smooth network (together with other quantities, e.g., the forcing term) at the quadrature points. We propose a method to approximate a class of smooth activations by CPWL functions and show that it has a quadratic convergence rate. We then derive an upper bound for the overall integration error of our proposed adaptive quadrature. The benefits of our quadrature are evaluated on a strong and weak formulation of the Poisson equation in dimensions one and two. Our numerical experiments suggest that compared to Monte-Carlo integration, our adaptive quadrature makes the convergence of NNs quicker and more robust to parameter initialisation while needing significantly fewer integration points and keeping similar training times.

翻译：物理信息神经网络（PINNs）及其变体近年来作为传统偏微分方程（PDE）求解器的替代方案崭露头角，但鲜有文献专注于为神经网络（NNs）设计精确的数值积分方法，而这对获取精确解至关重要。本研究提出用于神经网络精确积分的自适应求积法，并将其应用于低维PDE离散化中出现的损失函数。我们证明：在频谱的两端，连续分段线性（CPWL）激活函数能够约束积分误差，而光滑激活函数则有利于优化问题的收敛。通过考虑光滑激活函数的CPWL近似，我们实现了两者的平衡。利用CPWL激活函数，可对网络近似线性的区域进行自适应域分解，并基于该网格推导出自适应全局求积公式。随后，通过评估光滑网络（连同其他量如强迫项）在求积节点上的值来获得损失函数。我们提出了一种用CPWL函数近似特定光滑激活函数类的方法，并证明其具有二次收敛率。进而推导出所提自适应求积法整体积分误差的上界。通过一维和二维泊松方程的强形式与弱形式公式评估了该求积法的优势。数值实验表明：与蒙特卡洛积分相比，我们的自适应求积法能使神经网络收敛更快、对参数初始化更鲁棒，同时所需积分节点显著减少且训练时间相近。