Physics-Informed Neural Networks (PINNs) seek to solve partial differential equations (PDEs) with deep learning. Mainstream approaches that deploy fully-connected multi-layer deep learning architectures require prolonged training to achieve even moderate accuracy, while recent work on feature engineering allows higher accuracy and faster convergence. This paper introduces SAFE-NET, a Single-layered Adaptive Feature Engineering NETwork that achieves orders-of-magnitude lower errors with far fewer parameters than baseline feature engineering methods. SAFE-NET returns to basic ideas in machine learning, using Fourier features, a simplified single hidden layer network architecture, and an effective optimizer that improves the conditioning of the PINN optimization problem. Numerical results show that SAFE-NET converges faster and typically outperforms deeper networks and more complex architectures. It consistently uses fewer parameters -- on average, 65% fewer than the competing feature engineering methods -- while achieving comparable accuracy in less than 30% of the training epochs. Moreover, each SAFE-NET epoch is 95% faster than those of competing feature engineering approaches. These findings challenge the prevailing belief that modern PINNs effectively learn features in these scientific applications and highlight the efficiency gains possible through feature engineering.
翻译:物理信息神经网络(PINNs)旨在利用深度学习求解偏微分方程(PDEs)。主流方法采用全连接多层深度学习架构,但即使要达到中等精度也需要长时间训练;而近期关于特征工程的研究则能实现更高精度和更快收敛。本文提出的SAFE-NET(单层自适应特征工程网络)相较于基线特征工程方法,能以更少的参数实现数量级更低的误差。SAFE-NET回归机器学习的基本理念,采用傅里叶特征、简化的单隐层网络架构以及能够改善PINN优化问题条件特性的高效优化器。数值结果表明,SAFE-NET收敛速度更快,且通常优于更深的网络和更复杂的架构。该网络始终使用更少的参数——平均比竞争性特征工程方法少65%——同时能在不到30%的训练轮次内达到相当精度。此外,SAFE-NET每个训练轮次的计算速度比竞争性特征工程方法快95%。这些发现挑战了现代PINN能有效学习这些科学应用特征的普遍认知,并突显了通过特征工程实现效率提升的可能性。