The potential of learned models for fundamental scientific research and discovery is drawing increasing attention. Physics-informed neural networks (PINNs), where the loss function directly embeds governing equations of scientific phenomena, is one of the key techniques at the forefront of recent advances. These models are typically trained using stochastic gradient descent, akin to their standard deep learning counterparts. However, in this paper, we carry out a simple analysis showing that the loss functions arising in PINNs lead to a high degree of complexity and ruggedness that may not be conducive for gradient-descent and its variants. It is therefore clear that the use of neuro-evolutionary algorithms as alternatives to gradient descent for PINNs may be a better choice. Our claim is strongly supported herein by benchmark problems and baseline results demonstrating that convergence rates achieved by neuroevolution can indeed surpass that of gradient descent for PINN training. Furthermore, implementing neuroevolution with JAX leads to orders of magnitude speedup relative to standard implementations.

翻译：学习模型在基础科学研究与发现方面的潜力正受到越来越多的关注。物理信息神经网络（PINNs）是近期前沿进展中的关键技术之一，其损失函数直接嵌入了科学现象的控制方程。这类模型通常采用随机梯度下降进行训练，与其标准深度学习模型类似。然而，本文通过简单分析表明，PINNs中的损失函数具有高度的复杂性和崎岖性，可能不利于梯度下降及其变体的优化。因此，采用神经进化算法作为梯度下降的替代方法用于PINNs可能更为合适。基准问题与基线结果有力支持了我们的论点，证明神经进化在PINN训练中实现的收敛速率确实可超越梯度下降。此外，借助JAX实现神经进化相比标准实现可带来数量级的加速。