Finding solutions to partial differential equations (PDEs) is an important and essential component in many scientific and engineering discoveries. One of the common approaches empowered by deep learning is Physics-informed Neural Networks (PINNs). Recently, a new type of fundamental neural network model, Kolmogorov-Arnold Networks (KANs), has been proposed as a substitute of Multilayer Perceptions (MLPs), and possesses trainable activation functions. To enhance KANs in fitting accuracy, a modification of KANs, so called ReLU-KANs, using "square of ReLU" as the basis of its activation functions, has been suggested. In this work, we propose another basis of activation functions, namely, Higherorder-ReLU (HR), which is simpler than the basis of activation functions used in KANs, namely, Bsplines; allows efficient KAN matrix operations; and possesses smooth and non-zero higher-order derivatives, essential to physicsinformed neural networks. We name such KANs with Higher-order-ReLU (HR) as their activations, HRKANs. Our detailed experiments on two famous and representative PDEs, namely, the linear Poisson equation and nonlinear Burgers' equation with viscosity, reveal that our proposed Higher-order-ReLU-KANs (HRKANs) achieve the highest fitting accuracy and training robustness and lowest training time significantly among KANs, ReLU-KANs and HRKANs. The codes to replicate our experiments are available at https://github.com/kelvinhkcs/HRKAN.

翻译：求解偏微分方程（PDE）是许多科学与工程发现中一个重要且关键的组成部分。由深度学习赋能的一种常用方法是物理信息神经网络（PINNs）。最近，一种新型的基础神经网络模型——Kolmogorov-Arnold网络（KANs）被提出，作为多层感知机（MLPs）的替代方案，并拥有可训练的激活函数。为了提高KANs的拟合精度，一种改进的KANs，即所谓的ReLU-KANs，被提出，其激活函数的基础采用“ReLU的平方”。在本工作中，我们提出了另一种激活函数的基础，即高阶ReLU（HR）。它比KANs中使用的激活函数基础（即B样条）更简单；允许高效的KAN矩阵运算；并且拥有光滑且非零的高阶导数，这对于物理信息神经网络至关重要。我们将这种以高阶ReLU（HR）作为激活函数的KANs命名为HRKANs。我们在两个著名且具有代表性的PDE（即线性泊松方程和带粘性的非线性Burgers方程）上进行的详细实验表明，我们提出的高阶ReLU-KANs（HRKANs）在KANs、ReLU-KANs和HRKANs中，显著实现了最高的拟合精度和训练鲁棒性，以及最低的训练时间。复现我们实验的代码可在 https://github.com/kelvinhkcs/HRKAN 获取。