Limited by the complexity of basis function (B-spline) calculations, Kolmogorov-Arnold Networks (KAN) suffer from restricted parallel computing capability on GPUs. This paper proposes a novel ReLU-KAN implementation that inherits the core idea of KAN. By adopting ReLU (Rectified Linear Unit) and point-wise multiplication, we simplify the design of KAN's basis function and optimize the computation process for efficient CUDA computing. The proposed ReLU-KAN architecture can be readily implemented on existing deep learning frameworks (e.g., PyTorch) for both inference and training. Experimental results demonstrate that ReLU-KAN achieves a 20x speedup compared to traditional KAN with 4-layer networks. Furthermore, ReLU-KAN exhibits a more stable training process with superior fitting ability while preserving the "catastrophic forgetting avoidance" property of KAN. You can get the code in https://github.com/quiqi/relu_kan

翻译：受限于基函数（B样条）计算的复杂性，Kolmogorov-Arnold网络（KAN）在GPU上的并行计算能力受到制约。本文提出了一种继承KAN核心思想的新型ReLU-KAN实现方案。通过采用ReLU（线性整流单元）和逐点乘法，我们简化了KAN基函数的设计，并优化了计算流程以实现高效的CUDA计算。所提出的ReLU-KAN架构能够便捷地在现有深度学习框架（如PyTorch）上部署，同时支持推理和训练。实验结果表明，在4层网络配置下，ReLU-KAN相较于传统KAN实现了20倍的加速比。此外，ReLU-KAN在保持KAN“避免灾难性遗忘”特性的同时，展现出更稳定的训练过程和更优异的拟合能力。代码可通过https://github.com/quiqi/relu_kan获取。