Kolmogorov-Arnold Networks (KANs) are a recent neural network architecture offering an alternative to Multilayer Perceptrons (MLPs) with improved explainability and expressibility. However, KANs are significantly slower than MLPs due to the recursive nature of B-spline function computations, limiting their application. This work addresses these issues by proposing a novel base-spline Linear-Time B-splines Kolmogorov-Arnold Network (LTBs-KAN) with linear complexity. Unlike previous methods that rely on the Boor-Mansfield-Cox spline algorithm or other computationally intensive mathematical functions, our approach significantly reduces the computational burden. Additionally, we further reduce model's parameter through product-of-sums matrix factorization in the forward pass without sacrificing performance. Experiments on MNIST, Fashion-MNIST and CIFAR-10 demonstrate that LTBs-KAN achieves good time complexity and parameter reduction, when used as building architectural blocks, compared to other KAN implementations.
翻译:Kolmogorov-Arnold网络(KANs)是一种新兴的神经网络架构,为多层感知机(MLPs)提供了替代方案,具有更强的可解释性和表达能力。然而,由于B样条函数计算的递归特性,KANs的速度显著慢于MLPs,这限制了其应用。本文通过提出一种具有线性复杂度的新型基础样条线性时间B样条Kolmogorov-Arnold网络(LTBs-KAN),解决了这些问题。与以往依赖Boor-Mansfield-Cox样条算法或其他计算密集型数学函数的方法不同,我们的方法显著降低了计算负担。此外,我们通过在前向传播中引入积和矩阵分解,在保持性能的同时进一步减少了模型参数。在MNIST、Fashion-MNIST和CIFAR-10上的实验表明,与其他KAN实现相比,LTBs-KAN在用作建筑模块时,实现了良好的时间复杂度和参数缩减。