Traditional neural networks (multi-layer perceptrons) have become an important tool in data science due to their success across a wide range of tasks. However, their performance is sometimes unsatisfactory, and they often require a large number of parameters, primarily due to their reliance on the linear combination structure. Meanwhile, additive regression has been a popular alternative to linear regression in statistics. In this work, we introduce novel deep neural networks that incorporate the idea of additive regression. Our neural networks share architectural similarities with Kolmogorov-Arnold networks but are based on simpler yet flexible activation and basis functions. Additionally, we introduce several hybrid neural networks that combine this architecture with that of traditional neural networks. We derive their universal approximation properties and demonstrate their effectiveness through simulation studies and a real-data application. The numerical results indicate that our neural networks generally achieve better performance than traditional neural networks while using fewer parameters.

翻译：传统神经网络（多层感知机）因其在广泛任务中的成功表现，已成为数据科学中的重要工具。然而，其性能有时不尽如人意，且通常需要大量参数，这主要归因于其对线性组合结构的依赖。与此同时，在统计学中，加法回归已成为线性回归的一种流行替代方法。在本研究中，我们引入了融合加法回归思想的新型深度神经网络。我们的神经网络在架构上与Kolmogorov-Arnold网络具有相似性，但基于更简单且灵活的激活函数与基函数。此外，我们提出了几种混合神经网络，将此架构与传统神经网络架构相结合。我们推导了其通用逼近性质，并通过模拟研究和实际数据应用验证了其有效性。数值结果表明，我们的神经网络通常能以更少的参数实现优于传统神经网络的性能。