Deep neural networks (DNNs) have garnered significant attention in various fields of science and technology in recent years. Activation functions define how neurons in DNNs process incoming signals for them. They are essential for learning non-linear transformations and for performing diverse computations among successive neuron layers. In the last few years, researchers have investigated the approximation ability of DNNs to explain their power and success. In this paper, we explore the approximation ability of DNNs using a different activation function, called SignReLU. Our theoretical results demonstrate that SignReLU networks outperform rational and ReLU networks in terms of approximation performance. Numerical experiments are conducted comparing SignReLU with the existing activations such as ReLU, Leaky ReLU, and ELU, which illustrate the competitive practical performance of SignReLU.

翻译：深度神经网络（DNNs）近年来在科学技术的各个领域引起了广泛关注。激活函数决定了DNNs中神经元处理输入信号的方式，对于学习非线性变换以及实现连续神经元层间的多种计算至关重要。近年来，研究者们通过探究DNNs的逼近能力来解释其强大性能与成功机理。本文研究采用一种名为SignReLU的新型激活函数的DNNs的逼近能力。理论结果表明，SignReLU网络在逼近性能上优于有理函数网络和ReLU网络。通过数值实验将SignReLU与ReLU、Leaky ReLU和ELU等现有激活函数进行对比，揭示了SignReLU在实际应用中的竞争性表现。