The Rectified Linear Unit (ReLU) is a foundational activation function in artficial neural networks. Recent literature frequently misattributes its origin to the 2018 (initial) version of this paper, which exclusively investigated ReLU at the classification layer. This paper formally corrects the citation record by tracing the mathematical lineage of piecewise linear functions from early biological models to their definitive integration into deep learning by Nair & Hinton (2010). Alongside this historical rectification, we present a comprehensive empirical comparison of the ReLU, Hyperbolic Tangent (Tanh), and Logistic (Sigmoid) activation functions across image classification, text classification, and image reconstruction tasks. To ensure statistical robustness, we evaluated these functions using 10 independent randomized trials and assessed significance using the non-parametric Kruskal-Wallis $H$ test. The empirical data validates the theoretical limitations of saturating functions. Sigmoid failed to converge in deep convolutional vision tasks due to the vanishing gradient problem, thus yielding accuracies equivalent to random probability. Conversely, ReLU and Tanh exhibited stable convergence. ReLU achieved the highest mean accuracy and F1-score on image classification and text classification tasks, while Tanh yielded the highest peak signal to noise ratio in image reconstruction. Ultimately, this study confirms a statistically significant performance variance among activations, thus reaffirming the necessity of non-saturating functions in deep architectures, and restores proper historical attribution to prior literature.

翻译：修正线性单元（ReLU）是人工神经网络中的基础激活函数。近期文献中常将其起源错误地归因于本论文的2018年（初始）版本，而该版本仅研究了ReLU在分类层中的应用。本文通过追溯分段线性函数从早期生物模型到由Nair和Hinton（2010）将其最终整合至深度学习的数学谱系，正式纠正了引用记录。在此历史源流考证的基础上，我们全面对比了ReLU、双曲正切（Tanh）和逻辑斯蒂（Sigmoid）激活函数在图像分类、文本分类及图像重建任务中的性能表现。为确保统计稳健性，我们采用10次独立随机试验评估这些函数，并使用非参数Kruskal-Wallis $H$检验评估显著性。经验数据验证了饱和函数的理论局限性。由于梯度消失问题，Sigmoid在深度卷积视觉任务中无法收敛，导致其准确率等同于随机概率。相反，ReLU和Tanh表现出稳定收敛性。ReLU在图像分类和文本分类任务中取得了最高的平均准确率和F1分数，而Tanh在图像重建中实现了最高的峰值信噪比。最终，本研究确认了激活函数间存在统计学显著性的性能差异，从而重申了非饱和函数在深层架构中的必要性，并恢复了先前文献应有的历史归属。