In this work, we propose a novel activation mechanism aimed at establishing layer-level activation (LayerAct) functions. These functions are designed to be more noise-robust compared to traditional element-level activation functions by reducing the layer-level fluctuation of the activation outputs due to shift in inputs. Moreover, the LayerAct functions achieve a zero-like mean activation output without restricting the activation output space. We present an analysis and experiments demonstrating that LayerAct functions exhibit superior noise-robustness compared to element-level activation functions, and empirically show that these functions have a zero-like mean activation. Experimental results on three benchmark image classification tasks show that LayerAct functions excel in handling noisy image datasets, outperforming element-level activation functions, while the performance on clean datasets is also superior in most cases.

翻译：本文提出了一种新颖的激活机制，旨在构建层级激活（LayerAct）函数。这些函数通过降低因输入偏移导致的激活输出层级波动，相较于传统的元素级激活函数具有更强的噪声鲁棒性。此外，LayerAct函数在不限制激活输出空间的前提下，实现了近似零均值的激活输出。我们通过分析与实验表明，LayerAct函数相较于元素级激活函数展现出更优的噪声鲁棒性，并实证验证了这些函数具有近似零均值的激活输出特性。在三个基准图像分类任务上的实验结果表明，LayerAct函数在处理含噪图像数据集时表现优异，性能超越元素级激活函数，同时在大多数情况下对干净数据集的处理也更具优势。