In this work, we propose a novel activation mechanism called LayerAct for CNNs. This approach is motivated by our theoretical and experimental analyses, which demonstrate that Layer Normalization (LN) can mitigate a limitation of existing activation functions regarding noise robustness. However, LN is known to be disadvantageous in CNNs due to its tendency to make activation outputs homogeneous. The proposed method is designed to be more robust than existing activation functions by reducing the upper bound of influence caused by input shifts without inheriting LN's limitation. We provide analyses and experiments showing that LayerAct functions exhibit superior robustness compared to ElementAct functions. Experimental results on three clean and noisy benchmark datasets for image classification tasks indicate that LayerAct functions outperform other activation functions in handling noisy datasets while achieving superior performance on clean datasets in most cases.

翻译：本文提出了一种名为LayerAct的新型卷积神经网络激活机制。该方法的提出源于我们的理论与实验分析，这些分析表明层归一化（LN）能够缓解现有激活函数在噪声鲁棒性方面的局限性。然而，由于LN倾向于使激活输出同质化，其在CNN中的应用存在固有缺陷。所提出的方法通过降低输入偏移所引发影响的上界，在避免继承LN缺陷的同时，实现了比现有激活函数更强的鲁棒性。我们通过理论分析与实验验证表明，LayerAct函数相比ElementAct函数具有更优越的鲁棒性。在三个用于图像分类任务的干净及含噪声基准数据集上的实验结果表明，LayerAct函数在处理噪声数据集时优于其他激活函数，同时在多数情况下于干净数据集上也取得了更优的性能。