This paper examines the influence of internal Gaussian noise on the performance of deep feedforward neural networks, focusing on the role of the noise injection stage relative to the activation function. Two scenarios are analyzed: noise introduced before and after the activation function, for both additive and multiplicative noise influence. The case of noise before activation function is similar to perturbations in the input channel of neuron, while the noise introduced after activation function is analogous to noise occurring either within the neuron itself or in its output channel. The types of noise and the method of their introduction were inspired by analog neural networks. The results show that the activation function acts as an effective nonlinear filter of noise. Networks with noise introduced before the activation function consistently achieve higher accuracy than those with noise applied after it, with additive noise being more effectively suppressed in this case. For noise introduced after the activation function, multiplicative noise is less detrimental than additive noise, and earlier hidden layers contribute more significantly to performance degradation due to cumulative noise amplification governed by the statistical properties of subsequent weight matrices. The study also demonstrates that pooling-based noise reduction is effective in both cases when noise is introduced before and after the activation function, consistently improving network performance.

翻译：本文研究了内源高斯噪声对深度前馈神经网络性能的影响，重点分析了噪声注入阶段与激活函数之间的关联。我们探讨了两种场景：在激活函数之前和之后注入噪声，并分别考虑了加性噪声与乘性噪声的作用。激活函数前注入噪声的情况类似于神经元输入通道的扰动，而激活函数后注入噪声则对应神经元内部或其输出通道产生的噪声。噪声类型及其注入方法的选取受模拟神经网络启发。研究结果表明，激活函数可充当有效的非线性噪声滤波器。在激活函数前注入噪声的网络，其准确率始终优于在激活函数后注入噪声的网络，且在此情况下加性噪声能被更有效地抑制。当噪声在激活函数后注入时，乘性噪声的危害小于加性噪声，同时浅层隐藏层因后续权重矩阵统计特性导致的累积噪声放大，对性能退化的影响更为显著。本研究还证实，无论噪声在激活函数前还是后注入，基于池化的降噪方法均能有效提升网络性能。