Various normalization layers have been proposed to help the training of neural networks. Group Normalization (GN) is one of the effective and attractive studies that achieved significant performances in the visual recognition task. Despite the great success achieved, GN still has several issues that may negatively impact neural network training. In this paper, we introduce an analysis framework and discuss the working principles of GN in affecting the training process of the neural network. From experimental results, we conclude the real cause of GN's inferior performance against Batch normalization (BN): 1) \textbf{unstable training performance}, 2) \textbf{more sensitive} to distortion, whether it comes from external noise or perturbations introduced by the regularization. In addition, we found that GN can only help the neural network training in some specific period, unlike BN, which helps the network throughout the training. To solve these issues, we propose a new normalization layer built on top of GN, by incorporating the advantages of BN. Experimental results on the image classification task demonstrated that the proposed normalization layer outperforms the official GN to improve recognition accuracy regardless of the batch sizes and stabilize the network training.

翻译：为了帮助培训神经网络,提出了各种正常化层面的建议,以帮助培训神经网络。群体正常化(GN)是有效和有吸引力的研究之一,在视觉识别任务中取得了显著的成绩。尽管取得了巨大成功,但GN仍有若干问题可能对神经网络培训产生消极影响。在本文件中,我们引入了一个分析框架,并讨论了GN在影响神经网络培训过程中的工作原则。通过实验结果,我们完成了GN在批量标准化(BN:1)\textbf{unable培训绩效(BN):1\ textbf{unable培训绩效(BN),2\ textb{f{更敏感}对扭曲的真正表现的原因,无论这些扭曲来自外部噪音还是由正规化带来的扰动。此外,我们发现GN只有在特定时期里帮助神经网络培训,而不是BN(BN),它在整个培训过程中帮助网络。为了解决这些问题,我们提议在GN顶部上建立一个新的正常化层,将BN的优势纳入BN。在图像分类任务上的实验结果表明,拟议的标准化层超越了官方GN,不管分大小和稳定网络,提高准确性。