Long-tailed data is still a big challenge for deep neural networks, even though they have achieved great success on balanced data. We observe that vanilla training on long-tailed data with cross-entropy loss makes the instance-rich head classes severely squeeze the spatial distribution of the tail classes, which leads to difficulty in classifying tail class samples. Furthermore, the original cross-entropy loss can only propagate gradient short-lively because the gradient in softmax form rapidly approaches zero as the logit difference increases. This phenomenon is called softmax saturation. It is unfavorable for training on balanced data, but can be utilized to adjust the validity of the samples in long-tailed data, thereby solving the distorted embedding space of long-tailed problems. To this end, this paper proposes the Gaussian clouded logit adjustment by Gaussian perturbation of different class logits with varied amplitude. We define the amplitude of perturbation as cloud size and set relatively large cloud sizes to tail classes. The large cloud size can reduce the softmax saturation and thereby making tail class samples more active as well as enlarging the embedding space. To alleviate the bias in a classifier, we therefore propose the class-based effective number sampling strategy with classifier re-training. Extensive experiments on benchmark datasets validate the superior performance of the proposed method. Source code is available at https://github.com/Keke921/GCLLoss.

翻译：长尾数据仍然是深度神经网络面临的重大挑战，尽管它们在平衡数据上取得了巨大成功。我们观察到，在长尾数据上使用交叉熵损失进行标准训练会导致样本丰富的头部类别严重挤压尾部类别的空间分布，从而增加尾部类别样本的分类难度。此外，原始交叉熵损失只能传播短期梯度，因为随着逻辑差值的增加，softmax形式的梯度迅速趋近于零。这种现象被称为softmax饱和。虽然这对平衡数据训练不利，但可被用于调整长尾数据中样本的有效性，从而解决长尾问题中嵌入空间扭曲的问题。为此，本文提出对不同类别的逻辑值施加不同幅度的高斯扰动，即高斯模糊逻辑调整。我们将扰动幅度定义为云大小，并对尾部类别设置较大的云大小。较大的云大小能降低softmax饱和程度，从而使尾部类别样本更加活跃并扩大嵌入空间。为减轻分类器中的偏差，我们进一步提出了基于类别的有效数量采样策略与分类器重训练。在基准数据集上的大量实验验证了所提方法的优越性能。源代码可在https://github.com/Keke921/GCLLoss获取。