Unlike the case when using a balanced training dataset, the per-class recall (i.e., accuracy) of neural networks trained with an imbalanced dataset are known to vary a lot from category to category. The convention in long-tailed recognition is to manually split all categories into three subsets and report the average accuracy within each subset. We argue that under such an evaluation setting, some categories are inevitably sacrificed. On one hand, focusing on the average accuracy on a balanced test set incurs little penalty even if some worst performing categories have zero accuracy. On the other hand, classes in the "Few" subset do not necessarily perform worse than those in the "Many" or "Medium" subsets. We therefore advocate to focus more on improving the lowest recall among all categories and the harmonic mean of all recall values. Specifically, we propose a simple plug-in method that is applicable to a wide range of methods. By simply re-training the classifier of an existing pre-trained model with our proposed loss function and using an optional ensemble trick that combines the predictions of the two classifiers, we achieve a more uniform distribution of recall values across categories, which leads to a higher harmonic mean accuracy while the (arithmetic) average accuracy is still high. The effectiveness of our method is justified on widely used benchmark datasets.

翻译：与使用平衡训练数据集的情况不同，使用非平衡数据集训练的神经网络在各类别上的召回率（即准确率）已知呈现显著差异。长尾识别领域的惯例是将所有类别手动分为三个子集，并报告每个子集内的平均准确率。我们认为，在这种评估设置下，某些类别不可避免地会被牺牲。一方面，聚焦于平衡测试集上的平均准确率几乎不会带来惩罚，即使某些表现最差的类别准确率为零。另一方面，“少样本”子集中的类别并不必然比“多样本”或“中样本”子集中的类别表现更差。因此，我们主张更多地关注所有类别中的最低召回率以及所有召回率的调和平均数。具体地，我们提出了一种简单的即插即用方法，可广泛适用于多种方法。通过使用我们提出的损失函数重新训练现有预训练模型的分类器，并结合可选的集成技巧（即融合两个分类器的预测结果），我们实现了类别间召回率更均匀的分布，从而在保持高（算术）平均准确率的同时，获得更高的调和平均准确率。我们的方法在广泛使用的基准数据集上得到了有效性验证。