Class imbalance and noisy labels are the norm rather than the exception in many large-scale classification datasets. Nevertheless, most works in machine learning typically assume balanced and clean data. There have been some recent attempts to tackle, on one side, the problem of learning from noisy labels and, on the other side, learning from long-tailed data. Each group of methods make simplifying assumptions about the other. Due to this separation, the proposed solutions often underperform when both assumptions are violated. In this work, we present a simple two-stage approach based on recent advances in self-supervised learning to treat both challenges simultaneously. It consists of, first, task-agnostic self-supervised pre-training, followed by task-specific fine-tuning using an appropriate loss. Most significantly, we find that self-supervised learning approaches are effectively able to cope with severe class imbalance. In addition, the resulting learned representations are also remarkably robust to label noise, when fine-tuned with an imbalance- and noise-resistant loss function. We validate our claims with experiments on CIFAR-10 and CIFAR-100 augmented with synthetic imbalance and noise, as well as the large-scale inherently noisy Clothing-1M dataset.

翻译：分类不平衡和吵闹标签是许多大规模分类数据集的常规而非例外。然而,大多数机器学习工作通常假定平衡和干净的数据。最近曾试图一方面解决从噪音标签和长尾数据中学习的问题,另一方面则从长尾数据中学习。每组方法都简化了对另一类的假设。由于这种分离,在两种假设被违反时,提议的解决方案往往表现不佳。在这项工作中,我们根据在自我监督学习方面的最新进展提出一个简单的两阶段办法,以同时处理这两种挑战。它首先包括任务性自上而下的自上而下的培训前自上而上,然后以适当的损失对具体任务进行微调。最重要的是,我们发现自上而下的学习方法能够有效地应对严重的阶级不平衡。此外,由此产生的学习表现也非常有力,在与不平衡和噪音抗损失功能进行微调时,贴上噪音标签。我们用对CARFAR-10和CIFAR-100的实验来验证我们的索赔要求,先用合成的不平衡和噪音加固的合成数据加以验证。