An accurate and substantial dataset is essential for training a reliable and well-performing model. However, even manually annotated datasets contain label errors, not to mention automatically labeled ones. Previous methods for label denoising have primarily focused on detecting outliers and their permanent removal - a process that is likely to over- or underfilter the dataset. In this work, we propose AGRA: a new method for learning with noisy labels by using Adaptive GRAdient-based outlier removal. Instead of cleaning the dataset prior to model training, the dataset is dynamically adjusted during the training process. By comparing the aggregated gradient of a batch of samples and an individual example gradient, our method dynamically decides whether a corresponding example is helpful for the model at this point or is counter-productive and should be left out for the current update. Extensive evaluation on several datasets demonstrates AGRA's effectiveness, while a comprehensive results analysis supports our initial hypothesis: permanent hard outlier removal is not always what model benefits the most from.

翻译：准确且充足的数据集对于训练可靠且性能优良的模型至关重要。然而，即使是人工标注的数据集也存在标签错误，更不用说自动标注的数据集了。以往的标签去噪方法主要侧重于检测离群值并将其永久移除——这一过程容易导致数据集过滤过度或不足。在本文中，我们提出AGRA：一种利用自适应梯度离群值剔除进行噪声标签学习的新方法。该方法并非在模型训练前清理数据集，而是在训练过程中动态调整数据集。通过比较一批样本的聚合梯度与单个样本梯度，我们的方法动态决定相应样本在当前阶段是否对模型有益，抑或会产生反效果而应在当前更新中忽略。在多个数据集上的广泛评估证明了AGRA的有效性，而全面的结果分析也支持了我们的初始假设：永久性硬离群值剔除并非总能给模型带来最大收益。