An accurate and substantial dataset is essential for training a reliable and well-performing model. However, even manually annotated datasets contain label errors, not to mention automatically labeled ones. Previous methods for label denoising have primarily focused on detecting outliers and their permanent removal - a process that is likely to over- or underfilter the dataset. In this work, we propose AGRA: a new method for learning with noisy labels by using Adaptive GRAdient-based outlier removal. Instead of cleaning the dataset prior to model training, the dataset is dynamically adjusted during the training process. By comparing the aggregated gradient of a batch of samples and an individual example gradient, our method dynamically decides whether a corresponding example is helpful for the model at this point or is counter-productive and should be left out for the current update. Extensive evaluation on several datasets demonstrates AGRA's effectiveness, while a comprehensive results analysis supports our initial hypothesis: permanent hard outlier removal is not always what model benefits the most from.

翻译：准确且充足的数据集对于训练可靠且性能优异的模型至关重要。然而，即便是人工标注的数据集也包含标签错误，更不用说自动标注的数据集了。以往的标签去噪方法主要集中于检测离群点并对其进行永久移除——这一过程容易导致数据集过度过滤或过滤不足。本文提出AGRA：一种通过自适应梯度离群点移除进行含噪标签学习的新方法。该方法并非在模型训练前清洗数据集，而是在训练过程中动态调整数据集。通过比较一批样本的聚合梯度与单个样本梯度，我们的方法动态决定该样本在当前阶段对模型是否有益，或者是否会起反作用而应在当前更新中予以排除。在多个数据集上的广泛评估证明了AGRA的有效性，而全面的结果分析支持了我们最初的假设：永久移除硬离群点并非总是能让模型获益最多的方法。