Modern deep learning systems are data-hungry. Learning with web data is one of the feasible solutions, but will introduce label noise inevitably, which can hinder the performance of deep neural networks. Sample selection is an effective way to deal with label noise. The key is to separate clean samples based on some criterion. Previous methods pay more attention to the small loss criterion where small-loss samples are regarded as clean ones. Nevertheless, such a strategy relies on the learning dynamics of each data instance. Some noisy samples are still memorized due to frequently occurring corrupted learning patterns. To tackle this problem, a training-free surrogate model is preferred, freeing from the effect of memorization. In this work, we propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically. CLIP brings external knowledge to facilitate the selection of clean samples with its ability of text-image alignment. Furthermore, a margin adaptive loss is designed to regularize the selection bias introduced by CLIP, providing robustness to label noise. We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets. Our method achieves significant improvement without CLIP involved during the inference stage.

翻译：现代深度学习系统对数据需求量大，利用网络数据学习是可行的解决方案之一，但会不可避免地引入标签噪声，从而削弱深度神经网络的性能。样本选择是处理标签噪声的有效途径，其关键在于依据某种准则区分干净样本。以往方法更关注小损失准则，即视小损失样本为干净样本。然而，这种策略依赖于每个数据实例的学习动态，部分噪声样本仍会因频繁出现的错误学习模式而被记忆。为解决该问题，一种无需训练的替代模型更为理想，可避免记忆效应的影响。本文提出利用视觉-语言替代模型CLIP自动筛选噪声样本：CLIP通过其文本-图像对齐能力引入外部知识，促进干净样本的选择。此外，我们设计了一种边缘自适应损失函数，用于规约CLIP带来的选择偏差，从而增强对标签噪声的鲁棒性。我们在真实与合成的含噪声数据集上验证了所提方法的有效性。在推理阶段无需引入CLIP的情况下，我们的方法仍取得了显著性能提升。