Learning with noisy labels is a practically challenging problem in weakly supervised learning. In the existing literature, open-set noises are always considered to be poisonous for generalization, similar to closed-set noises. In this paper, we empirically show that open-set noisy labels can be non-toxic and even benefit the robustness against inherent noisy labels. Inspired by the observations, we propose a simple yet effective regularization by introducing Open-set samples with Dynamic Noisy Labels (ODNL) into training. With ODNL, the extra capacity of the neural network can be largely consumed in a way that does not interfere with learning patterns from clean data. Through the lens of SGD noise, we show that the noises induced by our method are random-direction, conflict-free and biased, which may help the model converge to a flat minimum with superior stability and enforce the model to produce conservative predictions on Out-of-Distribution instances. Extensive experimental results on benchmark datasets with various types of noisy labels demonstrate that the proposed method not only enhances the performance of many existing robust algorithms but also achieves significant improvement on Out-of-Distribution detection tasks even in the label noise setting.

翻译：使用噪音标签的学习是监管不力的学习中一个实际具有挑战性的问题。在现有文献中,开放型噪音总是被认为对一般化有害,类似于封闭型噪音。在本文中,我们实验性地表明,开放型噪音标签可能无毒,甚至有益于对内在噪音标签的稳健性。在观察的启发下,我们提出简单而有效的规范化,在培训中引入带有动态噪音标签(ODNL)的开放型样品。在ODNL中,神经网络的超能力可以在很大程度上以不干扰从清洁数据中学习模式的方式消耗。我们通过 SGD噪音的镜头显示,我们所用方法引发的噪音是随机方向的,没有冲突,有偏颇性,这可能有助于模型与更高稳定性统一到一个固定的最低点,并且执行模型,以对外部扩散实例作出保守的预测。关于带有各种噪音标签的基准数据集的广泛实验结果表明,拟议的方法不仅能提高许多现有稳健的算法的性,而且还能显著改进了标签的噪音探测任务。