Training deep neural networks(DNN) with noisy labels is challenging since DNN can easily memorize inaccurate labels, leading to poor generalization ability. Recently, the meta-learning based label correction strategy is widely adopted to tackle this problem via identifying and correcting potential noisy labels with the help of a small set of clean validation data. Although training with purified labels can effectively improve performance, solving the meta-learning problem inevitably involves a nested loop of bi-level optimization between model weights and hyper-parameters (i.e., label distribution). As compromise, previous methods resort to a coupled learning process with alternating update. In this paper, we empirically find such simultaneous optimization over both model weights and label distribution can not achieve an optimal routine, consequently limiting the representation ability of backbone and accuracy of corrected labels. From this observation, a novel multi-stage label purifier named DMLP is proposed. DMLP decouples the label correction process into label-free representation learning and a simple meta label purifier. In this way, DMLP can focus on extracting discriminative feature and label correction in two distinctive stages. DMLP is a plug-and-play label purifier, the purified labels can be directly reused in naive end-to-end network retraining or other robust learning methods, where state-of-the-art results are obtained on several synthetic and real-world noisy datasets, especially under high noise levels.

翻译：训练深度神经网络时，噪声标签具有挑战性，因为深度神经网络容易记忆不准确的标签，导致泛化能力较差。近年来，基于元学习的标签纠正策略被广泛采用，通过借助少量干净的验证数据识别并纠正潜在的噪声标签来解决这一问题。尽管使用净化后的标签可以有效提升性能，但解决元学习问题不可避免地涉及模型权重和超参数（即标签分布）之间的双层优化嵌套循环。作为折衷，先前的方法采用交替更新的耦合学习过程。本文通过实验发现，这种对模型权重和标签分布的同时优化无法达到最优路径，从而限制了骨干网络的表示能力和纠正标签的准确性。基于这一观察，我们提出了一种名为DMLP的新型多阶段标签净化器。DMLP将标签纠正过程解耦为无标签表示学习和简单的元标签净化器。通过这种方式，DMLP可以在两个不同阶段专注于提取判别性特征和标签纠正。DMLP是一种即插即用的标签净化器，净化后的标签可直接用于简单的端到端网络重训练或其他鲁棒学习方法，在多个合成和真实噪声数据集上取得了最先进的结果，尤其是在高噪声水平下。