Learning an empirically effective model with generalization using limited data is a challenging task for deep neural networks. In this paper, we propose a novel learning framework called PurifiedLearning to exploit task-irrelevant features extracted from task-irrelevant labels when training models on small-scale datasets. Particularly, we purify feature representations by using the expression of task-irrelevant information, thus facilitating the learning process of classification. Our work is built on solid theoretical analysis and extensive experiments, which demonstrate the effectiveness of PurifiedLearning. According to the theory we proved, PurifiedLearning is model-agnostic and doesn't have any restrictions on the model needed, so it can be combined with any existing deep neural networks with ease to achieve better performance. The source code of this paper will be available in the future for reproducibility.

翻译：使用有限数据进行一般化的经验有效模型学习是深层神经网络的一项艰巨任务。在本文中,我们提议建立一个名为“纯化学习”的新学习框架,以利用小型数据集培训模型时从与任务相关的标签中提取的与任务相关的特征。特别是,我们通过表达与任务相关的信息来净化特征表现,从而便利分类的学习过程。我们的工作建立在扎实的理论分析和广泛的实验的基础上,这些实验证明了纯化学习的有效性。根据我们所证明的理论,纯化学习是模型-不可知性的,对所需的模型没有任何限制,因此它可以与任何现有的深层神经网络相结合,从而更容易地取得更好的性能。今后,将可提供本文的源代码供再版使用。