Recent progress in semi- and self-supervised learning has caused a rift in the long-held belief about the need for an enormous amount of labeled data for machine learning and the irrelevancy of unlabeled data. Although it has been successful in various data, there is no dominant semi- and self-supervised learning method that can be generalized for tabular data (i.e. most of the existing methods require appropriate tabular datasets and architectures). In this paper, we revisit self-training which can be applied to any kind of algorithm including the most widely used architecture, gradient boosting decision tree, and introduce curriculum pseudo-labeling (a state-of-the-art pseudo-labeling technique in image) for a tabular domain. Furthermore, existing pseudo-labeling techniques do not assure the cluster assumption when computing confidence scores of pseudo-labels generated from unlabeled data. To overcome this issue, we propose a novel pseudo-labeling approach that regularizes the confidence scores based on the likelihoods of the pseudo-labels so that more reliable pseudo-labels which lie in high density regions can be obtained. We exhaustively validate the superiority of our approaches using various models and tabular datasets.

翻译：近年来，半监督与自监督学习的进展动摇了长期以来关于机器学习需要大量标注数据以及未标注数据无关的固有观念。尽管这些方法在各种数据上取得了成功，但目前尚无一种能够泛化至表格数据的通用半监督与自监督学习方法（即，现有方法大多需要适配的表格数据集与架构）。在本文中，我们重新审视了自训练方法——该方法可应用于包括最广泛使用的梯度提升决策树架构在内的任意算法——并引入了课程伪标签（一种图像领域前沿的伪标签技术）以适用于表格领域。此外，现有伪标签技术在对未标注数据生成的伪标签计算置信度时，未能保证聚类假设的有效性。为解决这一问题，我们提出了一种新颖的伪标签方法，该方法基于伪标签的似然性对置信度进行正则化，从而能够获取位于高密度区域中的更可靠的伪标签。我们通过多种模型与表格数据集全面验证了所提方法的优越性。