Semi-supervised learning (SSL) provides an effective means of leveraging unlabelled data to improve a model performance. Even though the domain has received a considerable amount of attention in the past years, most methods present the common drawback of lacking theoretical guarantees. Our starting point is to notice that the estimate of the risk that most discriminative SSL methods minimise is biased, even asymptotically. This bias impedes the use of standard statistical learning theory and can hurt empirical performance. We propose a simple way of removing the bias. Our debiasing approach is straightforward to implement and applicable to most deep SSL methods. We provide simple theoretical guarantees on the trustworthiness of these modified methods, without having to rely on the strong assumptions on the data distribution that SSL theory usually requires. In particular, we provide generalisation error bounds for the proposed methods. We evaluate debiased versions of different existing SSL methods, such as the Pseudo-label method and Fixmatch, and show that debiasing can compete with classic deep SSL techniques in various settings by providing better calibrated models. Additionally, we provide a theoretical explanation of the intuition of the popular SSL methods.

翻译：半监督学习（SSL）为利用无标签数据提升模型性能提供了一种有效手段。尽管该领域在过去几年中受到了广泛关注，但大多数方法普遍存在缺乏理论保证的缺陷。我们首先注意到，大多数判别性SSL方法所最小化的风险估计量即使在大样本下也存在偏差。这一偏差阻碍了标准统计学习理论的应用，并可能损害实证性能。我们提出了一种简单的偏差消除方法。我们的去偏方法易于实现，且适用于大多数深度SSL方法。我们为这些改进方法的可信度提供了简单的理论保证，无需依赖SSL理论通常对数据分布的强假设。特别地，我们为所提方法给出了泛化误差界。我们评估了不同现有SSL方法（如伪标签方法和FixMatch）的去偏版本，并证明去偏能在各种设置下通过提供校准更优的模型与经典深度SSL技术相竞争。此外，我们还对流行SSL方法的直觉给出了理论解释。