Recent semi-supervised learning (SSL) methods typically include a filtering strategy to improve the quality of pseudo labels. However, these filtering strategies are usually hand-crafted and do not change as the model is updated, resulting in a lot of correct pseudo labels being discarded and incorrect pseudo labels being selected during the training process. In this work, we observe that the distribution gap between the confidence values of correct and incorrect pseudo labels emerges at the very beginning of the training, which can be utilized to filter pseudo labels. Based on this observation, we propose a Self-Adaptive Pseudo-Label Filter (SPF), which automatically filters noise in pseudo labels in accordance with model evolvement by modeling the confidence distribution throughout the training process. Specifically, with an online mixture model, we weight each pseudo-labeled sample by the posterior of it being correct, which takes into consideration the confidence distribution at that time. Unlike previous handcrafted filters, our SPF evolves together with the deep neural network without manual tuning. Extensive experiments demonstrate that incorporating SPF into the existing SSL methods can help improve the performance of SSL, especially when the labeled data is extremely scarce.

翻译：当前半监督学习方法通常采用过滤策略以提升伪标签质量。然而，这些过滤策略多为手工设计，且不随模型更新而调整，导致训练过程中大量正确伪标签被丢弃，同时错误伪标签被选中。本研究发现，正确与错误伪标签置信度值之间的分布差异在训练初期便已显现，可被用于过滤伪标签。基于此发现，我们提出自适应伪标签过滤方法（SPF），通过建模训练全过程的置信度分布，使其自动随模型演化过滤伪标签噪声。具体而言，我们采用在线混合模型，根据当前置信度分布为每个伪标签样本赋予其正确的后验权重。与先前手工设计的过滤器不同，SPF能够随深度神经网络共同演化，无需人工调参。大量实验表明，将SPF融入现有半监督学习方法可有效提升模型性能，尤其在标注数据极度稀缺的场景下效果显著。