Pseudo-labeling is a crucial technique in semi-supervised learning (SSL), where artificial labels are generated for unlabeled data by a trained model, allowing for the simultaneous training of labeled and unlabeled data in a supervised setting. However, several studies have identified three main issues with pseudo-labeling-based approaches. Firstly, these methods heavily rely on predictions from the trained model, which may not always be accurate, leading to a confirmation bias problem. Secondly, the trained model may be overfitted to easy-to-learn examples, ignoring hard-to-learn ones, resulting in the \textit{"Matthew effect"} where the already strong become stronger and the weak weaker. Thirdly, most of the low-confidence predictions of unlabeled data are discarded due to the use of a high threshold, leading to an underutilization of unlabeled data during training. To address these issues, we propose a new method called ReFixMatch, which aims to utilize all of the unlabeled data during training, thus improving the generalizability of the model and performance on SSL benchmarks. Notably, ReFixMatch achieves 41.05\% top-1 accuracy with 100k labeled examples on ImageNet, outperforming the baseline FixMatch and current state-of-the-art methods.

翻译：[translated abstract in Chinese] 伪标签技术是半监督学习中的关键方法，其通过训练模型为未标注数据生成人工标签，使得在监督学习框架下可同时训练标注与未标注数据。然而，多项研究指出基于伪标签的方法存在三个主要问题：首先，这类方法高度依赖训练模型的预测结果，但预测结果并不总是准确，由此引发确认偏误问题；其次，训练模型可能对易学习样本过拟合而忽略难学习样本，导致"马太效应"——强者愈强、弱者愈弱；第三，由于使用高阈值，大部分低置信度的未标注数据预测结果被丢弃，造成训练过程中未标注数据利用率不足。针对上述问题，我们提出名为ReFixMatch的新方法，旨在训练过程中充分利用所有未标注数据，从而提升模型泛化能力及在半监督学习基准测试中的表现。值得关注的是，ReFixMatch在ImageNet上仅使用10万张标注样本即达到41.05%的top-1准确率，超越了基线方法FixMatch及当前最先进方法。