Advances in Semi-Supervised Learning (SSL) have almost entirely closed the gap between SSL and Supervised Learning at a fraction of the number of labels. However, recent performance improvements have often come \textit{at the cost of significantly increased training computation}. To address this, we propose Curriculum Batch Size (CBS), \textit{an unlabeled batch size curriculum which exploits the natural training dynamics of deep neural networks.} A small unlabeled batch size is used in the beginning of training and is gradually increased to the end of training. A fixed curriculum is used regardless of dataset, model or number of epochs, and reduced training computations is demonstrated on all settings. We apply CBS, strong labeled augmentation, Curriculum Pseudo Labeling (CPL) \citep{FlexMatch} to FixMatch \citep{FixMatch} and term the new SSL algorithm Fast FixMatch. We perform an ablation study to show that strong labeled augmentation and/or CPL do not significantly reduce training computations, but, in synergy with CBS, they achieve optimal performance. Fast FixMatch also achieves substantially higher data utilization compared to previous state-of-the-art. Fast FixMatch achieves between $2.1\times$ - $3.4\times$ reduced training computations on CIFAR-10 with all but 40, 250 and 4000 labels removed, compared to vanilla FixMatch, while attaining the same cited state-of-the-art error rate \citep{FixMatch}. Similar results are achieved for CIFAR-100, SVHN and STL-10. Finally, Fast MixMatch achieves between $2.6\times$ - $3.3\times$ reduced training computations in federated SSL tasks and online/streaming learning SSL tasks, which further demonstrate the generializbility of Fast MixMatch to different scenarios and tasks.

翻译：半监督学习（SSL）的进展几乎完全弥合了SSL与监督学习之间的差距，且仅需标签数量的一小部分。然而，近年来的性能提升往往以显著增加训练计算量作为代价。为解决这一问题，我们提出课程批次大小（CBS）——一种利用深度神经网络自然训练动态的无标签批次大小课程。训练初期使用较小的无标签批次大小，并逐渐增加至训练结束。无论数据集、模型或训练轮次如何，均采用固定课程，并在所有设置中证明了训练计算量的减少。我们将CBS、强标签增强、课程伪标签（CPL）应用于FixMatch，并将新的SSL算法命名为快速FixMatch。消融研究表明，强标签增强和/或CPL本身不会显著减少训练计算量，但与CBS协同作用时能实现最优性能。与先前最先进方法相比，快速FixMatch还实现了显著更高的数据利用率。在CIFAR-10数据集上，移除除40、250和4000个标签外的所有标签后，快速FixMatch训练计算量相比原始FixMatch减少2.1倍至3.4倍，同时达到相同的已引用最先进错误率。在CIFAR-100、SVHN和STL-10数据集上也取得了类似结果。最后，在联邦SSL任务和在线/流式学习SSL任务中，快速MixMatch的训练计算量减少2.6倍至3.3倍，进一步证明了快速MixMatch在不同场景和任务中的泛化能力。