Semi-supervised learning (SSL) is a promising approach for training deep classification models using labeled and unlabeled datasets. However, existing SSL methods rely on a large unlabeled dataset, which may not always be available in many real-world applications due to legal constraints (e.g., GDPR). In this paper, we investigate the research question: Can we train SSL models without real unlabeled datasets? Instead of using real unlabeled datasets, we propose an SSL method using synthetic datasets generated from generative foundation models trained on datasets containing millions of samples in diverse domains (e.g., ImageNet). Our main concepts are identifying synthetic samples that emulate unlabeled samples from generative foundation models and training classifiers using these synthetic samples. To achieve this, our method is formulated as an alternating optimization problem: (i) meta-learning of generative foundation models and (ii) SSL of classifiers using real labeled and synthetic unlabeled samples. For (i), we propose a meta-learning objective that optimizes latent variables to generate samples that resemble real labeled samples and minimize the validation loss. For (ii), we propose a simple unsupervised loss function that regularizes the feature extractors of classifiers to maximize the performance improvement obtained from synthetic samples. We confirm that our method outperforms baselines using generative foundation models on SSL. We also demonstrate that our methods outperform SSL using real unlabeled datasets in scenarios with extremely small amounts of labeled datasets. This suggests that synthetic samples have the potential to provide improvement gains more efficiently than real unlabeled data.

翻译：半监督学习（SSL）是一种利用标记和未标记数据训练深度分类模型的有效方法。然而，现有SSL方法依赖大规模未标记数据集，而在许多实际应用中，由于法律约束（如GDPR），这类数据集往往难以获取。本文探讨的核心问题是：能否在不使用真实未标记数据集的情况下训练SSL模型？为此，我们提出一种无需真实未标记数据集的SSL方法，转而采用由生成式基础模型合成的数据集——该模型基于包含数百万样本（如ImageNet）的多领域数据训练而成。核心思想是：识别生成式基础模型中能模拟未标记样本的合成样本，并利用这些合成样本训练分类器。具体实现上，我们将方法构建为交替优化问题：（i）生成式基础模型的元学习，（ii）基于真实标记样本与合成未标记样本的分类器SSL。针对问题（i），我们提出一种元学习目标函数，通过优化潜在变量生成与真实标记样本相似且最小化验证损失的样本；针对问题（ii），设计简单无监督损失函数，正则化分类器特征提取器以最大化合成样本带来的性能提升。实验表明，在SSL任务中，本方法优于基于生成式基础模型的基线方法。此外，在标记样本量极少的场景下，本方法甚至超越使用真实未标记数据集的SSL方法，证明合成样本比真实未标记数据能更高效地提升模型性能。