Generative models have become a powerful tool for synthesizing training data in computer vision tasks. Current approaches solely focus on aligning generated images with the target dataset distribution. As a result, they capture only the common features in the real dataset and mostly generate 'easy samples', which are already well learned by models trained on real data. In contrast, those rare 'hard samples', with atypical features but crucial for enhancing performance, cannot be effectively generated. Consequently, these approaches must synthesize large volumes of data to yield appreciable performance gains, yet the improvement remains limited. To overcome this limitation, we present a novel method that can learn to control the learning difficulty of samples during generation while also achieving domain alignment. Thus, it can efficiently generate valuable 'hard samples' that yield significant performance improvements for target tasks. This is achieved by incorporating learning difficulty as an additional conditioning signal in generative models, together with a designed encoder structure and training-generation strategy. Experimental results across multiple datasets show that our method can achieve higher performance with lower generation cost. Specifically, we obtain the best performance with only 10% additional synthetic data, saving 63.4 GPU hours of generation time compared to the previous SOTA on ImageNet. Moreover, our method provides insightful visualizations of category-specific hard factors, serving as a tool for analyzing datasets.

翻译：生成模型已成为计算机视觉任务中合成训练数据的强大工具。当前方法仅关注将生成图像与目标数据集分布对齐。因此，它们仅捕获真实数据集中的常见特征，主要生成模型在真实数据上已充分学习的"简单样本"。相反，那些具有非典型特征但对提升性能至关重要的罕见"困难样本"无法被有效生成。因此，这些方法必须合成大量数据才能产生可观的性能提升，且改进程度仍然有限。为克服这一局限，我们提出一种新颖方法，能够在生成过程中学习控制样本的学习难度，同时实现域对齐。该方法可高效生成对目标任务产生显著性能提升的有价值"困难样本"。这是通过将学习难度作为生成模型中的附加条件信号，结合设计的编码器结构与训练-生成策略实现的。跨多个数据集的实验结果表明，我们的方法能够以更低的生成成本获得更高性能。具体而言，仅使用10%的额外合成数据即可获得最佳性能，相比ImageNet上先前的最优方法节省了63.4 GPU小时的生成时间。此外，我们的方法提供了类别特定困难因素的可视化分析，可作为数据集分析的工具。