Few-shot image generation aims to generate data of an unseen category based on only a few samples. Apart from basic content generation, a bunch of downstream applications hopefully benefit from this task, such as low-data detection and few-shot classification. To achieve this goal, the generated images should guarantee category retention for classification beyond the visual quality and diversity. In our preliminary work, we present an ``editing-based'' framework Attribute Group Editing (AGE) for reliable few-shot image generation, which largely improves the generation performance. Nevertheless, AGE's performance on downstream classification is not as satisfactory as expected. This paper investigates the class inconsistency problem and proposes Stable Attribute Group Editing (SAGE) for more stable class-relevant image generation. SAGE takes use of all given few-shot images and estimates a class center embedding based on the category-relevant attribute dictionary. Meanwhile, according to the projection weights on the category-relevant attribute dictionary, we can select category-irrelevant attributes from the similar seen categories. Consequently, SAGE injects the whole distribution of the novel class into StyleGAN's latent space, thus largely remains the category retention and stability of the generated images. Going one step further, we find that class inconsistency is a common problem in GAN-generated images for downstream classification. Even though the generated images look photo-realistic and requires no category-relevant editing, they are usually of limited help for downstream classification. We systematically discuss this issue from both the generative model and classification model perspectives, and propose to boost the downstream classification performance of SAGE by enhancing the pixel and frequency components.
翻译:少样本图像生成旨在基于少量样本生成未见类别的数据。除了基本内容生成外,该任务有望为低数据检测与少样本分类等下游应用提供支持。为实现这一目标,生成的图像除了视觉质量与多样性外,还需确保类别保持性以支持分类任务。在前期工作中,我们提出了基于“编辑”的框架——属性组编辑(AGE)用于可靠的少样本图像生成,显著提升了生成性能。然而,AGE在下游分类任务中的表现未达预期。本文研究了类别不一致性问题,并提出稳定属性组编辑(SAGE)以实现更稳定的类别相关图像生成。SAGE利用所有给定的少样本图像,基于类别相关属性字典估计类中心嵌入。同时,根据类别相关属性字典上的投影权重,可从相似可见类别中筛选出类别无关属性。因此,SAGE将新类别的整体分布注入StyleGAN的潜在空间,从而显著保持生成图像的类别保持性与稳定性。进一步研究发现,类别不一致性是GAN生成图像用于下游分类时的普遍问题。即使生成的图像具有照片级真实感且无需类别相关编辑,它们通常对下游分类的帮助有限。我们分别从生成模型与分类模型的角度系统讨论了该问题,并提出通过增强像素与频率分量来提升SAGE在下游分类中的性能。