Denoising diffusion probabilistic models (DDPMs) have been proven capable of synthesizing high-quality images with remarkable diversity when trained on large amounts of data. However, to our knowledge, few-shot image generation tasks have yet to be studied with DDPM-based approaches. Modern approaches are mainly built on Generative Adversarial Networks (GANs) and adapt models pre-trained on large source domains to target domains using a few available samples. In this paper, we make the first attempt to study when do DDPMs overfit and suffer severe diversity degradation as training data become scarce. Then we fine-tune DDPMs pre-trained on large source domains to solve the overfitting problem when training data is limited. Although the directly fine-tuned models accelerate convergence and improve generation quality and diversity compared with training from scratch, they still fail to retain some diverse features and can only produce coarse images. Therefore, we design a DDPM pairwise adaptation (DDPM-PA) approach to optimize few-shot DDPM domain adaptation. DDPM-PA efficiently preserves information learned from source domains by keeping the relative pairwise distances between generated samples during adaptation. Besides, DDPM-PA enhances the learning of high-frequency details from source models and limited training data. DDPM-PA further improves generation quality and diversity and achieves results better than current state-of-the-art GAN-based approaches. We demonstrate the effectiveness of our approach on a series of few-shot image generation tasks qualitatively and quantitatively.

翻译：去噪扩散概率模型（DDPMs）已被证明在大量数据训练时能够合成具有显著多样性的高质量图像。然而，据我们所知，目前尚未有基于DDPM的方法研究少样本图像生成任务。现代方法主要建立在生成对抗网络（GANs）基础上，通过利用少量可用样本将预训练于大型源域的模型适配到目标域。本文首次研究了DDPMs在训练数据稀缺时何时会出现过拟合及严重多样性退化的问题。随后，我们对预训练于大型源域的DDPMs进行微调，以解决训练数据有限时的过拟合问题。尽管直接微调后的模型与从头训练相比能加速收敛并提升生成质量与多样性，但仍无法保留某些多样性特征，且仅能生成粗糙图像。为此，我们提出一种DDPM成对适配（DDPM-PA）方法，以优化少样本DDPM域适配。DDPM-PA通过保持适配过程中生成样本间的相对成对距离，有效保留从源域学到的信息。此外，DDPM-PA增强了对源模型及有限训练数据中高频细节的学习。DDPM-PA进一步提升了生成质量与多样性，并取得了优于当前最先进GAN方法的结果。我们通过一系列少样本图像生成任务定性与定量地验证了该方法有效性。