In this study, we delve into the task of few-shot Generative Domain Adaptation (GDA), which involves transferring a pre-trained generator from one domain to a new domain using only a few reference images. Inspired by the way human brains acquire knowledge in new domains, we present an innovative generator structure called Domain Re-Modulation (DoRM). DoRM not only meets the criteria of high quality, large synthesis diversity, and cross-domain consistency, which were achieved by previous research in GDA, but also incorporates memory and domain association, akin to how human brains operate. Specifically, DoRM freezes the source generator and introduces new mapping and affine modules (M&A modules) to capture the attributes of the target domain during GDA. This process resembles the formation of new synapses in human brains. Consequently, a linearly combinable domain shift occurs in the style space. By incorporating multiple new M&A modules, the generator gains the capability to perform high-fidelity multi-domain and hybrid-domain generation. Moreover, to maintain cross-domain consistency more effectively, we introduce a similarity-based structure loss. This loss aligns the auto-correlation map of the target image with its corresponding auto-correlation map of the source image during training. Through extensive experiments, we demonstrate the superior performance of our DoRM and similarity-based structure loss in few-shot GDA, both quantitatively and qualitatively. The code will be available at https://github.com/wuyi2020/DoRM.

翻译：在本研究中，我们深入探讨了小样本生成式域适应（Generative Domain Adaptation, GDA）任务，该任务旨在仅使用少量参考图像，将预训练生成器从一个域迁移至新域。受人类大脑在新域中获取知识方式的启发，我们提出了一种创新生成器结构——领域再调制（Domain Re-Modulation, DoRM）。DoRM 不仅满足先前GDA研究达成的高质量、大合成多样性与跨域一致性标准，还融入了类似人脑运作的记忆与域关联机制。具体而言，DoRM 冻结源生成器，并引入新的映射与仿射模块（M&A模块）以捕捉GDA过程中目标域的属性，此过程类似于人脑中新突触的形成。由此，风格空间中产生线性可组合的域偏移。通过集成多个新M&A模块，生成器获得执行高保真多域与混合域生成的能力。此外，为更有效维持跨域一致性，我们引入一种基于相似度的结构损失。该损失在训练过程中将目标图像的自相关图与其对应源图像的自相关图对齐。通过大量实验，我们从定量与定性角度验证了所提 DoRM 及基于相似度的结构损失在小样本GDA中的卓越性能。代码将于 https://github.com/wuyi2020/DoRM 公开。