The goal of Image-to-image (I2I) translation is to transfer an image from a source domain to a target domain, which has recently drawn increasing attention. One major branch of this research is to formulate I2I translation based on Generative Adversarial Network (GAN). As a zero-sum game, GAN can be reformulated as a Partially-observed Markov Decision Process (POMDP) for generators, where generators cannot access full state information of their environments. This formulation illustrates the information insufficiency in the GAN training. To mitigate this problem, we propose to add a communication channel between discriminators and generators. We explore multiple architecture designs to integrate the communication mechanism into the I2I translation framework. To validate the performance of the proposed approach, we have conducted extensive experiments on various benchmark datasets. The experimental results confirm the superiority of our proposed method.
翻译:摘要:图像到图像翻译(Image-to-image, I2I)旨在将图像从源域迁移至目标域,近年来受到广泛关注。该研究的一个重要分支是基于生成对抗网络(Generative Adversarial Network, GAN)构建I2I翻译模型。作为零和博弈,GAN可被重新表述为面向生成器的部分可观测马尔可夫决策过程(Partially-observed Markov Decision Process, POMDP),其中生成器无法获取环境的完整状态信息。该形式化揭示了GAN训练中的信息不充分问题。为解决此问题,我们提出在判别器与生成器之间构建通信通道。通过探索多种架构设计方案,将通信机制集成至I2I翻译框架中。为验证所提方法的性能,我们在多个基准数据集上开展了广泛实验,结果证实了本方法具有显著优越性。