Prior works about text-to-image synthesis typically concatenated the sentence embedding with the noise vector, while the sentence embedding and the noise vector are two different factors, which control the different aspects of the generation. Simply concatenating them will entangle the latent factors and encumber the generative model. In this paper, we attempt to decompose these two factors and propose Factor Decomposed Generative Adversarial Networks~(FDGAN). To achieve this, we firstly generate images from the noise vector and then apply the sentence embedding in the normalization layer for both generator and discriminators. We also design an additive norm layer to align and fuse the text-image features. The experimental results show that decomposing the noise and the sentence embedding can disentangle latent factors in text-to-image synthesis, and make the generative model more efficient. Compared with the baseline, FDGAN can achieve better performance, while fewer parameters are used.

翻译：关于文本到图像合成的先前工作通常将句子嵌入与噪声向量拼接，而句子嵌入和噪声向量是控制生成过程中不同方面的两个不同因子。简单拼接它们会纠缠潜在因子，并阻碍生成模型的效能。本文尝试分解这两个因子，并提出因子分解生成对抗网络（FDGAN）。为实现这一目标，我们首先从噪声向量生成图像，然后将句子嵌入应用于生成器和判别器的归一化层。我们还设计了一个加法归一化层，用于对齐和融合文本-图像特征。实验结果表明，分解噪声和句子嵌入可以解开文本到图像合成中的潜在因子，并使生成模型更高效。与基线相比，FDGAN能在使用更少参数的情况下实现更优性能。