Multi-domain image-to-image (I2I) translations can transform a source image according to the style of a target domain. One important, desired characteristic of these transformations, is their graduality, which corresponds to a smooth change between the source and the target image when their respective latent-space representations are linearly interpolated. However, state-of-the-art methods usually perform poorly when evaluated using inter-domain interpolations, often producing abrupt changes in the appearance or non-realistic intermediate images. In this paper, we argue that one of the main reasons behind this problem is the lack of sufficient inter-domain training data and we propose two different regularization methods to alleviate this issue: a new shrinkage loss, which compacts the latent space, and a Mixup data-augmentation strategy, which flattens the style representations between domains. We also propose a new metric to quantitatively evaluate the degree of the interpolation smoothness, an aspect which is not sufficiently covered by the existing I2I translation metrics. Using both our proposed metric and standard evaluation protocols, we show that our regularization techniques can improve the state-of-the-art multi-domain I2I translations by a large margin. Our code will be made publicly available upon the acceptance of this article.
翻译:多域图像到图像(I2I)翻译可根据目标域风格转换源图像。这类变换的一个重要理想特性是其渐进性,即当源图像与目标图像在潜在空间中的表示进行线性插值时,两者之间应呈现平滑过渡。然而,当前最先进方法在跨域插值评估中通常表现欠佳,常出现外观突变或生成非真实中间图像等问题。本文指出,该问题的主因之一是跨域训练数据不足,并提出两种正则化方法以缓解此问题:一种新型收缩损失(shrinkage loss),用于压缩潜在空间;以及一种混合数据增强策略(Mixup data-augmentation),用于平滑域间风格表征。我们还提出一种新指标,用于定量评估插值平滑度——这一特性在现有I2I翻译指标中尚未得到充分覆盖。通过所提指标与标准评估协议,我们证明了正则化技术能显著提升多域I2I翻译的最优性能。代码将在论文接收后开源。