Image-to-image (I2I) translation comprises a wide spectrum of tasks. Here we divide this problem into three levels: strong-fidelity translation, normal-fidelity translation, and weak-fidelity translation, indicating the extent to which the content of the original image is preserved. Although existing methods achieve good performance in weak-fidelity translation, they fail to fully preserve the content in both strong- and normal-fidelity tasks, e.g. sim2real, style transfer and low-level vision. In this work, we propose Hierarchy Flow, a novel flow-based model to achieve better content preservation during translation. Specifically, 1) we first unveil the drawbacks of standard flow-based models when applied to I2I translation. 2) Next, we propose a new design, namely hierarchical coupling for reversible feature transformation and multi-scale modeling, to constitute Hierarchy Flow. 3) Finally, we present a dedicated aligned-style loss for a better trade-off between content preservation and stylization during translation. Extensive experiments on a wide range of I2I translation benchmarks demonstrate that our approach achieves state-of-the-art performance, with convincing advantages in both strong- and normal-fidelity tasks. Code and models will be at https://github.com/WeichenFan/HierarchyFlow.
翻译:图像到图像(I2I)翻译涵盖广泛的任务。我们将该问题划分为三个层次:强保真翻译、正常保真翻译和弱保真翻译,分别表示原始图像内容保留的程度。尽管现有方法在弱保真翻译中表现良好,但在强保真和正常保真任务(如模拟到真实、风格迁移和低级视觉)中未能完全保留内容。本文提出多层次流(Hierarchy Flow),一种新颖的基于流的模型,旨在翻译过程中实现更好的内容保留。具体而言:1)首先揭示标准流模型应用于I2I翻译时的缺陷;2)其次提出新型设计——层次化耦合(hierarchical coupling),用于可逆特征变换和多尺度建模,构建多层次流;3)最后提出专用的对齐风格损失(aligned-style loss),以在翻译过程中更好地权衡内容保留与风格化。在广泛的I2I翻译基准测试上的大量实验表明,我们的方法达到了最先进的性能,在强保真和正常保真任务中均具有显著优势。代码和模型将发布在https://github.com/WeichenFan/HierarchyFlow。