In the task of texture transfer, reference texture images typically exhibit highly repetitive texture features, and the texture transfer results from different content images under the same style also share remarkably similar texture patterns. Encoding such highly similar texture features often requires deep layers and a large number of channels, making it is also the main source of the entire model's parameter count and computational load, and inference time. We propose a lightweight texture transfer based on texture feature preset (TFP). TFP takes full advantage of the high repetitiveness of texture features by providing preset universal texture feature maps for a given style. These preset feature maps can be fused and decoded directly with shallow color transfer feature maps of any content to generate texture transfer results, thereby avoiding redundant texture information from being encoded repeatedly. The texture feature map we preset is encoded through noise input images with consistent distribution (standard normal distribution). This consistent input distribution can completely avoid the problem of texture transfer differentiation, and by randomly sampling different noise inputs, we can obtain different texture features and texture transfer results under the same reference style. Compared to state-of-the-art techniques, our TFP not only produces visually superior results but also reduces the model size by 3.2-3538 times and speeds up the process by 1.8-5.6 times.
翻译:在纹理迁移任务中,参考纹理图像通常呈现高度重复的纹理特征,且同一样式下不同内容图像的纹理迁移结果也共享极为相似的纹理模式。编码此类高度相似的纹理特征通常需要深层网络与大量通道,这也是导致整个模型参数量、计算量以及推理时间的主要来源。我们提出了一种基于纹理特征预设(TFP)的轻量级纹理迁移方法。TFP充分利用纹理特征的高重复性,为给定样式预设通用的纹理特征图。这些预设特征图可直接与任意内容的浅层颜色迁移特征图进行融合与解码,生成纹理迁移结果,从而避免冗余纹理信息的重复编码。所预设的纹理特征图通过服从一致分布(标准正态分布)的噪声输入图像编码得到。这种一致的输入分布可完全避免纹理迁移差异化问题,且通过随机采样不同噪声输入,可在同一参考样式下获得不同的纹理特征与纹理迁移结果。与现有最优技术相比,我们的TFP不仅生成视觉上更优的结果,还将模型体积缩小3.2-3538倍,并将处理速度提升1.8-5.6倍。