Generative DNNs are a powerful tool for image synthesis, but they are limited by their computational load. On the other hand, given a trained model and a task, e.g. faces generation within a range of characteristics, the output image quality will be unevenly distributed among images with different characteristics. It follows, that we might restrain the models complexity on some instances, maintaining a high quality. We propose a method for diminishing computations by adding so-called early exit branches to the original architecture, and dynamically switching the computational path depending on how difficult it will be to render the output. We apply our method on two different SOTA models performing generative tasks: generation from a semantic map, and cross-reenactment of face expressions; showing it is able to output images with custom lower-quality thresholds. For a threshold of LPIPS <=0.1, we diminish their computations by up to a half. This is especially relevant for real-time applications such as synthesis of faces, when quality loss needs to be contained, but most of the inputs need fewer computations than the complex instances.
翻译:生成式深度神经网络是图像合成的强大工具,但其计算负载限制了应用。然而,针对特定任务(如指定特征范围内的人脸生成)的已训练模型,不同特征图像的输出质量分布并不均匀。这意味着我们可以在保持高质量的前提下,对部分实例降低模型复杂度。本文提出一种通过向原始架构添加所谓早期退出分支来减少计算量的方法,并根据输出渲染难度动态切换计算路径。我们将该方法应用于两种执行生成任务的最先进模型——基于语义图的生成与人脸表情交叉迁移,实验表明其能够以用户自定义的低质量阈值输出图像。当LPIPS阈值≤0.1时,计算量可降低近50%。该方法对于需要控制质量损失但多数输入实例计算复杂度低于复杂场景的实时应用(如人脸合成)尤为重要。