In recent years, diffusion models have emerged as the most powerful approach in image synthesis. However, applying these models directly to video synthesis presents challenges, as it often leads to noticeable flickering contents. Although recently proposed zero-shot methods can alleviate flicker to some extent, we still struggle to generate coherent videos. In this paper, we propose DiffSynth, a novel approach that aims to convert image synthesis pipelines to video synthesis pipelines. DiffSynth consists of two key components: a latent in-iteration deflickering framework and a video deflickering algorithm. The latent in-iteration deflickering framework applies video deflickering to the latent space of diffusion models, effectively preventing flicker accumulation in intermediate steps. Additionally, we propose a video deflickering algorithm, named patch blending algorithm, that remaps objects in different frames and blends them together to enhance video consistency. One of the notable advantages of DiffSynth is its general applicability to various video synthesis tasks, including text-guided video stylization, fashion video synthesis, image-guided video stylization, video restoring, and 3D rendering. In the task of text-guided video stylization, we make it possible to synthesize high-quality videos without cherry-picking. The experimental results demonstrate the effectiveness of DiffSynth. All videos can be viewed on our project page. Source codes will also be released.

翻译：近年来，扩散模型已成为图像合成领域最强大的方法。然而，将这些模型直接应用于视频合成仍面临挑战，因为往往会产生明显的闪烁内容。尽管近期提出的零样本方法能在一定程度上缓解闪烁问题，但生成连贯视频依然困难。本文提出DiffSynth——一种将图像合成流程转化为视频合成流程的创新方法。DiffSynth包含两个关键组件：潜在迭代去闪烁框架和视频去闪烁算法。潜在迭代去闪烁框架将视频去闪烁技术应用于扩散模型的潜在空间，有效防止中间步骤中闪烁的累积。此外，我们提出了一种名为"补丁混合算法"的视频去闪烁算法，该算法将不同帧中的对象重新映射并融合，以增强视频连贯性。DiffSynth的显著优势之一是其对多种视频合成任务的通用适用性，包括文本引导视频风格化、时尚视频合成、图像引导视频风格化、视频修复及3D渲染。在文本引导视频风格化任务中，我们实现了无需精心挑选即可合成高质量视频的能力。实验结果表明了DiffSynth的有效性。所有视频可查阅项目页面，源代码也将同步开源。