Text-guided video-to-video stylization transforms the visual appearance of a source video to a different appearance guided on textual prompts. Existing text-guided image diffusion models can be extended for stylized video synthesis. However, they struggle to generate videos with both highly detailed appearance and temporal consistency. In this paper, we propose a synchronized multi-frame diffusion framework to maintain both the visual details and the temporal consistency. Frames are denoised in a synchronous fashion, and more importantly, information of different frames is shared since the beginning of the denoising process. Such information sharing ensures that a consensus, in terms of the overall structure and color distribution, among frames can be reached in the early stage of the denoising process before it is too late. The optical flow from the original video serves as the connection, and hence the venue for information sharing, among frames. We demonstrate the effectiveness of our method in generating high-quality and diverse results in extensive experiments. Our method shows superior qualitative and quantitative results compared to state-of-the-art video editing methods.

翻译：文本驱动的视频到视频风格化旨在根据文本提示，将源视频的视觉外观转变为另一种风格。现有的文本驱动图像扩散模型可扩展用于风格化视频合成，但其难以生成兼具高度细节化外观与时序一致性的视频。本文提出一种同步多帧扩散框架，以同时保持视觉细节与时序一致性。帧以同步方式去噪，且更重要的是，从去噪过程开始，不同帧的信息便实现共享。这种信息共享确保在去噪过程的早期阶段（尚未出现不可逆偏差时），帧间就能在整体结构与颜色分布上达成共识。原始视频的光流作为帧间连接的桥梁及信息共享的载体。通过大量实验，我们证明了该方法在生成高质量、多样化结果方面的有效性。相较于当前最先进的视频编辑方法，我们的方法在定性与定量评估中均展现出更优性能。