In this study, we present an efficient and effective approach for achieving temporally consistent synthetic-to-real video translation in videos of varying lengths. Our method leverages off-the-shelf conditional image diffusion models, allowing us to perform multiple synthetic-to-real image generations in parallel. By utilizing the available optical flow information from the synthetic videos, our approach seamlessly enforces temporal consistency among corresponding pixels across frames. This is achieved through joint noise optimization, effectively minimizing spatial and temporal discrepancies. To the best of our knowledge, our proposed method is the first to accomplish diverse and temporally consistent synthetic-to-real video translation using conditional image diffusion models. Furthermore, our approach does not require any training or fine-tuning of the diffusion models. Extensive experiments conducted on various benchmarks for synthetic-to-real video translation demonstrate the effectiveness of our approach, both quantitatively and qualitatively. Finally, we show that our method outperforms other baseline methods in terms of both temporal consistency and visual quality.

翻译：在本研究中，我们提出了一种高效且有效的方法，用于实现不同长度视频中时间一致的合成到真实视频翻译。我们的方法利用现有的条件图像扩散模型，能够并行执行多个合成到真实的图像生成任务。通过利用合成视频中可用的光流信息，该方法无缝地强制帧间对应像素的时间一致性。这是通过联合噪声优化实现的，有效减少空间和时间差异。据我们所知，所提出的方法是首个利用条件图像扩散模型实现多样化且时间一致的合成到真实视频翻译的方法。此外，我们的方法无需对扩散模型进行任何训练或微调。在多个合成到真实视频翻译基准上进行的广泛实验，定量和定性均证明了该方法的效果。最后，我们表明，该方法在时间一致性和视觉质量方面均优于其他基线方法。