With the emerging diffusion models, recently, text-to-video generation has aroused increasing attention. But an important bottleneck therein is that generative videos often tend to carry some flickers and artifacts. In this work, we propose a dual-stream diffusion net (DSDN) to improve the consistency of content variations in generating videos. In particular, the designed two diffusion streams, video content and motion branches, could not only run separately in their private spaces for producing personalized video variations as well as content, but also be well-aligned between the content and motion domains through leveraging our designed cross-transformer interaction module, which would benefit the smoothness of generated videos. Besides, we also introduce motion decomposer and combiner to faciliate the operation on video motion. Qualitative and quantitative experiments demonstrate that our method could produce amazing continuous videos with fewer flickers.

翻译：随着新兴扩散模型的发展，文本到视频生成近期引起了越来越多的关注。然而，其中一个重要瓶颈在于生成的视频往往容易出现闪烁和伪影问题。本文提出了一种双流扩散网络（DSDN），以提升生成视频中内容变化的一致性。具体而言，我们设计的两个扩散流——视频内容分支和运动分支——不仅可以在各自的私有空间中独立运行以生成个性化的视频变化和内容，还能够通过我们设计的跨Transformer交互模块在内容域和运动域之间实现良好对齐，从而有利于生成视频的平滑性。此外，我们还引入了运动分解器和组合器以辅助视频运动操作。定性和定量实验表明，我们的方法能够生成令人惊叹的连续视频且闪烁现象显著减少。