With the emerging diffusion models, recently, text-to-video generation has aroused increasing attention. But an important bottleneck therein is that generative videos often tend to carry some flickers and artifacts. In this work, we propose a dual-stream diffusion net (DSDN) to improve the consistency of content variations in generating videos. In particular, the designed two diffusion streams, video content and motion branches, could not only run separately in their private spaces for producing personalized video variations as well as content, but also be well-aligned between the content and motion domains through leveraging our designed cross-transformer interaction module, which would benefit the smoothness of generated videos. Besides, we also introduce motion decomposer and combiner to faciliate the operation on video motion. Qualitative and quantitative experiments demonstrate that our method could produce amazing continuous videos with fewer flickers.

翻译：随着扩散模型的兴起，文本生成视频任务近期备受关注。然而，该领域存在一个关键瓶颈：生成的视频往往会出现闪烁和伪影问题。为此，本文提出了一种双流扩散网络（DSDN），旨在提升生成视频中内容变化的一致性。具体而言，我们设计的两个扩散流——视频内容分支与运动分支——不仅能够在各自独立的空间中运行，以生成个性化的视频变化与内容，还可通过我们设计的跨Transformer交互模块，在内容域与运动域之间实现良好对齐，从而提升生成视频的平滑性。此外，我们还引入了运动分解器与组合器，以简化对视频运动的操作。定性与定量实验均表明，我们的方法能够生成连续流畅且闪烁更少的优质视频。