With the emerging diffusion models, recently, text-to-video generation has aroused increasing attention. But an important bottleneck therein is that generative videos often tend to carry some flickers and artifacts. In this work, we propose a dual-stream diffusion net (DSDN) to improve the consistency of content variations in generating videos. In particular, the designed two diffusion streams, video content and motion branches, could not only run separately in their private spaces for producing personalized video variations as well as content, but also be well-aligned between the content and motion domains through leveraging our designed cross-transformer interaction module, which would benefit the smoothness of generated videos. Besides, we also introduce motion decomposer and combiner to faciliate the operation on video motion. Qualitative and quantitative experiments demonstrate that our method could produce amazing continuous videos with fewer flickers.

翻译：随着扩散模型的新兴发展，文本到视频生成近期引起日益关注。然而，其中一个重要瓶颈在于生成的视频常易产生闪烁与伪影。本文提出双流扩散网络（DSDN），以提升生成视频中内容变化的连贯性。具体而言，所设计的两个扩散流——视频内容分支与运动分支——不仅能在其私有空间中独立运行以生成个性化的视频变化及内容，还能通过我们设计的跨变换器交互模块实现内容域与运动域间的良好对齐，从而有利于生成视频的平滑性。此外，我们还引入运动分解器与组合器以促进视频运动的操作。定性与定量实验表明，我们的方法能够生成闪烁更少的优质连续视频。