Diffusion models have achieved great success in image generation. However, when leveraging this idea for video generation, we face significant challenges in maintaining the consistency and continuity across video frames. This is mainly caused by the lack of an effective framework to align frames of videos with desired temporal features while preserving consistent semantic and stochastic features. In this work, we propose a novel Sector-Shaped Diffusion Model (S2DM) whose sector-shaped diffusion region is formed by a set of ray-shaped reverse diffusion processes starting at the same noise point. S2DM can generate a group of intrinsically related data sharing the same semantic and stochastic features while varying on temporal features with appropriate guided conditions. We apply S2DM to video generation tasks, and explore the use of optical flow as temporal conditions. Our experimental results show that S2DM outperforms many existing methods in the task of video generation without any temporal-feature modelling modules. For text-to-video generation tasks where temporal conditions are not explicitly given, we propose a two-stage generation strategy which can decouple the generation of temporal features from semantic-content features. We show that, without additional training, our model integrated with another temporal conditions generative model can still achieve comparable performance with existing works. Our results can be viewd at https://s2dm.github.io/S2DM/.

翻译：扩散模型在图像生成领域取得了巨大成功。然而，将这一思想应用于视频生成时，我们在保持视频帧之间的一致性和连续性方面面临重大挑战。这主要是由于缺乏一个有效的框架，能在保留一致语义与随机特征的同时，使视频帧与所需的时间特征对齐。在本工作中，我们提出了一种新颖的扇形扩散模型（S2DM），其扇形扩散区域由一组从同一噪声点出发的射线状反向扩散过程构成。S2DM能够生成一组内在关联的数据——这些数据共享相同的语义与随机特征，而在适当引导条件下呈现出时间特征的变化。我们将S2DM应用于视频生成任务，并探索了将光流作为时间条件的使用方法。实验结果表明，S2DM在无需任何时间特征建模模块的情况下，在视频生成任务中优于许多现有方法。对于未显式给定时间条件的文生视频任务，我们提出了一种两阶段生成策略，可将时间特征的生成与语义内容特征的生成解耦。我们证明，无需额外训练，将我们的模型与另一个时间条件生成模型相结合，仍能取得与现有工作相当的性能。我们的结果可在https://s2dm.github.io/S2DM/查看。