The excellent text-to-image synthesis capability of diffusion models has driven progress in synthesizing coherent visual stories. The current state-of-the-art method combines the features of historical captions, historical frames, and the current captions as conditions for generating the current frame. However, this method treats each historical frame and caption as the same contribution. It connects them in order with equal weights, ignoring that not all historical conditions are associated with the generation of the current frame. To address this issue, we propose Causal-Story. This model incorporates a local causal attention mechanism that considers the causal relationship between previous captions, frames, and current captions. By assigning weights based on this relationship, Causal-Story generates the current frame, thereby improving the global consistency of story generation. We evaluated our model on the PororoSV and FlintstonesSV datasets and obtained state-of-the-art FID scores, and the generated frames also demonstrate better storytelling in visuals. The source code of Causal-Story can be obtained from https://github.com/styufo/Causal-Story.

翻译：扩散模型在文本到图像合成方面的卓越能力推动了连贯视觉故事合成的进展。当前最先进的方法将历史描述、历史帧和当前描述的特征相结合，作为生成当前帧的条件。然而，该方法将每个历史帧和描述视为同等贡献，以相等的权重按顺序连接它们，忽略了并非所有历史条件都与当前帧的生成相关。为解决这一问题，我们提出了Causal-Story模型。该模型引入了一种局部因果注意力机制，考虑了先前描述、帧与当前描述之间的因果关系。通过基于该关系分配权重，Causal-Story生成当前帧，从而提升故事生成的全局一致性。我们在PororoSV和FlintstonesSV数据集上评估了该模型，取得了最先进的FID分数，生成的帧在视觉上也展现出更优的故事叙述能力。Causal-Story的源代码可从https://github.com/styufo/Causal-Story获取。