With the advance of text-to-image models (e.g., Stable Diffusion) and corresponding personalization techniques such as DreamBooth and LoRA, everyone can manifest their imagination into high-quality images at an affordable cost. Subsequently, there is a great demand for image animation techniques to further combine generated static images with motion dynamics. In this report, we propose a practical framework to animate most of the existing personalized text-to-image models once and for all, saving efforts in model-specific tuning. At the core of the proposed framework is to insert a newly initialized motion modeling module into the frozen text-to-image model and train it on video clips to distill reasonable motion priors. Once trained, by simply injecting this motion modeling module, all personalized versions derived from the same base T2I readily become text-driven models that produce diverse and personalized animated images. We conduct our evaluation on several public representative personalized text-to-image models across anime pictures and realistic photographs, and demonstrate that our proposed framework helps these models generate temporally smooth animation clips while preserving the domain and diversity of their outputs. Code and pre-trained weights will be publicly available at https://animatediff.github.io/ .

翻译：随着文本到图像模型（如Stable Diffusion）及相应个性化技术（如DreamBooth和LoRA）的发展，每个人都能以可负担的成本将想象力转化为高质量图像。随后，对图像动画技术的需求日益增长，以进一步将生成的静态图像与运动动态相结合。在本报告中，我们提出了一种实用框架，可一劳永逸地将大多数现有个性化文本到图像模型动画化，从而节省模型特定调优的工作量。该框架的核心是在冻结的文本到图像模型中插入一个新初始化的运动建模模块，并在视频片段上训练该模块以提取合理的运动先验。一旦训练完成，只需注入此运动建模模块，所有源自同一基础T2I模型的个性化版本即可轻松转变为能够生成多样化且个性化动画图像的文本驱动模型。我们在多个代表性公开个性化文本到图像模型（涵盖动漫图片和真实照片）上进行了评估，结果表明我们提出的框架有助于这些模型生成时间上平滑的动画片段，同时保留其输出领域和多样性。代码和预训练权重将在https://animatediff.github.io/上公开提供。