We present DreamPose, a diffusion-based method for generating animated fashion videos from still images. Given an image and a sequence of human body poses, our method synthesizes a video containing both human and fabric motion. To achieve this, we transform a pretrained text-to-image model (Stable Diffusion) into a pose-and-image guided video synthesis model, using a novel finetuning strategy, a set of architectural changes to support the added conditioning signals, and techniques to encourage temporal consistency. We fine-tune on a collection of fashion videos from the UBC Fashion dataset. We evaluate our method on a variety of clothing styles and poses, and demonstrate that our method produces state-of-the-art results on fashion video animation. Video results are available on our project page.

翻译：我们提出了DreamPose，一种基于扩散模型的方法，用于从静态图像生成动画时尚视频。给定一张图像和一系列人体姿态序列，我们的方法能够合成同时包含人体运动与织物运动的视频。为实现这一目标，我们将预训练的文本到图像模型（稳定扩散）转化为融合姿态与图像引导的视频合成模型，采用新颖的微调策略、支持新增条件信号的架构变更以及增强时序一致性的技术。我们在UBC Fashion数据集的时尚视频集合上进行微调。我们在多种服装风格与姿态上评估了该方法，并展示了其在时尚视频动画方面达到的最优性能。视频结果可见于我们的项目页面。