Recent diffusion-based methods have achieved impressive results on animating images of human subjects. However, most of that success has built on human-specific body pose representations and extensive training with labeled real videos. In this work, we extend the ability of such models to animate images of characters with more diverse skeletal topologies. Given a small number (3-5) of example frames showing the character in different poses with corresponding skeletal information, our model quickly infers a rig for that character that can generate images corresponding to new skeleton poses. We propose a procedural data generation pipeline that efficiently samples training data with diverse topologies on the fly. We use it, along with a novel skeleton representation, to train our model on articulated shapes spanning a large space of textures and topologies. Then during fine-tuning, our model rapidly adapts to unseen target characters and generalizes well to rendering new poses, both for realistic and more stylized cartoon appearances. To better evaluate performance on this novel and challenging task, we create the first 2D video dataset that contains both humanoid and non-humanoid subjects with per-frame keypoint annotations. With extensive experiments, we demonstrate the superior quality of our results. Project page: https://traindragondiffusion.github.io/

翻译：近年来，基于扩散的方法在人体图像动画生成方面取得了令人瞩目的成果。然而，这些成功大多建立在针对人体特定的姿态表示以及大量标注真实视频训练的基础上。本研究将此类模型的动画生成能力扩展至具有更复杂骨骼拓扑结构的角色图像。给定少量（3-5帧）展示角色在不同姿态下并附带对应骨骼信息的示例帧，我们的模型能够快速推断出该角色的绑定系统，从而生成与全新骨骼姿态相对应的图像。我们提出了一种程序化数据生成流程，能够动态高效地采样具有多样化拓扑结构的训练数据。结合新颖的骨骼表示方法，我们利用该流程在涵盖广阔纹理与拓扑空间的铰接形状数据集上训练模型。在微调阶段，我们的模型能够快速适应未见过的目标角色，并在渲染新姿态时展现出优异的泛化能力，同时适用于写实风格与更具风格化的卡通外观。为更好地评估这一新颖且具有挑战性任务的性能，我们创建了首个包含人形与非人形角色、且每帧均有关键点标注的二维视频数据集。通过大量实验，我们证明了所提出方法在生成质量上的优越性。项目页面：https://traindragondiffusion.github.io/