In recent months, we witness a leap forward as denoising diffusion models were introduced to Motion Generation. Yet, the main gap in this field remains the low availability of data. Furthermore, the expensive acquisition process of motion biases the already modest data towards short single-person sequences. With such a shortage, more elaborate generative tasks are left behind. In this paper, we show that this gap can be mitigated using a pre-trained diffusion-based model as a generative prior. We demonstrate the prior is effective for fine-tuning, in a few-, and even a zero-shot manner. For the zero-shot setting, we tackle the challenge of long sequence generation. We introduce DoubleTake, an inference-time method with which we demonstrate up to 10-minute long animations of prompted intervals and their meaningful and controlled transition, using the prior that was trained for 10-second generations. For the few-shot setting, we consider two-person generation. Using two fixed priors and as few as a dozen training examples, we learn a slim communication block, ComMDM, to infuse interaction between the two resulting motions. Finally, using fine-tuning, we train the prior to semantically complete motions from a single prescribed joint. Then, we use our DiffusionBlending to blend a few such models into a single one that responds well to the combination of the individual control signals, enabling fine-grained joint- and trajectory-level control and editing. Using an off-the-shelf state-of-the-art (SOTA) motion diffusion model as a prior, we evaluate our approach for the three mentioned cases and show that we consistently outperform SOTA models that were designed and trained for those tasks.

翻译：近期，去噪扩散模型被引入运动生成领域，我们目睹了该领域的跨越式发展。然而，当前领域的主要瓶颈仍然是数据可用性不足。此外，运动数据昂贵的采集过程使得本已有限的数据进一步偏向于短时单人序列。在这种数据短缺的情况下，更复杂的生成任务被搁置。本文表明，利用预训练的扩散模型作为生成先验，可以缓解这一瓶颈。我们证明该先验在少量样本甚至零样本微调场景下均有效。针对零样本设置，我们解决了长序列生成的挑战，提出DoubleTake方法——通过该推理时方法，我们能够利用仅训练生成10秒序列的先验，生成长达10分钟的动画片段，并实现提示片段的语义化可控过渡。针对少量样本设置，我们探索了双人运动生成。通过固定两个先验模型并借助少量训练样本（仅需十几个示例），我们学习了一个轻量级通信模块ComMDM，以注入两个输出运动之间的交互。最后，通过微调，我们训练先验从单个指定关节出发完成运动语义补全，并利用DiffusionBlending将多个此类模型融合为单一模型，使其能响应个体控制信号的组合，从而实现细粒度的关节级与轨迹级控制与编辑。我们以现成的SOTA运动扩散模型作为先验，在上述三种场景下评估方法，结果表明我们始终优于为这些任务专门设计和训练的SOTA模型。