We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors. Since human motions are highly diverse and have a property of quite different distribution from conditional modalities, such as textual descriptors in natural languages, it is hard to learn a probabilistic mapping from the desired conditional modality to the human motion sequences. Besides, the raw motion data from the motion capture system might be redundant in sequences and contain noises; directly modeling the joint distribution over the raw motion sequences and conditional modalities would need a heavy computational overhead and might result in artifacts introduced by the captured noises. To learn a better representation of the various human motion sequences, we first design a powerful Variational AutoEncoder (VAE) and arrive at a representative and low-dimensional latent code for a human motion sequence. Then, instead of using a diffusion model to establish the connections between the raw motion sequences and the conditional inputs, we perform a diffusion process on the motion latent space. Our proposed Motion Latent-based Diffusion model (MLD) could produce vivid motion sequences conforming to the given conditional inputs and substantially reduce the computational overhead in both the training and inference stages. Extensive experiments on various human motion generation tasks demonstrate that our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks, with two orders of magnitude faster than previous diffusion models on raw motion sequences.

翻译：我们研究了一项具有挑战性的任务——条件式人体运动生成，该任务需要根据动作类别或文本描述等不同条件输入，生成合理的人体运动序列。由于人体运动具有高度多样性，且其分布特性与条件模态（如自然语言文本描述）存在显著差异，因此难以学习从条件模态到人体运动序列的概率映射。此外，运动捕捉系统采集的原始运动数据可能存在序列冗余和噪声；直接对原始运动序列与条件模态的联合分布进行建模将导致高昂的计算开销，并可能因捕捉噪声引入伪影。为更好地表征多样的人体运动序列，我们首先设计了强大的变分自编码器（Variational AutoEncoder, VAE），为人体运动序列提取出具有代表性且低维的潜在编码。随后，我们并未采用扩散模型建立原始运动序列与条件输入之间的直接关联，而是在运动潜在空间中执行扩散过程。所提出的基于运动潜在空间的扩散模型（Motion Latent-based Diffusion model, MLD）能够生成符合给定条件输入的生动画运动序列，并在训练与推理阶段大幅降低计算开销。针对多种人体运动生成任务的广泛实验表明，我们的MLD在众多人体运动生成任务中均显著超越现有最优方法，且其推理速度相较于基于原始运动序列的扩散模型提升两个数量级。