We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors. Since human motions are highly diverse and have a property of quite different distribution from conditional modalities, such as textual descriptors in natural languages, it is hard to learn a probabilistic mapping from the desired conditional modality to the human motion sequences. Besides, the raw motion data from the motion capture system might be redundant in sequences and contain noises; directly modeling the joint distribution over the raw motion sequences and conditional modalities would need a heavy computational overhead and might result in artifacts introduced by the captured noises. To learn a better representation of the various human motion sequences, we first design a powerful Variational AutoEncoder (VAE) and arrive at a representative and low-dimensional latent code for a human motion sequence. Then, instead of using a diffusion model to establish the connections between the raw motion sequences and the conditional inputs, we perform a diffusion process on the motion latent space. Our proposed Motion Latent-based Diffusion model (MLD) could produce vivid motion sequences conforming to the given conditional inputs and substantially reduce the computational overhead in both the training and inference stages. Extensive experiments on various human motion generation tasks demonstrate that our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks, with two orders of magnitude faster than previous diffusion models on raw motion sequences.

翻译：我们研究一项具有挑战性的任务——条件性人体运动生成，该任务根据动作类别或文本描述等不同条件输入生成合理的人体运动序列。由于人体运动具有高度多样性，且其分布与条件模态（如自然语言中的文本描述）存在显著差异，因此难以学习从期望条件模态到人体运动序列的概率映射。此外，动作捕捉系统采集的原始运动数据可能在序列中存在冗余和噪声；直接对原始运动序列与条件模态的联合分布进行建模将导致巨大的计算开销，并可能因捕获噪声引入伪影。为学习各类人体运动序列的更优表示，我们首先设计了一个强大的变分自编码器（VAE），为人体运动序列获得具有代表性且低维度的潜在编码。随后，我们并非使用扩散模型建立原始运动序列与条件输入之间的连接，而是在运动潜在空间执行扩散过程。我们提出的基于运动潜空间的扩散模型（MLD）能够生成符合给定条件输入的逼真运动序列，并在训练和推理阶段显著降低计算开销。在各类人体运动生成任务上的大量实验表明，我们的MLD在多个生成任务中均显著优于现有最优方法，且速度相比以往处理原始运动序列的扩散模型提升两个数量级。