Text-to-motion generation is an emerging and challenging problem, which aims to synthesize motion with the same semantics as the input text. However, due to the lack of diverse labeled training data, most approaches either limit to specific types of text annotations or require online optimizations to cater to the texts during inference at the cost of efficiency and stability. In this paper, we investigate offline open-vocabulary text-to-motion generation in a zero-shot learning manner that neither requires paired training data nor extra online optimization to adapt for unseen texts. Inspired by the prompt learning in NLP, we pretrain a motion generator that learns to reconstruct the full motion from the masked motion. During inference, instead of changing the motion generator, our method reformulates the input text into a masked motion as the prompt for the motion generator to ``reconstruct'' the motion. In constructing the prompt, the unmasked poses of the prompt are synthesized by a text-to-pose generator. To supervise the optimization of the text-to-pose generator, we propose the first text-pose alignment model for measuring the alignment between texts and 3D poses. And to prevent the pose generator from overfitting to limited training texts, we further propose a novel wordless training mechanism that optimizes the text-to-pose generator without any training texts. The comprehensive experimental results show that our method obtains a significant improvement against the baseline methods. The code is available at https://github.com/junfanlin/oohmg.

翻译：文本到动作生成是一个新兴且具有挑战性的问题，旨在合成与输入文本具有相同语义的动作。然而，由于缺乏多样化的标注训练数据，大多数方法要么局限于特定类型的文本注释，要么需要在推理过程中进行在线优化以适应文本，但牺牲了效率和稳定性。本文以零样本学习方式研究离线开集文本到动作生成，既不需要配对训练数据，也不需要额外的在线优化来适应未见过的文本。受自然语言处理中提示学习的启发，我们预训练了一个动作生成器，使其学会从掩码动作中重建完整动作。在推理过程中，我们的方法不改变动作生成器，而是将输入文本转化为一个掩码动作，作为动作生成器的提示以“重建”动作。在构建提示时，提示中的非掩码姿态由文本到姿态生成器合成。为了监督文本到姿态生成器的优化，我们提出了首个用于衡量文本与三维姿态对齐程度的文本-姿态对齐模型。为了防止姿态生成器对有限训练文本过拟合，我们进一步提出了一种新颖的无词训练机制，该机制在没有训练文本的情况下优化文本到姿态生成器。综合实验结果表明，我们的方法相比基线方法取得了显著提升。代码可在 https://github.com/junfanlin/oohmg 获取。