The recently emerging text-to-motion advances have spired numerous attempts for convenient and interactive human motion generation. Yet, existing methods are largely limited to generating body motions only without considering the rich two-hand motions, let alone handling various conditions like body dynamics or texts. To break the data bottleneck, we propose BOTH57M, a novel multi-modal dataset for two-hand motion generation. Our dataset includes accurate motion tracking for the human body and hands and provides pair-wised finger-level hand annotations and body descriptions. We further provide a strong baseline method, BOTH2Hands, for the novel task: generating vivid two-hand motions from both implicit body dynamics and explicit text prompts. We first warm up two parallel body-to-hand and text-to-hand diffusion models and then utilize the cross-attention transformer for motion blending. Extensive experiments and cross-validations demonstrate the effectiveness of our approach and dataset for generating convincing two-hand motions from the hybrid body-and-textual conditions. Our dataset and code will be disseminated to the community for future research.

翻译：近期文本生成动作技术的进展推动了诸多便捷、交互式人体动作生成的研究。然而，现有方法大多局限于仅生成身体动作，未能考虑丰富的双手运动，更无需提及处理身体动力学或文本等多种条件。为突破数据瓶颈，我们提出了BOTH57M——一个针对双手运动生成的新型多模态数据集。该数据集包含人体与手部的精确运动跟踪，并提供成对的指级手部标注与身体描述。进一步地，我们为这一新任务提供了强基线方法BOTH2Hands：从隐式身体动力学与显式文本提示生成逼真的双手运动。我们首先预热两个并行的身体到手部与文本到手部的扩散模型，随后利用交叉注意力Transformer进行运动混合。大量实验与交叉验证表明，我们的方法与数据集能够有效从混合身体与文本条件中生成令人信服的双手运动。数据集与代码将向社区公开，以支持未来研究。