The recently emerging text-to-motion advances have spired numerous attempts for convenient and interactive human motion generation. Yet, existing methods are largely limited to generating body motions only without considering the rich two-hand motions, let alone handling various conditions like body dynamics or texts. To break the data bottleneck, we propose BOTH57M, a novel multi-modal dataset for two-hand motion generation. Our dataset includes accurate motion tracking for the human body and hands and provides pair-wised finger-level hand annotations and body descriptions. We further provide a strong baseline method, BOTH2Hands, for the novel task: generating vivid two-hand motions from both implicit body dynamics and explicit text prompts. We first warm up two parallel body-to-hand and text-to-hand diffusion models and then utilize the cross-attention transformer for motion blending. Extensive experiments and cross-validations demonstrate the effectiveness of our approach and dataset for generating convincing two-hand motions from the hybrid body-and-textual conditions. Our dataset and code will be disseminated to the community for future research.

翻译：近期兴起的文本-运动技术极大推动了便捷交互式人体运动生成的研究。然而现有方法主要局限于仅生成人体运动，既未考虑丰富的双手运动，更无法处理人体动态或文本等多重条件。为突破数据瓶颈，我们提出BOTH57M——一个面向双手运动生成的新型多模态数据集。该数据集包含人体与双手的精确运动捕捉，并提供成对的指级手部标注与人体描述。我们进一步提出强基线方法BOTH2Hands，以完成全新任务：从隐式人体动态与显式文本提示中生成逼真的双手运动。首先预热两个并行的身体-手部与文本-手部扩散模型，继而利用交叉注意力Transformer进行运动融合。大量实验与交叉验证表明，本方法与数据集在混合身体-文本条件下生成可信双手运动具有显著有效性。数据集与代码将向社区公开以供后续研究。