This paper investigates humanoid whole-body dexterous manipulation, where the efficient collection of high-quality demonstration data remains a central bottleneck. Existing teleoperation systems often suffer from limited portability, occlusion, or insufficient precision, which hinders their applicability to complex whole-body tasks. To address these challenges, we introduce HumDex, a portable teleoperation system designed for humanoid whole-body dexterous manipulation. Our system leverages IMU-based motion tracking to address the portability-precision trade-off, enabling accurate full-body tracking while remaining easy to deploy. For dexterous hand control, we further introduce a learning-based retargeting method that generates smooth and natural hand motions without manual parameter tuning. Beyond teleoperation, HumDex enables efficient collection of human motion data. Building on this capability, we propose a two-stage imitation learning framework that first pre-trains on diverse human motion data to learn generalizable priors, and then fine-tunes on robot data to bridge the embodiment gap for precise execution. We demonstrate that this approach significantly improves generalization to new configurations, objects, and backgrounds with minimal data acquisition costs. The entire system is fully reproducible and open-sourced at https://github.com/physical-superintelligence-lab/HumDex.
翻译:本文研究人形机器人全身灵巧操作,其中高质量演示数据的高效采集仍是核心瓶颈。现有遥操作系统常受限于可移植性差、遮挡问题或精度不足,阻碍了其在复杂全身任务中的应用。为应对这些挑战,我们提出了HumDex——一个专为人形机器人全身灵巧操作设计的便携式遥操作系统。该系统采用基于IMU的运动追踪技术,在可移植性与精度之间取得平衡,在保持易部署性的同时实现精确全身追踪。针对灵巧手部控制,我们进一步提出基于学习的重定向方法,无需手动参数调整即可生成平滑自然的手部动作。除遥操作外,HumDex支持高效采集人体运动数据。基于此能力,我们提出两阶段模仿学习框架:首先在多样化人体运动数据上进行预训练以学习可泛化的先验知识,随后在机器人数据上进行微调以弥合具身差距实现精准执行。实验证明该方法能以极低数据采集成本,显著提升对新配置、新物体及新背景的泛化能力。整个系统完全可复现,已在https://github.com/physical-superintelligence-lab/HumDex开源。