Teleoperation is a key approach for collecting high-quality, physically consistent demonstrations for robotic manipulation. However, teleoperation for dexterous manipulation remains constrained by: (i) inaccurate hand-robot motion mapping, which limits teleoperated dexterity, and (ii) limited tactile feedback that forces vision-dominated interaction and hinders perception of contact geometry and force variation. To address these challenges, we present TAG, a low-cost glove system that integrates precise hand motion capture with high-resolution tactile feedback, enabling effective tactile-in-the-loop dexterous teleoperation. For motion capture, TAG employs a non-contact magnetic sensing design that provides drift-free, electromagnetically robust 21-DoF joint tracking with joint angle estimation errors below 1 degree. Meanwhile, to restore tactile sensation, TAG equips each finger with a 32-actuator tactile array within a compact 2 cm^2 module, allowing operators to directly feel physical interactions at the robot end-effector through spatial activation patterns. Through real-world teleoperation experiments and user studies, we show that TAG enables reliable real-time perception of contact geometry and dynamic force, improves success rates in contact-rich teleoperation tasks, and increases the reliability of demonstration data collection for learning-based manipulation.
翻译:遥操作是收集机器人操作中高质量、物理一致演示的关键方法。然而,灵巧操作的遥操作仍受以下限制:(i)手-机器人运动映射不准确限制了遥操作的灵巧性,(ii)有限的触觉反馈迫使交互以视觉为主导,并阻碍了对接触几何形状和力变化的感知。为应对这些挑战,我们提出TAG,一种低成本手套系统,它集成了精确的手部运动捕捉与高分辨率触觉反馈,实现了有效的触觉在环灵巧遥操作。在运动捕捉方面,TAG采用非接触式磁传感设计,提供无漂移、电磁鲁棒的21自由度关节跟踪,关节角度估计误差低于1度。同时,为恢复触觉感知,TAG在每根手指上配备一个2平方厘米紧凑模块内的32执行器触觉阵列,使操作者能够通过空间激活模式直接感受机器人末端执行器的物理交互。通过真实世界的遥操作实验和用户研究,我们展示TAG能够可靠地实时感知接触几何形状和动态力,提高接触密集型遥操作任务的成功率,并增加基于学习的操作中演示数据收集的可靠性。