Many manipulation tasks require careful force modulation. With insufficient force the task may fail, while excessive force could cause damage. The high cost, bulky size and fragility of commercial force/torque (F/T) sensors have limited large-scale, force-aware policy learning. We introduce UMI-FT, a handheld data-collection platform that mounts compact, six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose. Using the multimodal data collected from this device, we train an adaptive compliance policy that predicts position targets, grasp force, and stiffness for execution on standard compliance controllers. In evaluations on three contact-rich, force-sensitive tasks (whiteboard wiping, skewering zucchini, and lightbulb insertion), UMI-FT enables policies that reliably regulate external contact forces and internal grasp forces, outperforming baselines that lack compliance or force sensing. UMI-FT offers a scalable path to learning compliant manipulation from in-the-wild demonstrations. We open-source the hardware and software to facilitate broader adoption at:https://umi-ft.github.io/.
翻译:许多操作任务需要精细的力调节。力不足可能导致任务失败,而过大的力则可能造成损坏。商用六维力/力矩传感器的成本高昂、体积庞大且易碎,这限制了大范围、力感知策略学习的发展。我们介绍了UMI-FT,一个手持式数据采集平台,它在每个手指上安装了紧凑的六维力/力矩传感器,能够实现手指级的力/力矩测量,同时采集RGB、深度和姿态数据。利用该设备收集的多模态数据,我们训练了一个自适应顺应性策略,该策略预测位置目标、抓握力和刚度,以便在标准的顺应性控制器上执行。在三个接触丰富且对力敏感的任务(白板擦拭、西葫芦串刺和灯泡安装)的评估中,UMI-FT使策略能够可靠地调节外部接触力和内部抓握力,其性能优于缺乏顺应性或力感知的基线方法。UMI-FT为从真实世界演示中学习顺应性操作提供了一条可扩展的途径。我们开源了硬件和软件以促进更广泛的采用,地址为:https://umi-ft.github.io/。