Contact-rich manipulation depends on applying the correct grasp forces throughout the manipulation task, especially when handling fragile or deformable objects. Most existing imitation learning approaches often treat visuotactile feedback only as an additional observation, leaving applied forces as an uncontrolled consequence of gripper commands. In this work, we present Force-Aware Robotic Manipulation (FARM), an imitation learning framework that integrates high-dimensional tactile data to infer tactile-conditioned force signals, which in turn define a matching force-based action space. We collect human demonstrations using a modified version of the handheld Universal Manipulation Interface (UMI) gripper that integrates a GelSight Mini visual tactile sensor. For deploying the learned policies, we developed an actuated variant of the UMI gripper with geometry matching our handheld version. During policy rollouts, the proposed FARM diffusion policy jointly predicts robot pose, grip width, and grip force. FARM outperforms several baselines across three tasks with distinct force requirements -- high-force, low-force, and dynamic force adaptation -- demonstrating the advantages of its two key components: leveraging force-grounded, high-dimensional tactile observations and a force-based control space. The codebase and design files are open-sourced and available at https://tactile-farm.github.io .
翻译:接触密集型操作任务依赖于在整个操作过程中施加恰当的抓取力,这在处理易碎或可变形物体时尤为重要。现有的大多数模仿学习方法通常仅将视觉触觉反馈视为附加观测信息,导致施加的力成为夹爪指令的不可控结果。本研究提出力感知机器人操作框架,该模仿学习框架整合高维触觉数据来推断触觉条件化的力信号,进而定义与之匹配的基于力的动作空间。我们使用集成了GelSight Mini视觉触觉传感器的改进版手持通用操作接口夹爪收集人类演示数据。为部署学习得到的策略,我们开发了与手持版本几何匹配的驱动式通用操作接口夹爪变体。在策略执行过程中,所提出的力感知机器人操作扩散策略联合预测机器人位姿、夹爪宽度和夹持力。该框架在具有不同力需求(高力、低力和动态力适应)的三项任务中均优于多个基线方法,验证了其两个关键组件的优势:利用基于力的高维触觉观测和基于力的控制空间。代码库与设计文件均已开源,可通过https://tactile-farm.github.io获取。