Robotic manipulation can benefit from wrist-mounted force/torque (F/T) sensors, but conventional F/T sensors can be expensive, difficult to install, and damaged by high loads. We present Visual Force/Torque Sensing (VFTS), a method that visually estimates the 6-axis F/T measurement that would be reported by a conventional F/T sensor. In contrast to approaches that sense loads using internal cameras placed behind soft exterior surfaces, our approach uses an external camera with a fisheye lens that observes a soft gripper. VFTS includes a deep learning model that takes a single RGB image as input and outputs a 6-axis F/T estimate. We trained the model with sensor data collected while teleoperating a robot (Stretch RE1 from Hello Robot Inc.) to perform manipulation tasks. VFTS outperformed F/T estimates based on motor currents, generalized to a novel home environment, and supported three autonomous tasks relevant to healthcare: grasping a blanket, pulling a blanket over a manikin, and cleaning a manikin's limbs. VFTS also performed well with a manually operated pneumatic gripper. Overall, our results suggest that an external camera observing a soft gripper can perform useful visual force/torque sensing for a variety of manipulation tasks.

翻译：机器人操作可从腕部力/力矩传感器中受益，但传统力/力矩传感器成本高、安装困难且易被高负载损坏。本文提出视觉力/力矩传感方法，通过视觉方式估计传统力/力矩传感器可能输出的六轴力/力矩测量值。与采用内部相机置于软性外表面后方进行负载感测的方法不同，本方法使用配备鱼眼镜头的观察软体抓取器的外部相机。视觉力/力矩传感包含一个深度学习模型，该模型以单张RGB图像为输入，输出六轴力/力矩估计值。我们利用远程操作机器人进行操控任务时采集的传感器数据对模型进行训练。视觉力/力矩传感在电机电流力/力矩估计、泛化至新家庭环境等任务中表现更优，并支持三项医疗相关自主任务：抓取毯子、将毯子覆盖至人体模型、清洁人体模型四肢。该方法在手动操作气动抓取器时同样表现良好。总体而言，实验结果表明，通过外部相机观察软体抓取器，可为多种操作任务提供有效的视觉力/力矩传感。