Contact-rich tasks continue to present a variety of challenges for robotic manipulation. In this work, we leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact rich tasks that involve relative motion (slipping/sliding) between the end-effector and object. We introduce two algorithmic contributions, tactile force matching and learned mode switching, as complimentary methods for improving IL. Tactile force matching enhances kinesthetic teaching by reading approximate forces during the demonstration and generating an adapted robot trajectory that recreates the recorded forces. Learned mode switching uses IL to couple visual and tactile sensor modes with the learned motion policy, simplifying the transition from reaching to contacting. We perform robotic manipulation experiments on four door opening tasks with a variety of observation and method configurations to study the utility of our proposed improvements and multimodal visuotactile sensing. Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%, emphasizing the value of see-through tactile sensing for IL, both for data collection to allow force matching, and for policy execution to allow accurate task feedback.

翻译：接触密集型任务持续为机器人操作带来诸多挑战。本研究在模仿学习框架内，利用多模态视觉触觉传感器执行涉及末端执行器与物体间相对运动（滑移/滑动）的接触密集型任务。我们提出两项算法贡献：触觉力匹配与学习型模式切换，作为提升模仿学习的互补方法。触觉力匹配通过演示期间读取近似力数据并生成能复现记录力的自适应机器人轨迹，增强了动觉示教效果。学习型模式切换运用模仿学习将视觉与触觉传感器模式与习得的运动策略相耦合，简化了从接近到接触的过渡过程。我们在四种开门任务上开展机器人操作实验，通过多种观测与方法配置研究我们提出的改进方案及多模态视觉触觉感知的效用。实验结果表明：力匹配使平均策略成功率提升62.5%，视觉触觉模式切换提升30.3%，而将视觉触觉数据作为策略输入则提升42.5%。这些数据凸显了透射式触觉感知在模仿学习中的价值——既为支持力匹配的数据采集提供条件，也为策略执行阶段实现精确任务反馈奠定基础。