Contact-rich tasks continue to present many challenges for robotic manipulation. In this work, we leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks that involve relative motion (e.g., slipping and sliding) between the end-effector and the manipulated object. We introduce two algorithmic contributions, tactile force matching and learned mode switching, as complimentary methods for improving IL. Tactile force matching enhances kinesthetic teaching by reading approximate forces during the demonstration and generating an adapted robot trajectory that recreates the recorded forces. Learned mode switching uses IL to couple visual and tactile sensor modes with the learned motion policy, simplifying the transition from reaching to contacting. We perform robotic manipulation experiments on four door-opening tasks with a variety of observation and algorithm configurations to study the utility of multimodal visuotactile sensing and our proposed improvements. Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%, emphasizing the value of see-through tactile sensing for IL, both for data collection to allow force matching, and for policy execution to enable accurate task feedback. Project site: https://papers.starslab.ca/sts-il .

翻译：接触密集型任务对机器人操作仍构成诸多挑战。本研究在多模态视觉触觉传感器框架下，结合模仿学习（IL）方法，执行涉及末端执行器与操作对象间相对运动（如滑移与滑动）的接触密集型任务。我们提出两项算法改进：触觉力匹配与学习型模式切换，作为提升模仿学习性能的互补方法。触觉力匹配通过演示阶段读取近似力数据并生成能复现记录力的自适应机器人轨迹，增强了动觉示教效果。学习型模式切换运用模仿学习将视觉与触觉传感器模式与习得的运动策略耦合，简化了从接近到接触的过渡过程。我们在四种开门任务上开展机器人操作实验，通过多种观测配置与算法组合，探究多模态视觉触觉感知及所提改进方法的效用。实验结果表明：力匹配使策略平均成功率提升62.5%，视觉触觉模式切换提升30.3%，而将视觉触觉数据作为策略输入则提升42.5%。这些数据凸显了透视触觉传感在模仿学习中的双重价值：既为支持力匹配的数据采集提供条件，也为策略执行阶段实现精确任务反馈奠定基础。项目主页：https://papers.starslab.ca/sts-il 。