Kinesthetic Teaching is a popular approach to collecting expert robotic demonstrations of contact-rich tasks for imitation learning (IL), but it typically only measures motion, ignoring the force placed on the environment by the robot. Furthermore, contact-rich tasks require accurate sensing of both reaching and touching, which can be difficult to provide with conventional sensing modalities. We address these challenges with a See-Through-your-Skin (STS) visuotactile sensor, using the sensor both (i) as a measurement tool to improve kinesthetic teaching, and (ii) as a policy input in contact-rich door manipulation tasks. An STS sensor can be switched between visual and tactile modes by leveraging a semi-transparent surface and controllable lighting, allowing for both pre-contact visual sensing and during-contact tactile sensing with a single sensor. First, we propose tactile force matching, a methodology that enables a robot to match forces read during kinesthetic teaching using tactile signals. Second, we develop a policy that controls STS mode switching, allowing a policy to learn the appropriate moment to switch an STS from its visual to its tactile mode. Finally, we study multiple observation configurations to compare and contrast the value of visual and tactile data from an STS with visual data from a wrist-mounted eye-in-hand camera. With over 3,000 test episodes from real-world manipulation experiments, we find that the inclusion of force matching raises average policy success rates by 62.5%, STS mode switching by 30.3%, and STS data as a policy input by 42.5%. Our results highlight the utility of see-through tactile sensing for IL, both for data collection to allow force matching, and for policy execution to allow accurate task feedback.

翻译：动觉教学是一种流行的用于收集接触密集型任务的机器人专家示范以进行模仿学习的方法，但通常仅测量运动，忽略了机器人施加在环境上的力。此外，接触密集型任务需要精确感知接触与触碰，而传统传感模态难以提供这种能力。我们通过一种透视皮肤（STS）视触觉传感器解决这些挑战，将传感器同时用作：（i）改进动觉教学的测量工具，以及（ii）接触密集型门操作任务中的策略输入。STS传感器利用半透明表面和可控照明，可在视觉与触觉模式之间切换，从而实现单个传感器的接触前视觉感知与接触中触觉感知。首先，我们提出触觉力匹配方法，使机器人能够利用触觉信号匹配动觉教学过程中读取的力。其次，我们开发了一种控制STS模式切换的策略，使策略能够学习将STS从视觉模式切换到触觉模式的适当时机。最后，我们研究了多种观测配置，以比较STS的视觉与触觉数据与腕部安装的眼在手摄像头视觉数据的价值。基于超过3000次真实世界操作实验测试，我们发现加入力匹配使平均策略成功率提高62.5%，STS模式切换提高30.3%，将STS数据作为策略输入提高42.5%。我们的结果突显了透视触觉传感在模仿学习中的实用性——既可用于数据采集以实现力匹配，也可用于策略执行以提供准确的任务反馈。