Tactile information is a critical tool for fine-grain manipulation. As humans, we rely heavily on tactile information to understand objects in our environments and how to interact with them. We use touch not only to perform manipulation tasks but also to learn how to perform these tasks. Therefore, to create robotic agents that can learn to complete manipulation tasks at a human or super-human level of performance, we need to properly incorporate tactile information into both skill execution and skill learning. In this paper, we investigate how we can incorporate tactile information into imitation learning platforms to improve performance on complex tasks. To do this, we tackle the challenge of plugging in a USB cable, a dexterous manipulation task that relies on fine-grain visuo-tactile serving. By incorporating tactile information into imitation learning frameworks, we are able to train a robotic agent to plug in a USB cable - a first for imitation learning. Additionally, we explore how tactile information can be used to train non-tactile agents through a contrastive-loss pretraining process. Our results show that by pretraining with tactile information, the performance of a non-tactile agent can be significantly improved, reaching a level on par with visuo-tactile agents. For demonstration videos and access to our codebase, see the project website: https://sites.google.com/andrew.cmu.edu/visuo-tactile-cable-plugging/home

翻译：触觉信息是精细操作的关键工具。作为人类，我们严重依赖触觉信息来理解环境中的物体以及如何与之交互。我们不仅利用触觉执行操作任务，还通过触觉学习如何完成这些任务。因此，为了构建能够以人类或超人类水平学习并完成操作任务的机器人智能体，我们需要将触觉信息恰当地融入技能执行与技能学习过程中。本文研究如何将触觉信息整合到模仿学习平台中，以提升复杂任务的表现。为此，我们攻克了USB电缆插拔这一依赖精细触觉-视觉伺服控制的灵巧操作难题。通过将触觉信息引入模仿学习框架，我们成功训练机器人智能体完成USB电缆插拔——这首次实现了模仿学习在此类任务中的应用。此外，我们探索了通过对比损失预训练过程，利用触觉信息训练非触觉智能体的方法。结果表明，通过触觉信息预训练，非触觉智能体的性能可显著提升，达到与触觉-视觉智能体相当的水平。演示视频及代码库访问请参见项目网站：https://sites.google.com/andrew.cmu.edu/visuo-tactile-cable-plugging/home