In robotics and artificial intelligence, the integration of tactile processing is becoming increasingly pivotal, especially in learning to execute intricate tasks like alignment and insertion. However, existing works focusing on tactile methods for insertion tasks predominantly rely on robot teleoperation data and reinforcement learning, which do not utilize the rich insights provided by human's control strategy guided by tactile feedback. For utilizing human sensations, methodologies related to learning from humans predominantly leverage visual feedback, often overlooking the invaluable tactile feedback that humans inherently employ to finish complex manipulations. Addressing this gap, we introduce "MimicTouch", a novel framework that mimics human's tactile-guided control strategy. In this framework, we initially collect multi-modal tactile datasets from human demonstrators, incorporating human tactile-guided control strategies for task completion. The subsequent step involves instructing robots through imitation learning using multi-modal sensor data and retargeted human motions. To further mitigate the embodiment gap between humans and robots, we employ online residual reinforcement learning on the physical robot. Through comprehensive experiments, we validate the safety of MimicTouch in transferring a latent policy learned through imitation learning from human to robot. This ongoing work will pave the way for a broader spectrum of tactile-guided robotic applications.

翻译：在机器人学和人工智能领域，触觉处理的集成正变得愈发关键，尤其是在学习执行对齐与插入等精细任务时。然而，现有专注于触觉方法处理插入任务的研究主要依赖机器人遥操作数据和强化学习，并未充分利用人类在触觉反馈引导下所提供的丰富控制策略。为了利用人类感知，以人类为中心的学习方法主要依赖视觉反馈，往往忽视了人类在完成复杂操作时自然运用的宝贵触觉反馈。为填补这一空白，我们提出了"MimicTouch"——一个模仿人类触觉引导控制策略的新型框架。在该框架中，我们首先从人类演示者处收集多模态触觉数据集，其中包含人类在完成任务时采用的触觉引导控制策略。随后，利用多模态传感器数据和重定向的人体运动，通过模仿学习引导机器人进行学习。为进一步缩小人类与机器人之间的具身差距，我们在实体机器人上采用了在线残差强化学习。通过综合实验，我们验证了MimicTouch在将模仿学习得到的潜在策略从人类安全迁移至机器人方面的有效性。这项正在进行的工作将为更广泛的触觉引导机器人应用铺平道路。