In robotics and artificial intelligence, the integration of tactile processing is becoming increasingly pivotal, especially in learning to execute intricate tasks like alignment and insertion. However, existing works focusing on tactile methods for insertion tasks predominantly rely on robot teleoperation data and reinforcement learning, which do not utilize the rich insights provided by human's control strategy guided by tactile feedback. For utilizing human sensations, methodologies related to learning from humans predominantly leverage visual feedback, often overlooking the invaluable tactile feedback that humans inherently employ to finish complex manipulations. Addressing this gap, we introduce "MimicTouch", a novel framework that mimics human's tactile-guided control strategy. In this framework, we initially collect multi-modal tactile datasets from human demonstrators, incorporating human tactile-guided control strategies for task completion. The subsequent step involves instructing robots through imitation learning using multi-modal sensor data and retargeted human motions. To further mitigate the embodiment gap between humans and robots, we employ online residual reinforcement learning on the physical robot. Through comprehensive experiments, we validate the safety of MimicTouch in transferring a latent policy learned through imitation learning from human to robot. This ongoing work will pave the way for a broader spectrum of tactile-guided robotic applications.

翻译：在机器人学和人工智能领域，触觉处理的融合正变得日益关键，尤其是在学习执行对齐、插入等精细操作任务时。然而，现有针对插入任务的触觉方法主要依赖机器人遥操作数据和强化学习，未能充分利用人类在触觉反馈指导下所蕴含的丰富控制策略。在利用人类感知方面，现有的人机学习方法主要依赖视觉反馈，常忽视人类在完成复杂操作时本能使用的宝贵触觉反馈。为弥补这一不足，我们提出"MimicTouch"——一种模仿人类触觉引导控制策略的新型框架。在该框架中，我们首先从人类演示者处采集多模态触觉数据集，其中包含人类为完成任务所采用的触觉引导控制策略；随后，通过使用多模态传感器数据和重定向的人体动作，指导机器人进行模仿学习。为进一步缩小人与机器人之间的具身差异，我们还在实体机器人上应用在线残差强化学习。通过全面实验，我们验证了MimicTouch在将从人类通过模仿学习获得的隐策略安全迁移至机器人方面的有效性。这项正在进行的研究将为更广泛的触觉引导型机器人应用奠定基础。