Robotic manipulation holds the potential to replace humans in the execution of tedious or dangerous tasks. However, control-based approaches are not suitable due to the difficulty of formally describing open-world manipulation in reality, and the inefficiency of existing learning methods. Thus, applying manipulation in a wide range of scenarios presents significant challenges. In this study, we propose a novel method for skill learning in robotic manipulation called Tactile Active Inference Reinforcement Learning (Tactile-AIRL), aimed at achieving efficient training. To enhance the performance of reinforcement learning (RL), we introduce active inference, which integrates model-based techniques and intrinsic curiosity into the RL process. This integration improves the algorithm's training efficiency and adaptability to sparse rewards. Additionally, we utilize a vision-based tactile sensor to provide detailed perception for manipulation tasks. Finally, we employ a model-based approach to imagine and plan appropriate actions through free energy minimization. Simulation results demonstrate that our method achieves significantly high training efficiency in non-prehensile objects pushing tasks. It enables agents to excel in both dense and sparse reward tasks with just a few interaction episodes, surpassing the SAC baseline. Furthermore, we conduct physical experiments on a gripper screwing task using our method, which showcases the algorithm's rapid learning capability and its potential for practical applications.

翻译：机器人操作具有替代人类执行枯燥或危险任务的潜力。然而，由于现实中难以形式化描述开放世界操作，且现有学习方法效率低下，基于控制的方法并不适用。因此，在广泛场景中应用操作技术仍面临重大挑战。本研究提出一种名为“触觉主动推断强化学习”（Tactile-AIRL）的新型机器人操作技能学习方法，旨在实现高效训练。为提升强化学习（RL）性能，我们引入主动推断机制，将基于模型的技术和内在好奇心融入RL过程。这种融合提高了算法的训练效率和对稀疏奖励的适应性。此外，我们利用基于视觉的触觉传感器为操作任务提供精细感知。最后，通过自由能最小化，采用基于模型的方法进行动作想象与规划。仿真结果表明，本方法在非抓取物体推搡任务中实现了极高的训练效率——仅需少量交互回合即可使智能体在密集和稀疏奖励任务中均表现优异，超越SAC基线。此外，我们使用该方法在夹爪拧螺丝任务上开展了实物实验，展示了算法的快速学习能力及其在实际应用中的潜力。