Learning contact-rich manipulation skills is essential. Such skills require the robots to interact with the environment with feasible manipulation trajectories and suitable compliance control parameters to enable safe and stable contact. However, learning these skills is challenging due to data inefficiency in the real world and the sim-to-real gap in simulation. In this paper, we introduce a hybrid offline-online framework to learn robust manipulation skills. We employ model-free reinforcement learning for the offline phase to obtain the robot motion and compliance control parameters in simulation \RV{with domain randomization}. Subsequently, in the online phase, we learn the residual of the compliance control parameters to maximize robot performance-related criteria with force sensor measurements in real time. To demonstrate the effectiveness and robustness of our approach, we provide comparative results against existing methods for assembly, pivoting, and screwing tasks.

翻译：学习接触丰富操作技能至关重要。此类技能要求机器人在可行操作轨迹和适当柔顺控制参数下与环境交互，以实现安全稳定的接触。然而，由于现实世界中的数据低效性以及仿真中的仿真到现实差异，学习这些技能极具挑战性。本文提出一种混合离线-在线框架，用于学习鲁棒的操作技能。我们在离线阶段采用无模型强化学习，在仿真中通过领域随机化获取机器人运动与柔顺控制参数。随后，在线阶段学习柔顺控制参数的残差，以基于力传感器测量实时最大化机器人性能相关准则。为展示我们方法的有效性和鲁棒性，我们针对装配、旋转和拧螺丝任务提供了与现有方法的对比结果。