In this paper, a novel switching pushing skill algorithm is proposed to improve the efficiency of planar non-prehensile manipulation, which draws inspiration from human pushing actions and comprises two sub-problems, i.e., discrete decision-making of pushing point and continuous feedback control of pushing action. In order to solve the sub-problems above, a combination of Model Predictive Control (MPC) and Deep Reinforcement Learning (DRL) method is employed. Firstly, the selection of pushing point is modeled as a Markov decision process,and an off-policy DRL method is used by reshaping the reward function to train the decision-making model for selecting pushing point from a pre-constructed set based on the current state. Secondly, a motion constraint region (MCR) is constructed for the specific pushing point based on the distance from the target, followed by utilizing the MPC controller to regulate the motion of the object within the MCR towards the target pose. The trigger condition for switching the pushing point occurs when the object reaches the boundary of the MCR under the pushing action. Subsequently, the pushing point and the controller are updated iteratively until the target pose is reached. We conducted pushing experiments on four distinct object shapes in both simulated and physical environments to evaluate our method. The results indicate that our method achieves a significantly higher training efficiency, with a training time that is only about 20% of the baseline method while maintaining around the same success rate. Moreover, our method outperforms the baseline method in terms of both training and execution efficiency of pushing operations, allowing for rapid learning of robot pushing skills.

翻译：本文提出一种新颖的切换推动技能算法，旨在提升平面非预抓取操控的效率。该算法受人类推动动作启发，包含两个子问题：推动点的离散决策与推动动作的连续反馈控制。为解决上述子问题，本文采用模型预测控制与深度强化学习相结合的方法。首先，将推动点选择建模为马尔可夫决策过程，通过重塑奖励函数，采用离策略深度强化学习方法训练决策模型，使其能基于当前状态从预构建的候选集中选择推动点。其次，针对特定推动点，根据与目标距离构建运动约束区域，并利用MPC控制器在MCR内调控物体运动至目标位姿。当物体在推动作用下到达MCR边界时，触发推动点切换条件。随后迭代更新推动点和控制器，直至达到目标位姿。我们在模拟与物理环境中对四种不同形状物体进行了推动实验。结果表明，本方法在保持相近成功率的同时，训练效率显著提升，训练时间仅为基线方法的约20%。此外，在推动操作的训练与执行效率方面，本方法均优于基线方法，可实现机器人推动技能的快速学习。