In contact-rich tasks, the hybrid, multi-modal nature of contact dynamics poses great challenges in model representation, planning, and control. Recent efforts have attempted to address these challenges via data-driven methods, learning dynamical models in combination with model predictive control. Those methods, while effective, rely solely on minimizing forward prediction errors to hope for better task performance with MPC controllers. This weak correlation can result in data inefficiency as well as limitations to overall performance. In response, we propose a novel strategy: using a policy gradient algorithm to find a simplified dynamics model that explicitly maximizes task performance. Specifically, we parameterize the stochastic policy as the perturbed output of the MPC controller, thus, the learned model representation can directly associate with the policy or task performance. We apply the proposed method to contact-rich tasks where a three-fingered robotic hand manipulates previously unknown objects. Our method significantly enhances task success rate by up to 15% in manipulating diverse objects compared to the existing method while sustaining data efficiency. Our method can solve some tasks with success rates of 70% or higher using under 30 minutes of data. All videos and codes are available at https://sites.google.com/view/lcs-rl.

翻译：在接触密集型任务中，接触动力学的混合、多模态特性给模型表示、规划和控制带来了巨大挑战。近期研究尝试通过数据驱动方法应对这些挑战，结合模型预测控制学习动力学模型。这些方法虽然有效，但仅依赖最小化前向预测误差来期望MPC控制器获得更好的任务性能。这种弱相关性可能导致数据效率低下以及整体性能受限。为此，我们提出一种新策略：利用策略梯度算法寻找能显式最大化任务性能的简化动力学模型。具体而言，我们将随机策略参数化为MPC控制器输出的扰动形式，从而使学习到的模型表示直接与策略或任务性能相关联。我们将该方法应用于三指机械手操控未知物体的接触密集型任务。与现有方法相比，本方法在操控不同物体时任务成功率提升高达15%，同时保持数据效率。利用少于30分钟的数据，本方法即可在部分任务中达到70%以上的成功率。所有视频和代码均可在 https://sites.google.com/view/lcs-rl 获取。