In contact-rich tasks, the hybrid, multi-modal nature of contact dynamics poses great challenges in model representation, planning, and control. Recent efforts have attempted to address these challenges via data-driven methods, learning dynamical models in combination with model predictive control. Those methods, while effective, rely solely on minimizing forward prediction errors to hope for better task performance with MPC controllers. This weak correlation can result in data inefficiency as well as limitations to overall performance. In response, we propose a novel strategy: using a policy gradient algorithm to find a simplified dynamics model that explicitly maximizes task performance. Specifically, we parameterize the stochastic policy as the perturbed output of the MPC controller, thus, the learned model representation can directly associate with the policy or task performance. We apply the proposed method to contact-rich tasks where a three-fingered robotic hand manipulates previously unknown objects. Our method significantly enhances task success rate by up to 15% in manipulating diverse objects compared to the existing method while sustaining data efficiency. Our method can solve some tasks with success rates of 70% or higher using under 30 minutes of data. All videos and codes are available at https://sites.google.com/view/lcs-rl.

翻译：在接触密集任务中，接触动力学的混合、多模态特性对模型表示、规划与控制构成了重大挑战。近期研究尝试通过数据驱动方法应对这些挑战，将动力学模型学习与模型预测控制相结合。这类方法虽然有效，但仅依赖最小化前向预测误差，期望借此提升MPC控制器的任务性能。这种弱相关性可能导致数据效率低下，并限制整体性能。为此，我们提出一种新策略：利用策略梯度算法寻找能显式最大化任务性能的简化动力学模型。具体而言，我们将随机策略参数化为MPC控制器输出的扰动形式，从而使学习到的模型表示能够直接与策略或任务性能关联。我们将该方法应用于接触密集任务——由三指机械手操作未知物体。与现有方法相比，我们的方法在操作多样化物体时，任务成功率最高提升15%，同时保持数据效率。使用不到30分钟的数据，我们的方法即可解决部分任务，成功率可达70%或更高。所有视频与代码详见https://sites.google.com/view/lcs-rl。