The advent of tactile sensors in robotics has sparked many ideas on how robots can leverage direct contact measurements of their environment interactions to improve manipulation tasks. An important line of research in this regard is that of grasp force control, which aims to manipulate objects safely by limiting the amount of force exerted on the object. While prior works have either hand-modeled their force controllers, employed model-based approaches, or have not shown sim-to-real transfer, we propose a model-free deep reinforcement learning approach trained in simulation and then transferred to the robot without further fine-tuning. We therefore present a simulation environment that produces realistic normal forces, which we use to train continuous force control policies. An evaluation in which we compare against a baseline and perform an ablation study shows that our approach outperforms the hand-modeled baseline and that our proposed inductive bias and domain randomization facilitate sim-to-real transfer. Code, models, and supplementary videos are available on https://sites.google.com/view/rl-force-ctrl

翻译：触觉传感器在机器人领域的应用激发了诸多关于如何利用直接接触测量环境交互以改进操作任务的研究思路。其中一项重要的研究方向是抓取力控制，旨在通过限制作用于物体上的力的大小来实现安全操作。尽管先前的研究或手动建模力控制器，或采用基于模型的方法，但均未展示从仿真到现实的迁移。本文提出一种无模型的深度强化学习方法，该方法在仿真环境中训练后可直接迁移至机器人，无需额外微调。为此，我们构建了一个能产生真实法向力的仿真环境，并用于训练连续力控制策略。通过与基线方法比较及消融实验的评估表明，我们的方法优于手动建模的基线，且所提出的归纳偏置与域随机化有助于实现从仿真到现实的迁移。代码、模型及补充视频见 https://sites.google.com/view/rl-force-ctrl