Object pushing presents a key non-prehensile manipulation problem that is illustrative of more complex robotic manipulation tasks. While deep reinforcement learning (RL) methods have demonstrated impressive learning capabilities using visual input, a lack of tactile sensing limits their capability for fine and reliable control during manipulation. Here we propose a deep RL approach to object pushing using tactile sensing without visual input, namely tactile pushing. We present a goal-conditioned formulation that allows both model-free and model-based RL to obtain accurate policies for pushing an object to a goal. To achieve real-world performance, we adopt a sim-to-real approach. Our results demonstrate that it is possible to train on a single object and a limited sample of goals to produce precise and reliable policies that can generalize to a variety of unseen objects and pushing scenarios without domain randomization. We experiment with the trained agents in harsh pushing conditions, and show that with significantly more training samples, a model-free policy can outperform a model-based planner, generating shorter and more reliable pushing trajectories despite large disturbances. The simplicity of our training environment and effective real-world performance highlights the value of rich tactile information for fine manipulation. Code and videos are available at https://sites.google.com/view/tactile-rl-pushing/.

翻译：物体推动是一个关键的非抓取操作问题，它代表了更复杂的机器人操作任务。虽然深度强化学习方法已在使用视觉输入时展现出令人印象深刻的学习能力，但缺乏触觉感知限制了它们在操作过程中实现精细且可靠控制的能力。本文提出了一种仅使用触觉感知（即触觉推动）进行物体推动的深度强化学习方法。我们提出了一种以目标为条件的公式化方法，使得无模型和基于模型的强化学习都能获得将物体推向目标的精确策略。为获得真实世界性能，我们采用了模拟到现实的方法。结果表明，在单个物体和有限样本目标上进行训练，可以产生精确可靠的策略，这些策略能够在无需域随机化的情况下泛化到各种未见过的物体和推动场景。我们在苛刻的推动条件下对训练好的智能体进行实验，发现通过显著增加训练样本，无模型策略可以超越基于模型的规划器，即使在大扰动下也能生成更短且更可靠的推动轨迹。我们训练环境的简洁性和有效的真实世界性能凸显了丰富触觉信息对精细操作的价值。代码和视频可在 https://sites.google.com/view/tactile-rl-pushing/ 获取。