From refrigerators to kitchen drawers, humans interact with articulated objects effortlessly every day while completing household chores. For automating these tasks, service robots must be capable of manipulating arbitrary articulated objects. Recent deep learning methods have been shown to predict valuable priors on the affordance of articulated objects from vision. In contrast, many other works estimate object articulations by observing the articulation motion, but this requires the robot to already be capable of manipulating the object. In this article, we propose a novel approach combining these methods by using a factor graph for online estimation of articulation which fuses learned visual priors and proprioceptive sensing during interaction into an analytical model of articulation based on Screw Theory. With our method, a robotic system makes an initial prediction of articulation from vision before touching the object, and then quickly updates the estimate from kinematic and force sensing during manipulation. We evaluate our method extensively in both simulations and real-world robotic manipulation experiments. We demonstrate several closed-loop estimation and manipulation experiments in which the robot was capable of opening previously unseen drawers. In real hardware experiments, the robot achieved a 75% success rate for autonomous opening of unknown articulated objects.

翻译：从冰箱到厨房抽屉，人类在日常家务中能够毫不费力地与铰接物体进行交互。为实现此类任务的自动化，服务机器人必须具备操作任意铰接物体的能力。近期深度学习方法已证明能够通过视觉预测铰接物体功能可供性的有效先验。相比之下，许多其他工作通过观察铰接运动来估计物体铰接状态，但这要求机器人已具备操作该物体的能力。本文提出一种结合这些方法的新途径：利用因子图进行铰接状态的在线估计，该方法将学习到的视觉先验与交互过程中的本体感知信息融合到基于螺旋理论的铰接解析模型中。通过我们的方法，机器人系统可在接触物体前通过视觉对铰接状态进行初始预测，随后在操作过程中根据运动学与力传感数据快速更新估计值。我们在仿真和真实机器人操作实验中对该方法进行了全面评估。通过多个闭环估计与操作实验，我们证明机器人能够开启先前未见过的抽屉。在真实硬件实验中，机器人对未知铰接物体的自主开启成功率达到了75%。