We propose a method to track the 6D pose of an object over time, while the object is under non-prehensile manipulation by a robot. At any given time during the manipulation of the object, we assume access to the robot joint controls and an image from a camera. We use the robot joint controls to perform a physics-based prediction of how the object might be moving. We then combine this prediction with the observation coming from the camera, to estimate the object pose as accurately as possible. We use a particle filtering approach to combine the control information with the visual information. We compare the proposed method with two baselines: (i) using only an image-based pose estimation system at each time-step, and (ii) a particle filter which does not perform the computationally expensive physics predictions, but assumes the object moves with constant velocity. Our results show that making physics-based predictions is worth the computational cost, resulting in more accurate tracking, and estimating object pose even when the object is not clearly visible to the camera.

翻译：本文提出一种在机器人对物体进行非抓取操作时，实时跟踪物体6维姿态的方法。在物体操作过程中的任意时刻，我们假设可以获取机器人关节控制指令与摄像头图像。首先利用机器人关节控制指令进行基于物理的物体运动状态预测，随后将预测结果与摄像头观测信息相融合，以尽可能精确地估计物体姿态。我们采用粒子滤波方法融合控制信息与视觉信息。将所提方法与两种基线方法进行对比：(i) 仅在各时刻使用基于图像的姿态估计系统，以及(ii) 采用不进行高计算量物理预测、假设物体匀速运动的粒子滤波方法。实验结果表明，基于物理的预测方法尽管增加了计算成本，但能实现更精确的跟踪效果，即使在物体未被摄像头清晰捕捉的情况下仍能有效估计其姿态。