When manipulating an object to accomplish complex tasks, humans rely on both vision and touch to keep track of the object's 6D pose. However, most existing object pose tracking systems in robotics rely exclusively on visual signals, which hinder a robot's ability to manipulate objects effectively. To address this limitation, we introduce TEG-Track, a tactile-enhanced 6D pose tracking system that can track previously unseen objects held in hand. From consecutive tactile signals, TEG-Track optimizes object velocities from marker flows when slippage does not occur, or regresses velocities using a slippage estimation network when slippage is detected. The estimated object velocities are integrated into a geometric-kinematic optimization scheme to enhance existing visual pose trackers. To evaluate our method and to facilitate future research, we construct a real-world dataset for visual-tactile in-hand object pose tracking. Experimental results demonstrate that TEG-Track consistently enhances state-of-the-art generalizable 6D pose trackers in synthetic and real-world scenarios. Our code and dataset are available at https://github.com/leolyliu/TEG-Track.

翻译：在操作物体完成复杂任务时，人类依赖视觉与触觉共同跟踪物体的6D位姿。然而，现有机器人物体位姿跟踪系统大多仅依赖视觉信号，这限制了机器人有效操作物体的能力。为解决这一局限，我们提出TEG-Track——一种触觉增强的6D位姿跟踪系统，可跟踪手中先前未见过的物体。通过连续触觉信号，TEG-Track在未发生滑动时通过标记流优化物体速度，或在检测到滑动时通过滑动估计网络回归速度。估计的物体速度被集成到几何-运动学优化方案中，以增强现有视觉位姿跟踪器。为评估方法并促进未来研究，我们构建了用于视觉-触觉手持物体位姿跟踪的真实世界数据集。实验结果表明，TEG-Track在合成与真实场景中一致性地提升了最先进的可泛化6D位姿跟踪器的性能。我们的代码与数据集已开源至https://github.com/leolyliu/TEG-Track。