Tactile sensing is crucial for robots aiming to achieve human-level dexterity. Among tactile-dependent skills, tactile-based object tracking serves as the cornerstone for many tasks, including manipulation, in-hand manipulation, and 3D reconstruction. In this work, we introduce NormalFlow, a fast, robust, and real-time tactile-based 6DoF tracking algorithm. Leveraging the precise surface normal estimation of vision-based tactile sensors, NormalFlow determines object movements by minimizing discrepancies between the tactile-derived surface normals. Our results show that NormalFlow consistently outperforms competitive baselines and can track low-texture objects like table surfaces. For long-horizon tracking, we demonstrate when rolling the sensor around a bead for 360 degrees, NormalFlow maintains a rotational tracking error of 2.5 degrees. Additionally, we present state-of-the-art tactile-based 3D reconstruction results, showcasing the high accuracy of NormalFlow. We believe NormalFlow unlocks new possibilities for high-precision perception and manipulation tasks that involve interacting with objects using hands. The video demo, code, and dataset are available on our website: https://joehjhuang.github.io/normalflow.

翻译：触觉感知对于旨在实现人类水平灵巧操作的机器人至关重要。在依赖触觉的技能中，基于触觉的物体跟踪是许多任务（包括操作、手内操作和三维重建）的基石。在本工作中，我们提出了NormalFlow，一种快速、鲁棒且实时的基于触觉的六自由度跟踪算法。NormalFlow利用视觉触觉传感器精确的表面法向估计，通过最小化触觉导出的表面法向之间的差异来确定物体运动。我们的结果表明，NormalFlow始终优于竞争基线，并且能够跟踪低纹理物体（如桌面）。对于长时程跟踪，我们展示了当传感器围绕一个珠子滚动360度时，NormalFlow能保持2.5度的旋转跟踪误差。此外，我们展示了最先进的基于触觉的三维重建结果，体现了NormalFlow的高精度。我们相信，NormalFlow为涉及使用手与物体交互的高精度感知和操作任务开启了新的可能性。视频演示、代码和数据集可在我们的网站上获取：https://joehjhuang.github.io/normalflow。