The advancements in vision-based tactile sensors have boosted the aptitude of robots to perform contact-rich manipulation, particularly when precise positioning and contact state of the manipulated objects are crucial for successful execution. In this work, we present 9DTact, a straightforward yet versatile tactile sensor that offers 3D shape reconstruction and 6D force estimation capabilities. Conceptually, 9DTact is designed to be highly compact, robust, and adaptable to various robotic platforms. Moreover, it is low-cost and DIY-friendly, requiring minimal assembly skills. Functionally, 9DTact builds upon the optical principles of DTact and is optimized to achieve 3D shape reconstruction with enhanced accuracy and efficiency. Remarkably, we leverage the optical and deformable properties of the translucent gel so that 9DTact can perform 6D force estimation without the participation of auxiliary markers or patterns on the gel surface. More specifically, we collect a dataset consisting of approximately 100,000 image-force pairs from 175 complex objects and train a neural network to regress the 6D force, which can generalize to unseen objects. To promote the development and applications of vision-based tactile sensors, we open-source both the hardware and software of 9DTact as well as present a 1-hour video tutorial.

翻译：视觉触觉传感器的进步提升了机器人执行接触密集操作的能力，尤其是在操作对象的精确定位和接触状态对成功执行至关重要的情况下。本文提出9DTact——一种简单而多功能的触觉传感器，具备三维形状重建和六维力估计能力。在概念上，9DTact设计紧凑、鲁棒性强，且易于适配各类机器人平台。此外，其成本低廉且适合自主搭建，仅需基础的组装技能。功能上，9DTact基于DTact的光学原理构建，通过优化实现更高精度和效率的三维形状重建。值得注意的是，我们利用半透明凝胶的光学与形变特性，使9DTact能在无需凝胶表面辅助标记或图案的情况下实现六维力估计。具体而言，我们采集了包含175个复杂物体约10万组图像-力配对的数据集，并训练神经网络回归六维力，该网络可泛化至未知物体。为促进视觉触觉传感器的发展与应用，我们开源了9DTact的硬件与软件设计，并提供了1小时视频教程。