Combining 3D vision with tactile sensing could unlock a greater level of dexterity for robots and improve several manipulation tasks. However, obtaining a close-up 3D view of the location where manipulation contacts occur can be challenging, particularly in confined spaces, cluttered environments, or without installing more sensors on the end effector. In this context, this paper presents StereoTac, a novel vision-based sensor that combines tactile sensing with 3D vision. The proposed sensor relies on stereoscopic vision to capture a 3D representation of the environment before contact and uses photometric stereo to reconstruct the tactile imprint generated by an object during contact. To this end, two cameras were integrated in a single sensor, whose interface is made of a transparent elastomer coated with a thin layer of paint with a level of transparency that can be adjusted by varying the sensor's internal lighting conditions. We describe the sensor's fabrication and evaluate its performance for both tactile perception and 3D vision. Our results show that the proposed sensor can reconstruct a 3D view of a scene just before grasping and perceive the tactile imprint after grasping, allowing for monitoring of the contact during manipulation.

翻译：将三维视觉与触觉感知相结合，有望提升机器人的灵巧操作水平，并改善多项操控任务。然而，在机械臂接触发生的位置获取近距离三维视图颇具挑战性，尤其是在狭小空间、杂乱环境中，或无法在末端执行器上安装更多传感器的情况下。为此，本文提出StereoTac——一种基于视觉的新型传感器，它集成了触觉感知与三维视觉功能。该传感器利用立体视觉在接触前捕获环境的三维表征，并通过光度立体视觉重建物体接触时产生的触觉印记。为实现这一目标，我们在单个传感器中集成两个摄像头，其界面由透明弹性体制成，表面涂覆一层可通过调节传感器内部光照条件改变透明度的薄漆。本文介绍了传感器的制作工艺，并评估了其在触觉感知与三维视觉方面的性能。实验结果表明，该传感器能在抓取前重建场景的三维视图，并在抓取后感知触觉印记，从而实现对操作过程中接触状态的实时监测。