Equipping robots with the sense of touch is critical to emulating the capabilities of humans in real world manipulation tasks. Visuotactile sensors are a popular tactile sensing strategy due to data output compatible with computer vision algorithms and accurate, high resolution estimates of local object geometry. However, these sensors struggle to accommodate high deformations of the sensing surface during object interactions, hindering more informative contact with cm-scale objects frequently encountered in the real world. The soft interfaces of visuotactile sensors are often made of hyperelastic elastomers, which are difficult to simulate quickly and accurately when extremely deformed for tactile information. Additionally, many visuotactile sensors that rely on strict internal light conditions or pattern tracking will fail if the surface is highly deformed. In this work, we propose an algorithm that fuses proximity and visuotactile point clouds for contact patch segmentation that is entirely independent from membrane mechanics. This algorithm exploits the synchronous, high-res proximity and visuotactile modalities enabled by an extremely deformable, selectively transmissive soft membrane, which uses visible light for visuotactile sensing and infrared light for proximity depth. We present the hardware design, membrane fabrication, and evaluation of our contact patch algorithm in low (10%), medium (60%), and high (100%+) membrane strain states. We compare our algorithm against three baselines: proximity-only, tactile-only, and a membrane mechanics model. Our proposed algorithm outperforms all baselines with an average RMSE under 2.8mm of the contact patch geometry across all strain ranges. We demonstrate our contact patch algorithm in four applications: varied stiffness membranes, torque and shear-induced wrinkling, closed loop control for whole body manipulation, and pose estimation.

翻译：赋予机器人触觉对于在真实世界操控任务中模拟人类能力至关重要。视触觉传感器因其数据输出与计算机视觉算法兼容，并能准确高分辨率地估计局部物体几何形状，成为一种流行的触觉感知策略。然而，这类传感器在物体交互过程中难以适应传感表面的高度形变，从而阻碍了在真实世界中常见厘米级物体互动中获取更丰富的信息。视触觉传感器的软性界面通常由超弹性材料制成，当发生极端形变时，难以快速准确地模拟触觉信息。此外，许多依赖严格内部光照条件或图案追踪的视触觉传感器，在表面高度形变时将会失效。本文提出一种融合近程与视触觉点云的算法，用于接触斑块分割，该算法完全独立于薄膜力学。该算法利用由一种可极端形变、选择性透射软薄膜（该薄膜利用可见光进行视触觉感知，利用红外光进行近程深度感知）实现的同步高分辨率近程与视触觉模态。我们展示了硬件设计、薄膜制作，并在低（10%）、中（60%）、高（100%+）薄膜应变状态下评估了接触斑块算法。我们将该算法与三种基线方法对比：仅近程、仅触觉以及薄膜力学模型。所提算法在所有应变范围内均优于基线方法，接触斑块几何形状的平均均方根误差低于2.8毫米。我们还在四种应用中展示了该接触斑块算法：不同刚度薄膜、扭矩与剪切诱导褶皱、全身操控的闭环控制以及位姿估计。