Contact often occurs without macroscopic surface deformation, such as during interaction with liquids, semi-liquids, or ultra-soft materials. However, most existing tactile sensors rely on deformation to infer contact, making such light-contact interactions difficult to perceive robustly. To address this, we present LightTact, a visual-tactile fingertip sensor that makes contact directly visible via a deformation-independent principle. LightTact features an ambient-blocking optical configuration that suppresses both external light and internal illumination at non-contact regions, while transmitting only the scattered light generated at true contacts. As a result, LightTact produces high-contrast raw images in which non-contact pixels remain near-black (mean gray value < 3) and contact pixels preserve the natural appearance of the contacting surface. Built on this, LightTact achieves accurate pixel-level contact segmentation that is robust to material properties, contact force, surface appearance, and environmental lighting. We further demonstrate that LightTact unlocks new robotic manipulation behaviors that require detection of extremely light contact, including water spreading, facial-cream dipping, and soft thin-film interaction. In addition, we show that LightTact's spatially aligned visual-tactile images can be directly interpreted by vision-language models.
翻译:接触往往发生在没有宏观表面形变的情况下,例如与液体、半液体或超软材料相互作用时。然而,现有的大多数触觉传感器依赖形变来推断接触,这使得此类轻接触交互难以被鲁棒地感知。为解决这一问题,我们提出了LightTact,一种基于形变无关原理、通过直接可视化实现接触感知的视觉-触觉指尖传感器。LightTact采用了一种环境光阻隔光学结构,该结构能同时抑制外部光线和非接触区域的内部照明,仅透射真实接触点产生的散射光。因此,LightTact可生成高对比度的原始图像,其中非接触像素保持近黑色(平均灰度值<3),而接触像素则保留接触表面的自然外观。基于此,LightTact实现了精确的像素级接触分割,其对材料特性、接触力、表面外观和环境光照具有鲁棒性。我们进一步证明,LightTact能够实现需要检测极轻微接触的新型机器人操作行为,包括水扩散、面霜蘸取和软薄膜交互。此外,我们还展示了LightTact的空间对齐视觉-触觉图像可直接由视觉-语言模型进行解析。