People with Visual Impairments (PVI) typically recognize objects through haptic perception. Knowing objects and materials before touching is desired by the target users but under-explored in the field of human-centered robotics. To fill this gap, in this work, a wearable vision-based robotic system, MateRobot, is established for PVI to recognize materials and object categories beforehand. To address the computational constraints of mobile platforms, we propose a lightweight yet accurate model MateViT to perform pixel-wise semantic segmentation, simultaneously recognizing both objects and materials. Our methods achieve respective 40.2% and 51.1% of mIoU on COCOStuff-10K and DMS datasets, surpassing the previous method with +5.7% and +7.0% gains. Moreover, on the field test with participants, our wearable system reaches a score of 28 in the NASA-Task Load Index, indicating low cognitive demands and ease of use. Our MateRobot demonstrates the feasibility of recognizing material property through visual cues and offers a promising step towards improving the functionality of wearable robots for PVI. The source code has been made publicly available at https://junweizheng93.github.io/publications/MATERobot/MATERobot.html.

翻译：视觉障碍人群（PVI）通常通过触觉感知识别物体。目标用户期望在接触前获知物体及其材料属性，但这一方向在人机交互机器人领域尚未得到充分探索。为填补这一空白，本文建立了一套基于视觉的可穿戴机器人系统MateRobot，使PVI能够预先识别材料与物体类别。针对移动平台的计算资源限制，我们提出了轻量级且高精度的模型MateViT，实现像素级语义分割，同步识别物体与材料。该方法在COCOStuff-10K和DMS数据集上分别达到40.2%和51.1%的mIoU，相较此前方法提升+5.7%和+7.0%。此外，在参与者实地测试中，我们的可穿戴系统在NASA任务负荷指数上获得28分，表明其认知需求低且易于使用。MateRobot验证了通过视觉线索识别材料属性的可行性，为提升面向PVI的可穿戴机器人功能迈出了重要一步。源代码已公开于https://junweizheng93.github.io/publications/MATERobot/MATERobot.html。