People with Visual Impairments (PVI) typically recognize objects through haptic perception. Knowing objects and materials before touching is desired by the target users but under-explored in the field of human-centered robotics. To fill this gap, in this work, a wearable vision-based robotic system, MateRobot, is established for PVI to recognize materials and object categories beforehand. To address the computational constraints of mobile platforms, we propose a lightweight yet accurate model MateViT to perform pixel-wise semantic segmentation, simultaneously recognizing both objects and materials. Our methods achieve respective 40.2% and 51.1% of mIoU on COCOStuff-10K and DMS datasets, surpassing the previous method with +5.7% and +7.0% gains. Moreover, on the field test with participants, our wearable system reaches a score of 28 in the NASA-Task Load Index, indicating low cognitive demands and ease of use. Our MateRobot demonstrates the feasibility of recognizing material property through visual cues and offers a promising step towards improving the functionality of wearable robots for PVI. The source code has been made publicly available at https://junweizheng93.github.io/publications/MATERobot/MATERobot.html.

翻译：视障人士通常通过触觉感知来识别物体。在目标用户群体中，他们希望在触碰前就能了解物体及其材质，但这一需求在人机交互机器人领域尚待深入探索。为填补这一空白，本研究构建了一套基于视觉的可穿戴机器人系统MateRobot，帮助视障人士预判物体材质与类别。针对移动平台的计算资源限制，我们提出了一种轻量级高精度模型MateViT，实现像素级语义分割，同步识别物体与材质。该方法在COCOStuff-10K和DMS数据集上分别取得了40.2%和51.1%的平均交并比（mIoU），较现有方法分别提升+5.7%和+7.0%。此外，在参与者实地测试中，本可穿戴系统在NASA任务负荷指数中取得28分，表明其认知需求低且易于使用。MateRobot验证了通过视觉线索识别材质属性的可行性，为提升视障人士可穿戴机器人功能迈出了重要一步。源代码已开源：https://junweizheng93.github.io/publications/MATERobot/MATERobot.html。