LLM-Glasses is a wearable navigation system which assists visually impaired people by utilizing YOLO-World object detection, GPT-4o-based reasoning, and haptic feedback for real-time guidance. The device translates visual scene understanding into intuitive tactile feedback on the temples, allowing hands-free navigation. Three studies evaluate the system: recognition of 13 haptic patterns with an average recognition rate of 81.3%, VICON-based guidance with predefined paths using haptic cues, and an LLM-guided scene evaluation with decision accuracies of 91.8% without obstacles, 84.6% with static obstacles, and 81.5% with dynamic obstacles. These results show that LLM-Glasses can deliver reliable navigation support in controlled environments and motivate further work on responsiveness and deployment in more complex real-world scenarios.
翻译:LLM-Glasses 是一种可穿戴导航系统,通过结合 YOLO-World 目标检测、基于 GPT-4o 的推理以及触觉反馈技术,为视障人士提供实时引导。该设备将视觉场景理解转化为镜腿上的直观触觉反馈,实现无需手持的导航。我们通过三项研究评估了该系统:对 13 种触觉模式的识别平均识别率达到 81.3%;基于 VICON 系统、利用触觉提示沿预设路径的引导实验;以及基于 LLM 的场景评估实验,其在无障碍、静态障碍和动态障碍环境下的决策准确率分别为 91.8%、84.6% 和 81.5%。这些结果表明,LLM-Glasses 能够在受控环境中提供可靠的导航支持,并激励我们进一步研究其在更复杂真实场景中的响应能力与部署应用。