Persons with visual impairments (PwVI) have difficulties understanding and navigating spaces around them. Current wayfinding technologies either focus solely on navigation or provide limited communication about the environment. Motivated by recent advances in visual-language grounding and semantic navigation, we propose DRAGON, a guiding robot powered by a dialogue system and the ability to associate the environment with natural language. By understanding the commands from the user, DRAGON is able to guide the user to the desired landmarks on the map, describe the environment, and answer questions from visual observations. Through effective utilization of dialogue, the robot can ground the user's free-form descriptions to landmarks in the environment, and give the user semantic information through spoken language. We conduct a user study with blindfolded participants in an everyday indoor environment. Our results demonstrate that DRAGON is able to communicate with the user smoothly, provide a good guiding experience, and connect users with their surrounding environment in an intuitive manner. Videos and code are available at https://sites.google.com/view/dragon-wayfinding/home.

翻译：视力障碍者（PwVI）在理解和导航周围空间方面存在困难。当前寻路技术要么仅聚焦于导航，要么提供有限的环境信息沟通。受视觉语言基础与语义导航最新进展的启发，我们提出DRAGON——一款由对话系统驱动、具备将环境与自然语言关联能力的引导机器人。通过理解用户指令，DRAGON能够引导用户抵达地图上的期望地标、描述环境，并基于视觉观察回答用户问题。通过有效利用对话，机器人可将用户的自由形式描述与环境中的地标进行语义关联，并通过语音向用户传递语义信息。我们在日常室内环境中开展了蒙眼参与者用户研究。结果表明，DRAGON能够与用户流畅沟通，提供良好的引导体验，并以直观方式连接用户与周围环境。视频与代码详见https://sites.google.com/view/dragon-wayfinding/home。