Persons with visual impairments (PwVI) have difficulties understanding and navigating spaces around them. Current wayfinding technologies either focus solely on navigation or provide limited communication about the environment. Motivated by recent advances in visual-language grounding and semantic navigation, we propose DRAGON, a guiding robot powered by a dialogue system and the ability to associate the environment with natural language. By understanding the commands from the user, DRAGON is able to guide the user to the desired landmarks on the map, describe the environment, and answer questions from visual observations. Through effective utilization of dialogue, the robot can ground the user's free-form descriptions to landmarks in the environment, and give the user semantic information through spoken language. We conduct a user study with blindfolded participants in an everyday indoor environment. Our results demonstrate that DRAGON is able to communicate with the user smoothly, provide a good guiding experience, and connect users with their surrounding environment in an intuitive manner. Videos and code are available at https://sites.google.com/view/dragon-wayfinding/home.

翻译：视障人士在理解和导航周围空间时面临困难。当前的导航技术要么仅专注于路径指引，要么对环境提供有限的沟通。受视觉语言接地与语义导航领域最新进展的启发，我们提出DRAGON——一款由对话系统驱动、具备将环境与自然语言关联能力的引导机器人。通过理解用户指令，DRAGON能够引导用户抵达地图上的目标地标、描述环境信息，并基于视觉观察回答用户问题。通过有效利用对话交互，机器人可将用户的自由形式描述映射至环境中的地标，并通过语音向用户传递语义信息。我们在日常室内环境中对蒙眼参与者开展了用户研究。结果表明，DRAGON能够与用户流畅沟通，提供良好的引导体验，并以直观方式帮助用户与周围环境建立连接。视频与代码请参见https://sites.google.com/view/dragon-wayfinding/home。