Persons with visual impairments (PwVI) have difficulties understanding and navigating spaces around them. Current wayfinding technologies either focus solely on navigation or provide limited communication about the environment. Motivated by recent advances in visual-language grounding and semantic navigation, we propose DRAGON, a guiding robot powered by a dialogue system and the ability to associate the environment with natural language. By understanding the commands from the user, DRAGON is able to guide the user to the desired landmarks on the map, describe the environment, and answer questions from visual observations. Through effective utilization of dialogue, the robot can ground the user's free-form descriptions to landmarks in the environment, and give the user semantic information through spoken language. We conduct a user study with blindfolded participants in an everyday indoor environment. Our results demonstrate that DRAGON is able to communicate with the user smoothly, provide a good guiding experience, and connect users with their surrounding environment in an intuitive manner.

翻译：视觉障碍者（PwVI）在理解和导航周围空间时面临困难。现有路径引导技术要么仅专注于导航功能，要么对环境信息的传递十分有限。受视觉语言接地与语义导航领域最新进展的启发，我们提出DRAGON——一种由对话系统驱动、具备将环境与自然语言关联能力的引导机器人。通过理解用户指令，DRAGON能引导用户抵达地图上的目标地标、描述环境特征，并通过视觉观察回答用户提问。借助对话机制的有效运用，机器人可将用户的自由形式描述与环境中特定地标进行语义匹配，并通过语音向用户传递语义信息。我们在日常室内环境中组织了蒙眼参与者用户研究，结果表明DRAGON能与用户进行流畅交流，提供良好的引导体验，并以直观方式建立用户与周围环境的连接。