Over the past few years, there has been a great deal of research on navigation tasks in indoor environments using deep reinforcement learning agents. Most of these tasks use only visual information in the form of first-person images to navigate to a single goal. More recently, tasks that simultaneously use visual and auditory information to navigate to the sound source and even navigation tasks with multiple goals instead of one have been proposed. However, there has been no proposal for a generalized navigation task combining these two types of tasks and using both visual and auditory information in a situation where multiple sound sources are goals. In this paper, we propose a new framework for this generalized task: multi-goal audio-visual navigation. We first define the task in detail, and then we investigate the difficulty of the multi-goal audio-visual navigation task relative to the current navigation tasks by conducting experiments in various situations. The research shows that multi-goal audio-visual navigation has the difficulty of the implicit need to separate the sources of sound. Next, to mitigate the difficulties in this new task, we propose a method named sound direction map (SDM), which dynamically localizes multiple sound sources in a learning-based manner while making use of past memories. Experimental results show that the use of SDM significantly improves the performance of multiple baseline methods, regardless of the number of goals.

翻译：近年来，基于深度强化学习智能体的室内环境导航任务研究取得了大量成果。此类任务大多仅利用第一人称视角图像形式的视觉信息导航至单一目标。近期，研究者提出了同时利用视觉与听觉信息导航至声源的任务，甚至提出了包含多个目标的导航任务。然而，目前尚未出现一种将上述两类任务融合、在多个声源作为目标的情境下同时利用视听信息的通用导航任务方案。本文提出了一种针对该通用任务的新框架：多目标视听导航。我们首先详细定义了该任务，随后通过多种情境下的实验，探究了多目标视听导航任务相较于现有导航任务的难度。研究表明，该任务隐含着需要分离声源的复杂性。为缓解这一新任务的难点，我们提出了一种名为声源方向地图（SDM）的方法。该方法以学习的方式动态定位多个声源，同时充分利用过往记忆。实验结果表明，无论目标数量如何，使用SDM均能显著提升多种基线方法的性能。