The objective of the sound source localization task is to enable machines to detect the location of sound-making objects within a visual scene. While the audio modality provides spatial cues to locate the sound source, existing approaches only use audio as an auxiliary role to compare spatial regions of the visual modality. Humans, on the other hand, utilize both audio and visual modalities as spatial cues to locate sound sources. In this paper, we propose an audio-visual spatial integration network that integrates spatial cues from both modalities to mimic human behavior when detecting sound-making objects. Additionally, we introduce a recursive attention network to mimic human behavior of iterative focusing on objects, resulting in more accurate attention regions. To effectively encode spatial information from both modalities, we propose audio-visual pair matching loss and spatial region alignment loss. By utilizing the spatial cues of audio-visual modalities and recursively focusing objects, our method can perform more robust sound source localization. Comprehensive experimental results on the Flickr SoundNet and VGG-Sound Source datasets demonstrate the superiority of our proposed method over existing approaches. Our code is available at: https://github.com/VisualAIKHU/SIRA-SSL

翻译：声源定位任务的目标是使机器能够检测视觉场景中发声物体的位置。虽然音频模态为定位声源提供了空间线索，但现有方法仅将音频作为辅助角色来比较视觉模态的空间区域。相比之下，人类同时利用音频和视觉模态作为空间线索来定位声源。本文提出一种视听空间整合网络，通过融合两种模态的空间线索来模拟人类检测发声物体时的行为。此外，我们引入递归注意力网络模拟人类对物体进行迭代聚焦的行为，从而获得更精确的注意力区域。为有效编码两种模态的空间信息，我们提出视听对匹配损失和空间区域对齐损失。通过利用视听模态的空间线索并递归聚焦物体，本方法能够实现更稳健的声源定位。在Flickr SoundNet和VGG-Sound Source数据集上的全面实验结果表明，本方法优于现有方法。我们的代码开源在：https://github.com/VisualAIKHU/SIRA-SSL