Sound source localization is a typical and challenging task that predicts the location of sound sources in a video. Previous single-source methods mainly used the audio-visual association as clues to localize sounding objects in each image. Due to the mixed property of multiple sound sources in the original space, there exist rare multi-source approaches to localizing multiple sources simultaneously, except for one recent work using a contrastive random walk in the graph with images and separated sound as nodes. Despite their promising performance, they can only handle a fixed number of sources, and they cannot learn compact class-aware representations for individual sources. To alleviate this shortcoming, in this paper, we propose a novel audio-visual grouping network, namely AVGN, that can directly learn category-wise semantic features for each source from the input audio mixture and image to localize multiple sources simultaneously. Specifically, our AVGN leverages learnable audio-visual class tokens to aggregate class-aware source features. Then, the aggregated semantic features for each source can be used as guidance to localize the corresponding visual regions. Compared to existing multi-source methods, our new framework can localize a flexible number of sources and disentangle category-aware audio-visual representations for individual sound sources. We conduct extensive experiments on MUSIC, VGGSound-Instruments, and VGG-Sound Sources benchmarks. The results demonstrate that the proposed AVGN can achieve state-of-the-art sounding object localization performance on both single-source and multi-source scenarios. Code is available at \url{https://github.com/stoneMo/AVGN}.

翻译：声音源定位是一项典型且具有挑战性的任务，旨在预测视频中声源的位置。以往的单源方法主要利用音频-视觉关联作为线索，在每张图像中定位发声物体。由于原始空间中多个声源的混合特性，同时定位多个声源的多源方法较为罕见，除了最近一项利用对比随机游走方法、以图像和分离后的声音为节点构建图的工作。尽管这些方法表现出色，但它们只能处理固定数量的声源，且无法为单个声源学习紧凑的类别感知表示。为缓解这一不足，本文提出了一种新颖的音频-视觉分组网络（AVGN），该网络能直接从输入音频混合和图像中学习每个声源的类别级语义特征，从而同时定位多个声源。具体而言，我们的AVGN利用可学习的音频-视觉类别标记来聚合类别感知的声源特征。随后，每个声源的聚合语义特征可作为指导来定位对应的视觉区域。与现有的多源方法相比，我们的新框架能够定位灵活数量的声源，并解耦出每个声源的类别感知音频-视觉表示。我们在MUSIC、VGGSound-Instruments和VGG-Sound Sources基准上进行了广泛实验。结果表明，所提出的AVGN在单源和多源场景下均能实现最先进的发声物体定位性能。代码见\url{https://github.com/stoneMo/AVGN}。