Audio question answering (AQA), acting as a widely used proxy task to explore scene understanding, has got more attention. The AQA is challenging for it requires comprehensive temporal reasoning from different scales' events of an audio scene. However, existing methods mostly extend the structures of visual question answering task to audio ones in a simple pattern but may not perform well when perceiving a fine-grained audio scene. To this end, we present a Multi-scale Window Attention Fusion Model (MWAFM) consisting of an asynchronous hybrid attention module and a multi-scale window attention module. The former is designed to aggregate unimodal and cross-modal temporal contexts, while the latter captures sound events of varying lengths and their temporal dependencies for a more comprehensive understanding. Extensive experiments are conducted to demonstrate that the proposed MWAFM can effectively explore temporal information to facilitate AQA in the fine-grained scene.Code: https://github.com/GeWu-Lab/MWAFM

翻译：音频问答（AQA）作为一种广泛用于探究场景理解的代理任务，已获得更多关注。由于AQA需要从音频场景中不同尺度事件进行全面的时间推理，因此具有挑战性。然而，现有方法大多以简单模式将视觉问答任务的结构扩展到音频领域，在感知细粒度音频场景时可能表现不佳。为此，我们提出了一种多尺度窗口注意力融合模型（MWAFM），该模型由异步混合注意力模块和多尺度窗口注意力模块组成。前者用于聚合单模态和跨模态时间上下文，后者则捕捉不同长度的声音事件及其时间依赖关系，以实现更全面的理解。大量实验表明，所提出的MWAFM能够有效探索时间信息，从而在细粒度场景中促进AQA。代码链接：https://github.com/GeWu-Lab/MWAFM