Sound events are entities with semantic identities, locations, and trajectories, but current audio-language models usually reason about clips as global event content. Conversely, sound event localization models track source directions over time but offer limited semantic coverage for language reasoning. To address this gap, we introduce ST-AudioQA, a spatio-temporal audio QA dataset and benchmark built from first-order ambisonic (FOA) renderings of static and moving sound sources. Each scene provides source identity, activity, direction, distance, and motion metadata, enabling dense trajectory supervision and questions about what is sounding, where it is, how it moves, and how sources relate. We further propose ST-Audio Encoder, a time-resolved FOA audio encoder that learns event semantics together with source trajectories, and ST-AudioLM, which connects the audio tokens from the encoder to an LLM for spatio-temporal audio QA. Experiments show that this representation improves the semantic-localization tradeoff and yields stronger reasoning performance than static spatial and localization-oriented baselines.

翻译：声音事件是具有语义身份、位置和轨迹的实体，但当前的音频－语言模型通常将音频片段推理为全局事件内容。相反，声音事件定位模型能随时间追踪声源方向，但对语言推理的语义覆盖有限。为解决这一差距，我们提出ST-AudioQA，一个基于一阶环绕声（FOA）渲染的静态和移动声源构建的时空音频问答数据集与基准。每个场景提供声源身份、活动、方向、距离和运动元数据，从而实现密集的轨迹监督以及关于什么在发声、它在哪里、如何运动以及声源间关系的问题。我们进一步提出ST-Audio Encoder，一种时间解析的FOA音频编码器，能够同时学习事件语义和声源轨迹，以及ST-AudioLM，它将编码器输出的音频令牌连接到大型语言模型（LLM）以进行时空音频问答。实验表明，这种表示改善了语义－定位权衡，并相比静态空间和面向定位的基线方法表现出更强的推理性能。