Large audio language models (LALMs) extend large language models with an audio encoder and large-scale audio data. However, the scarcity of high-quality annotated audio data remains a fundamental bottleneck for scaling. Through probing signal detectability analysis, we identify fine-grained spectrotemporal perceptual weaknesses in a foundation LALM. To address these challenges, we propose Spectrotemporal Counting (SpectCount), a data-efficient fine-tuning approach based on fully synthetic audio signals generated on-the-fly, without relying on real-world audio, annotations, or pretrained generative models. SpectCount not only resolves the observed weaknesses but also improves performance on diverse auditory benchmarks spanning sound, music, and speech, unseen during fine-tuning. These results suggest that weakness-targeted synthetic signals provide a data-efficient path toward enhanced auditory understanding capabilities in LALMs.

翻译：摘要：大型音频语言模型通过引入音频编码器和大规模音频数据扩展了大型语言模型的能力。然而，高质量标注音频数据的稀缺性仍是制约其规模化的根本瓶颈。通过探针信号可检测性分析，我们识别出基础大型音频语言模型在精细频谱时间感知方面的缺陷。为解决这些问题，我们提出频谱时间计数方法——一种基于全合成信号的数据高效微调方案，该方法可实时生成信号，无需依赖真实音频、人工标注或预训练生成模型。SpectCount不仅能弥补已发现的感知缺陷，还能提升模型在微调中未见过的声音、音乐和语音等多类听觉基准任务上的表现。这些结果表明，针对弱点的合成信号为增强大型音频语言模型的听觉理解能力提供了一条数据高效的路径。