Audio is a fundamental modality for analyzing speech, music, and environmental sounds. Although pretrained audio models have significantly advanced audio understanding, they remain fragile in real-world settings where data distributions shift over time. In this work, we present the first systematic benchmark for audio continual learning (CL) with pretrained models (PTMs), together with a comprehensive analysis of its unique challenges. Unlike in vision, where parameter-efficient fine-tuning (PEFT) has proven effective for CL, directly transferring such strategies to audio leads to poor performance. This stems from a fundamental property of audio backbones: they focus on low-level spectral details rather than structured semantics, causing severe upstream-downstream misalignment. Through extensive empirical study, we identify analytic classifiers with first-session adaptation (FSA) as a promising direction, but also reveal two major limitations: representation saturation in coarse-grained scenarios and representation drift in fine-grained scenarios. To address these challenges, we propose PACE, a novel method that enhances FSA via a regularized analytic classifier and enables multi-session adaptation through adaptive subspace-orthogonal PEFT for improved semantic alignment. In addition, we introduce spectrogram-based boundary-aware perturbations to mitigate representation overlap and improve stability. Experiments on six diverse audio CL benchmarks demonstrate that PACE substantially outperforms state-of-the-art baselines, marking an important step toward robust and scalable audio continual learning with PTMs.

翻译：音频是分析语音、音乐和环境声的基础模态。尽管预训练音频模型显著推进了音频理解，但在数据分布随时间变化的现实场景中，这些模型依然脆弱。本研究首次提出了基于预训练模型的音频持续学习系统化基准，并对其独特挑战进行了全面分析。与视觉领域中参数高效微调已被证明对持续学习有效不同，直接将此类策略迁移至音频领域会导致性能低下。这源于音频主干网络的一个基本特性：它们关注低层级频谱细节而非结构化语义，导致严重的上游-下游错位。通过大量实证研究，我们确定采用首会话适应的解析分类器是可行方向，但也揭示了两大局限性：粗粒度场景中的表征饱和与细粒度场景中的表征漂移。为应对这些挑战，我们提出PACE——一种通过正则化解析分类器增强首会话适应，并利用自适应子空间正交参数高效微调实现多会话适应以改进语义对齐的新方法。此外，我们引入基于频谱图的边界感知扰动来缓解表征重叠并提升稳定性。在六个多样化音频持续学习基准上的实验表明，PACE显著优于现有先进基线方法，标志着基于预训练模型的鲁棒可扩展音频持续学习迈出了重要一步。