Audio encoders are critical to modern audio applications as large language models (LLMs) increasingly rely on a single encoder for diverse inputs. While self-supervised learning (SSL) has yielded strong domain-specific encoders like speech or music experts, multi-domain approaches like USAD and SPEAR remain limited in coverage and evaluation. Recent studies also suggest supervised encoders align better with audio LLMs. We present USAD 2.0, a universal encoder integrating knowledge from both SSL and supervised foundation models. USAD 2.0 introduces domain-aware distillation to address teacher mismatch, extends coverage to the music domain, and adds second-stage supervised distillation for downstream use. We further scale the model to one billion parameters via depth scaling. Experiments show USAD 2.0 achieves strong or state-of-the-art performance across probing and LLM-based evaluations.

翻译：音频编码器对于现代音频应用至关重要，因为大型语言模型（LLM）越来越依赖单一编码器处理多样化输入。虽然自监督学习（SSL）已催生出如语音或音乐专家等强大的领域专用编码器，但诸如USAD和SPEAR等多领域方法在覆盖范围和评估方面仍存在局限。近期研究还表明，监督编码器与音频LLM的契合度更高。我们提出USAD 2.0，一种融合了SSL和监督基础模型知识的通用编码器。USAD 2.0引入了领域感知蒸馏以解决教师模型不匹配问题，将覆盖范围扩展至音乐领域，并加入面向下游任务的第二阶段监督蒸馏。我们进一步通过深度缩放将模型参数扩展至十亿规模。实验表明，USAD 2.0在基于探测和LLM的评估中均达到了卓越或最优性能。