Solving tasks such as speaker recognition, music classification, or semantic audio event tagging with deep learning models typically requires computationally demanding networks. General-purpose audio embeddings (GPAEs) are dense representations of audio signals that allow lightweight, shallow classifiers to tackle various audio tasks. The idea is that a single complex feature extractor would extract dense GPAEs, while shallow MLPs can produce task-specific predictions. If the extracted dense representations are general enough to allow the simple downstream classifiers to generalize to a variety of tasks in the audio domain, a single costly forward pass suffices to solve multiple tasks in parallel. In this work, we try to reduce the cost of GPAE extractors to make them suitable for resource-constrained devices. We use efficient MobileNets trained on AudioSet using Knowledge Distillation from a Transformer ensemble as efficient GPAE extractors. We explore how to obtain high-quality GPAEs from the model, study how model complexity relates to the quality of extracted GPAEs, and conclude that low-complexity models can generate competitive GPAEs, paving the way for analyzing audio streams on edge devices w.r.t. multiple audio classification and recognition tasks.

翻译：解决说话人识别、音乐分类或语义音频事件标记等任务通常需要计算密集型深度神经网络。通用音频嵌入（GPAE）是音频信号的稠密表示形式，允许使用轻量级浅层分类器处理多种音频任务。其核心理念是：单个复杂特征提取器提取稠密GPAE，而浅层多层感知机（MLP）可生成特定任务的预测结果。若提取的稠密表示具有足够通用性，使简单的下游分类器能够泛化至音频领域的各类任务，则仅需一次高成本前向传播即可并行解决多个任务。本研究致力于降低GPAE提取器的成本，使其适用于资源受限设备。我们采用基于Transformer集成通过知识蒸馏在AudioSet上训练的高效MobileNets作为GPAE提取器。探讨如何从模型中获取高质量GPAE，研究模型复杂度与提取GPAE质量之间的关系，并得出结论：低复杂度模型可生成具有竞争力的GPAE，为边缘设备上针对多类音频分类与识别任务的音频流分析奠定基础。