We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data. Specifically, we designed CALM to retrieve the probability distribution of text-audio clips over classes using a set of audio-label pairs and combined it with CLAP's zero-shot classification results. Furthermore, we designed a training-free version of the Treff adapter by using CALM as a cosine similarity measure. Experiments showed that the proposed Treff adapter is comparable and even better than fully-supervised methods and adaptation methods in low-shot and data-abundant scenarios. While the Treff adapter shows that combining large-scale pretraining and rapid learning of domain-specific knowledge is non-trivial for obtaining generic representations for few-shot learning, it is still limited to audio classification tasks. In the future, we will explore how to use audio-language models in diverse audio domains.

翻译：我们提出了Treff适配器，这是一种面向CLAP（对比语言-音频预训练模型）的高效训练适配器，通过利用少量标注数据来提升零样本分类性能。具体而言，我们设计了CALM机制，利用音频-标签配对集合检索文本-音频片段在各类别上的概率分布，并将其与CLAP的零样本分类结果进行融合。此外，我们通过将CALM作为余弦相似度度量，设计了一款无需训练的Treff适配器版本。实验表明，在低样本与数据充足场景下，所提出的Treff适配器性能可与全监督方法及适配方法相媲美甚至更优。尽管Treff适配器表明大规模预训练与领域特定知识的快速学习在获取少样本学习通用表征方面具有非平凡性，但其仍局限于音频分类任务。未来我们将探索如何将音频-语言模型应用于多样化的音频领域。