Recent advances in using language models to obtain cross-modal audio-text representations have overcome the limitations of conventional training approaches that use predefined labels. This has allowed the community to make progress in tasks like zero-shot classification, which would otherwise not be possible. However, learning such representations requires a large amount of human-annotated audio-text pairs. In this paper, we study unsupervised approaches to improve the learning framework of such representations with unpaired text and audio. We explore domain-unspecific and domain-specific curation methods to create audio-text pairs that we use to further improve the model. We also show that when domain-specific curation is used in conjunction with a soft-labeled contrastive loss, we are able to obtain significant improvement in terms of zero-shot classification performance on downstream sound event classification or acoustic scene classification tasks.

翻译：利用语言模型获取跨模态音频-文本表征的最新进展，克服了传统使用预定义标签的训练方法的局限性。这使得研究社区能够在零样本分类等任务上取得突破，而这些任务在以往是无法实现的。然而，学习此类表征需要大量人工标注的音频-文本配对数据。本文研究了利用未配对文本和音频改进此类表征学习框架的无监督方法。我们探索了领域非特定和领域特定的数据筛选方法，以创建用于进一步优化模型的音频-文本配对数据。我们还证明，当将领域特定筛选与软标签对比损失相结合时，能够在下游声音事件分类或声学场景分类任务的零样本分类性能上获得显著提升。