Contrastive Language Audio Pretraining (CLAP) is a widely-used method to bridge the gap between audio and text domains. Current CLAP methods enable sound and music retrieval in English, ignoring multilingual spoken content. To address this, we introduce general language audio pretraining (GLAP), which expands CLAP with multilingual and multi-domain abilities. GLAP demonstrates its versatility by achieving competitive performance on standard audio-text retrieval benchmarks like Clotho and AudioCaps, while significantly surpassing existing methods in speech retrieval and classification tasks. Additionally, GLAP achieves strong results on widely used sound-event zero-shot benchmarks, while simultaneously outperforming previous methods on speech content benchmarks. Further keyword spotting evaluations across 50 languages emphasize GLAP's advanced multilingual capabilities. Finally, multilingual sound and music understanding is evaluated across four languages. Checkpoints and Source: https://github.com/xiaomi-research/dasheng-glap.

翻译：对比语言音频预训练（CLAP）是一种广泛用于弥合音频与文本领域间差距的方法。现有CLAP方法仅支持英语环境的声音与音乐检索，忽略了多语言语音内容。为此，我们提出通用语言音频预训练（GLAP），通过扩展多语言与多领域能力来增强CLAP框架。GLAP在Clotho、AudioCaps等标准音频-文本检索基准上取得了具有竞争力的性能，同时在语音检索与分类任务中显著超越现有方法，展现了其通用性。此外，GLAP在广泛使用的声音事件零样本基准上获得优异结果，并在语音内容基准测试中超越先前方法。进一步在50种语言上开展的关键词唤醒评估凸显了GLAP先进的多语言能力。最后，我们在四种语言中评估了多语言声音与音乐理解性能。模型检查点与源代码：https://github.com/xiaomi-research/dasheng-glap。