Passive acoustic monitoring offers a scalable, non-invasive method for tracking global biodiversity and anthropogenic impacts on species. Although deep learning has become a vital tool for processing this data, current models are inflexible, typically cover only a handful of species, and are limited by data scarcity. In this work, we propose BioLingual, a new model for bioacoustics based on contrastive language-audio pretraining. We first aggregate bioacoustic archives into a language-audio dataset, called AnimalSpeak, with over a million audio-caption pairs holding information on species, vocalization context, and animal behavior. After training on this dataset to connect language and audio representations, our model can identify over a thousand species' calls across taxa, complete bioacoustic tasks zero-shot, and retrieve animal vocalization recordings from natural text queries. When fine-tuned, BioLingual sets a new state-of-the-art on nine tasks in the Benchmark of Animal Sounds. Given its broad taxa coverage and ability to be flexibly queried in human language, we believe this model opens new paradigms in ecological monitoring and research, including free-text search on the world's acoustic monitoring archives. We open-source our models, dataset, and code.

翻译：被动声学监测为追踪全球生物多样性及人类活动对物种的影响提供了一种可扩展、非侵入性的方法。尽管深度学习已成为处理此类数据的重要工具，但现有模型缺乏灵活性，通常仅覆盖少数物种，且受限于数据稀缺性。本研究提出BioLingual——一种基于对比语言-音频预训练的新型生物声学模型。我们首先将生物声学档案整合为名为AnimalSpeak的语言-音频数据集，该数据集包含超过百万条音频-文本对，涵盖物种信息、发声情境及动物行为。在该数据集上训练模型以连接语言与音频表征后，我们的模型能够跨分类群识别千余种动物的叫声，以零样本方式完成生物声学任务，并通过自然语言查询检索动物发声录音。经微调后，BioLingual在动物声音基准测试的九个任务中刷新了最优性能。鉴于其广泛的物种覆盖范围及通过人类语言进行灵活查询的能力，该模型有望为生态监测与研究开辟新范式，包括对全球声学监测档案的自由文本检索。我们将模型、数据集及代码全部开源。