Deep learning (DL) models have emerged as a powerful tool in avian bioacoustics to diagnose environmental health and biodiversity. However, inconsistencies in research pose notable challenges hindering progress. Reliable DL models need to analyze bird calls flexibly across various species and environments to fully harness the potential of bioacoustics in a cost-effective passive acoustic monitoring scenario. Data fragmentation and opacity across studies complicate a comprehensive evaluation of model performance. To overcome these challenges, we present the BirdSet benchmark, a unified framework consolidating research efforts with a holistic approach for the classification of bird vocalizations in computational avian bioacoustics. BirdSet aggregates open-source bird recordings into a curated dataset collection. This unified approach provides an in-depth understanding of model performance and identifies potential shortcomings across different tasks. By providing baseline results of current models, we aim to facilitate comparability and ease accessibility for newcomers. Additionally, we release an open-source package \benchmark containing a comprehensive data pipeline that enables easy and fast model evaluation, available at https://github.com/DBD-research-group/BirdSet.

翻译：深度学习模型已成为鸟类生物声学中诊断环境健康与生物多样性的有力工具。然而，研究中的不一致性构成了阻碍进展的显著挑战。为了在低成本被动声学监测场景中充分发挥生物声学的潜力，可靠的深度学习模型需要能够灵活分析跨物种与跨环境的鸟类叫声。研究中的数据碎片化与透明度不足使得模型性能的全面评估变得困难。为克服这些挑战，我们提出BirdSet基准——一个以整体方法整合研究工作的统一框架，专注于计算鸟类生物声学中鸟类发声的分类任务。BirdSet将开放获取的鸟类录音聚合为精选数据集集合。这种统一方法能深入理解模型性能，并识别不同任务中的潜在缺陷。通过提供当前模型的基线结果，我们旨在促进可比较性并降低新手入门门槛。此外，我们发布了包含完整数据流水线的开源软件包\benchmark，支持便捷快速的模型评估，代码地址为https://github.com/DBD-research-group/BirdSet。