Passive acoustic monitoring enables large-scale biodiversity assessment, but reliable classification of bioacoustic sounds requires not only high accuracy but also well-calibrated uncertainty estimates to ground decision-making. In bioacoustics, calibration is challenged by overlapping vocalisations, long-tailed species distributions, and distribution shifts between training and deployment data. The calibration of multi-label deep learning classifiers within the domain of bioacoustics has not yet been assessed. We systematically benchmark the calibration of four state-of-the-art multi-label bird sound classifiers on the BirdSet benchmark, evaluating both global, per-dataset and per-class calibration using threshold-free calibration metrics (ECE, MCS) alongside discrimination metrics (cmAP). Model calibration varies significantly across datasets and classes. While Perch v2 and ConvNeXt$_{BS}$ show better global calibration, results vary between datasets. Both models indicate consistent underconfidence, while AudioProtoPNet and BirdMAE are mostly overconfident. Surprisingly, calibration seems to be better for less frequent classes. Using simple post hoc calibration methods we demonstrate a straightforward way to improve calibration. A small labelled calibration set is sufficient to significantly improve calibration with Platt scaling, while global calibration parameters suffer from dataset variability. Our findings highlight the importance of evaluating and improving uncertainty calibration in bioacoustic classifiers.

翻译：被动声学监测能够实现大规模生物多样性评估，但生物声学声音的可靠分类不仅需要高精度，还需要经过良好校准的不确定性估计以支撑决策。在生物声学领域，校准面临重叠发声、长尾物种分布以及训练与部署数据间分布偏移等挑战。目前尚未对生物声学领域内多标签深度学习分类器的校准性能进行评估。我们在BirdSet基准上系统性地评估了四种先进多标签鸟类声音分类器的校准性能，使用无需阈值的校准指标（ECE、MCS）与区分度指标（cmAP）共同评估了全局、每数据集及每类别的校准表现。模型校准在不同数据集和类别间存在显著差异。虽然Perch v2和ConvNeXt$_{BS}$展现出更优的全局校准性能，但其结果在不同数据集间有所波动。这两个模型均呈现一致性的欠自信状态，而AudioProtoPNet和BirdMAE则主要表现为过度自信。值得注意的是，校准效果在低频类别中反而更好。通过应用简单的后处理校准方法，我们展示了一种直接改进校准性能的途径。使用少量带标签的校准集配合Platt缩放即可显著提升校准效果，而全局校准参数则受数据集变异性影响。我们的研究结果强调了评估和改进生物声学分类器不确定性校准的重要性。