Due to the high variation in the application requirements of sound event detection (SED) systems, it is not sufficient to evaluate systems only in a single operating mode. Therefore, the community recently adopted the polyphonic sound detection score (PSDS) as an evaluation metric, which is the normalized area under the PSD receiver operating characteristic (PSD-ROC). It summarizes the system performance over a range of operating modes resulting from varying the decision threshold that is used to translate the system output scores into a binary detection output. Hence, it provides a more complete picture of the overall system behavior and is less biased by specific threshold tuning. However, besides the decision threshold there is also the post-processing that can be changed to enter another operating mode. In this paper we propose the post-processing independent PSDS (piPSDS) as a generalization of the PSDS. Here, the post-processing independent PSD-ROC includes operating points from varying post-processings with varying decision thresholds. Thus, it summarizes even more operating modes of an SED system and allows for system comparison without the need of implementing a post-processing and without a bias due to different post-processings. While piPSDS can in principle combine different types of post-processing, we hear, as a first step, present median filter independent PSDS (miPSDS) results for this year's DCASE Challenge Task4a systems. Source code is publicly available in our sed_scores_eval package (https://github.com/fgnt/sed_scores_eval).

翻译：由于声事件检测（SED）系统的应用需求存在高度差异性，仅在单一工作模式下进行系统评估是不够的。为此，学界近期采用了多声道声音检测评分（PSDS）作为评估指标，该指标是PSD接收者操作特征曲线（PSD-ROC）的归一化面积。它通过汇总将系统输出分数转换为二元检测输出时，因决策阈值变化而产生的多种工作模式下的系统性能，从而更全面地反映系统整体行为，并减少特定阈值调整带来的偏差。然而，除了决策阈值外，后处理同样可以调整以进入不同工作模式。本文提出后处理无关PSDS（piPSDS）作为PSDS的推广形式。在此框架下，后处理无关的PSD-ROC包含不同后处理方式配合不同决策阈值产生的工作点，从而归纳了SED系统更多的工作模式，使得无需实现具体后处理、排除不同后处理造成的偏差即可进行系统比较。尽管piPSDS原则上可融合多种后处理类型，作为初步工作，我们首先在本年度DCASE挑战赛Task4a系统中展示了中值滤波无关PSDS（miPSDS）结果。相关源代码已开源至sed_scores_eval工具包（https://github.com/fgnt/sed_scores_eval）。