Speech anonymization is commonly evaluated using averagecase metrics such as the equal error rate, which can hide large disparities in re-identification risks across individuals. In this paper, we conduct a large-scale per-speaker privacy analysis using a linkability-based metric under a worst-case scenario. Nearly 5,000 speakers are evaluated across multiple anonymization systems, attacker architectures, and conversation lengths. While linkability scores are highly polarized at the speaker level, the sets of easy to re-identify and hard to re-identify speakers vary substantially across configurations. We show that no single factor explains speaker vulnerability. Instead, the re-identification risk emerges from the interaction between the attacker, the anonymizer, and the amount of available speech. These results challenge the notion of intrinsic speaker-level privacy risks and emphasize the need for evaluation protocols that are explicitly conditioned on the attacker and anonymizer.

翻译：语音匿名化通常使用等错误率等平均指标进行评估，但这可能掩盖不同个体间重识别风险的巨大差异。本文采用基于可链接性的指标，在最坏场景下对近5,000名说话人进行了大规模逐说话人隐私分析，涵盖多种匿名化系统、攻击者架构及对话长度。研究发现：说话人层面的可链接性得分呈现高度两极分化，但容易重识别与难以重识别的说话人集合在不同配置下差异显著。我们证明说话人脆弱性并非由单一因素决定，而是源于攻击者、匿名化系统与可用语音数据量三者之间的交互作用。这些结果挑战了"说话人固有隐私风险"这一概念，并强调评估协议必须明确针对攻击者和匿名化系统进行条件化设计。