Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups. But what about toxicity detection in speech? To investigate the extent to which text-based biases are mitigated by speech-based systems, we produce a set of high-quality group annotations for the multilingual MuTox dataset, and then leverage these annotations to systematically compare speech- and text-based toxicity classifiers. Our findings indicate that access to speech data during inference supports reduced bias against group mentions, particularly for ambiguous and disagreement-inducing samples. Our results also suggest that improving classifiers, rather than transcription pipelines, is more helpful for reducing group bias. We publicly release our annotations and provide recommendations for future toxicity dataset construction.

翻译：文本毒性检测系统存在显著偏见，在提及人口群体的样本上产生不成比例的误报率。但语音毒性检测的情况如何？为探究基于语音的系统能在多大程度上缓解基于文本的偏见，我们为多语言MuTox数据集构建了一套高质量群体标注，并利用这些标注系统比较了基于语音和基于文本的毒性分类器。研究发现，在推理过程中使用语音数据有助于降低针对群体提及的偏见，特别是对于模糊且易引发分歧的样本。结果还表明，改进分类器比优化转录流程更能有效减少群体偏见。我们公开了标注数据，并为未来毒性数据集的构建提供了建议。