Previous work has established that a person's demographics and speech style affect how well speech processing models perform for them. But where does this bias come from? In this work, we present the Speech Embedding Association Test (SpEAT), a method for detecting bias in one type of model used for many speech tasks: pre-trained models. The SpEAT is inspired by word embedding association tests in natural language processing, which quantify intrinsic bias in a model's representations of different concepts, such as race or valence (something's pleasantness or unpleasantness) and capture the extent to which a model trained on large-scale socio-cultural data has learned human-like biases. Using the SpEAT, we test for six types of bias in 16 English speech models (including 4 models also trained on multilingual data), which come from the wav2vec 2.0, HuBERT, WavLM, and Whisper model families. We find that 14 or more models reveal positive valence (pleasantness) associations with abled people over disabled people, with European-Americans over African-Americans, with females over males, with U.S. accented speakers over non-U.S. accented speakers, and with younger people over older people. Beyond establishing that pre-trained speech models contain these biases, we also show that they can have real world effects. We compare biases found in pre-trained models to biases in downstream models adapted to the task of Speech Emotion Recognition (SER) and find that in 66 of the 96 tests performed (69%), the group that is more associated with positive valence as indicated by the SpEAT also tends to be predicted as speaking with higher valence by the downstream model. Our work provides evidence that, like text and image-based models, pre-trained speech based-models frequently learn human-like biases. Our work also shows that bias found in pre-trained models can propagate to the downstream task of SER.

翻译：先前研究表明，说话人的社会人口属性及言语风格会影响语音处理模型的性能表现。然而这种偏差源于何处？本文提出语音嵌入关联测试（SpEAT）方法，用于检测常用于多种语音任务的预训练模型中的偏差。SpEAT受自然语言处理中词嵌入关联测试启发，后者可量化模型在种族、效价（事物愉悦度/非愉悦度）等不同概念表征中的内在偏见，并捕捉基于大规模社会文化数据训练的模型习得人类类似偏见的程度。通过SpEAT，我们检测了16个英语语音模型（含4个同时基于多语言训练的模型）中的六类偏差，这些模型分别来自wav2vec 2.0、HuBERT、WavLM和Whisper模型族。研究发现：14个及以上模型在能力健全者vs残障人士、欧裔美国人vs非裔美国人、女性vs男性、美国口音vs非美国口音、年轻人vs老年人之间展现出正向效价（愉悦度）关联。除证实预训练语音模型含有这些偏见外，我们还揭示其可能产生的现实影响。通过对比预训练模型偏见与下游语音情感识别（SER）任务适配模型的偏见，发现在96项检测中有66项（69%）表明：SpEAT检测显示与正向效价关联更强的人群，其下游模型预测的语音效价值也更高。本研究证明，类似基于文本与图像的模型，基于语音的预训练模型常习得人类类似的偏见，且这些偏见会传播至下游的SER任务。