In Speech Emotion Recognition (SER), textual data is often used alongside audio signals to address their inherent variability. However, the reliance on human annotated text in most research hinders the development of practical SER systems. To overcome this challenge, we investigate how Automatic Speech Recognition (ASR) performs on emotional speech by analyzing the ASR performance on emotion corpora and examining the distribution of word errors and confidence scores in ASR transcripts to gain insight into how emotion affects ASR. We utilize four ASR systems, namely Kaldi ASR, wav2vec2, Conformer, and Whisper, and three corpora: IEMOCAP, MOSI, and MELD to ensure generalizability. Additionally, we conduct text-based SER on ASR transcripts with increasing word error rates to investigate how ASR affects SER. The objective of this study is to uncover the relationship and mutual impact of ASR and SER, in order to facilitate ASR adaptation to emotional speech and the use of SER in real world.

翻译：在语音情感识别（SER）中，文本数据常与音频信号结合使用，以应对其固有的变异性。然而，多数研究对人机标注文本的依赖阻碍了实用化SER系统的发展。为克服这一挑战，我们通过分析自动语音识别（ASR）在情感语料库上的性能表现，并考察ASR转录文本中单词错误和置信度得分的分布规律，探究情感如何影响ASR。我们采用四种ASR系统（Kaldi ASR、wav2vec2、Conformer和Whisper）以及三个语料库（IEMOCAP、MOSI和MELD）以确保结果的泛化性。此外，我们基于具有递增单词错误率的ASR转录文本开展文本级SER研究，以揭示ASR对SER的影响机制。本研究旨在揭示ASR与SER之间的关联及相互影响，从而促进ASR对情感语音的适应性改进，并推动SER在真实场景中的应用。