In this paper, we explored how to boost speech emotion recognition (SER) with the state-of-the-art speech pre-trained model (PTM), data2vec, text generation technique, GPT-4, and speech synthesis technique, Azure TTS. First, we investigated the representation ability of different speech self-supervised pre-trained models, and we found that data2vec has a good representation ability on the SER task. Second, we employed a powerful large language model (LLM), GPT-4, and emotional text-to-speech (TTS) model, Azure TTS, to generate emotionally congruent text and speech. We carefully designed the text prompt and dataset construction, to obtain the synthetic emotional speech data with high quality. Third, we studied different ways of data augmentation to promote the SER task with synthetic speech, including random mixing, adversarial training, transfer learning, and curriculum learning. Experiments and ablation studies on the IEMOCAP dataset demonstrate the effectiveness of our method, compared with other data augmentation methods, and data augmentation with other synthetic data.

翻译：本文探索了如何利用最先进的语音预训练模型（PTM）data2vec、文本生成技术GPT-4以及语音合成技术Azure TTS来提升语音情感识别（SER）性能。首先，我们研究了不同语音自监督预训练模型的表示能力，发现data2vec在SER任务中具有优秀的表示能力。其次，我们采用强大的大语言模型（LLM）GPT-4和情感文本转语音（TTS）模型Azure TTS来生成情感一致的文本和语音。通过精心设计文本提示词和数据集构建方式，获得了高质量的合成情感语音数据。第三，我们研究了多种数据增强方法，包括随机混合、对抗训练、迁移学习和课程学习，以利用合成语音促进SER任务。在IEMOCAP数据集上的实验和消融研究证明，与其它数据增强方法和使用其他合成数据的数据增强相比，我们的方法更具有效性。