This paper presents a paradigm that adapts general large-scale pretrained models (PTMs) to speech emotion recognition task. Although PTMs shed new light on artificial general intelligence, they are constructed with general tasks in mind, and thus, their efficacy for specific tasks can be further improved. Additionally, employing PTMs in practical applications can be challenging due to their considerable size. Above limitations spawn another research direction, namely, optimizing large-scale PTMs for specific tasks to generate task-specific PTMs that are both compact and effective. In this paper, we focus on the speech emotion recognition task and propose an improved emotion-specific pretrained encoder called Vesper. Vesper is pretrained on a speech dataset based on WavLM and takes into account emotional characteristics. To enhance sensitivity to emotional information, Vesper employs an emotion-guided masking strategy to identify the regions that need masking. Subsequently, Vesper employs hierarchical and cross-layer self-supervision to improve its ability to capture acoustic and semantic representations, both of which are crucial for emotion recognition. Experimental results on the IEMOCAP, MELD, and CREMA-D datasets demonstrate that Vesper with 4 layers outperforms WavLM Base with 12 layers, and the performance of Vesper with 12 layers surpasses that of WavLM Large with 24 layers.

翻译：本文提出了一种将通用大规模预训练模型适配于语音情感识别任务的范式。尽管预训练模型为通用人工智能开辟了新视角，但其构建初衷面向通用任务，因此在特定任务上的效能仍有提升空间。此外，由于预训练模型规模庞大，将其部署至实际应用场景面临挑战。上述局限性催生了另一研究方向：针对特定任务优化大规模预训练模型，以生成兼具紧凑性与高效性的任务专用预训练模型。本文聚焦语音情感识别任务，提出一种改进的情感专用预训练编码器Vesper。Vesper基于WavLM在语音数据集上预训练，并充分考虑情感特征。为增强对情感信息的敏感性，Vesper采用情感引导掩码策略识别需要掩码的区域；进而通过层级与跨层自监督机制提升其捕获声学表征与语义表征的能力——二者对情感识别均至关重要。在IEMOCAP、MELD和CREMA-D数据集上的实验结果表明，4层结构的Vesper性能优于12层WavLM Base，而12层Vesper的表现更超越24层WavLM Large。