Large self-supervised pre-trained speech models have achieved remarkable success across various speech-processing tasks. The self-supervised training of these models leads to universal speech representations that can be used for different downstream tasks, ranging from automatic speech recognition (ASR) to speaker identification. Recently, Whisper, a transformer-based model was proposed and trained on large amount of weakly supervised data for ASR; it outperformed several state-of-the-art self-supervised models. Given the superiority of Whisper for ASR, in this paper we explore the transferability of the representation for four other speech tasks in SUPERB benchmark. Moreover, we explore the robustness of Whisper representation for ``in the wild'' tasks where speech is corrupted by environment noise and room reverberation. Experimental results show Whisper achieves promising results across tasks and environmental conditions, thus showing potential for cross-task real-world deployment.

翻译：大型自监督预训练语音模型在各种语音处理任务中已取得显著成功。这些模型的自监督训练产生了可应用于不同下游任务的通用语音表示，涵盖从自动语音识别（ASR）到说话人识别等任务。近期，基于Transformer的Whisper模型被提出，并在大量弱监督ASR数据上进行了训练；其性能超越了多个最先进的自监督模型。鉴于Whisper在ASR中的优越性，本文探索了其表示在SUPERB基准中其他四种语音任务上的可迁移性。此外，我们还研究了Whisper表示在语音受环境噪声和房间混响污染的“野外”任务中的鲁棒性。实验结果表明，Whisper在多种任务和环境条件下均取得了令人满意的结果，展现了跨任务实际部署的潜力。